Juju Charms Collection
python-django package

Merge lp:~thumper/charms/trusty/python-django/support-1.7 into lp:charms/python-django

  1. Trusty Tahr (14.04)
  2. support-1.7
  3. Merge into trunk
Proposed by Tim Penhey
Status: Merged
Merged at revision: 40
Proposed branch: lp:~thumper/charms/trusty/python-django/support-1.7
Merge into: lp:charms/python-django
Prerequisite: lp:~thumper/charms/trusty/python-django/clean-contrib
Diff against target: 1473 lines (+686/-471)
11 files modified
.bzrignore (+1/-0)
Makefile (+22/-6)
hooks/hooks.py (+442/-333)
hooks/tests/test_template.py (+0/-125)
hooks/tests/test_unit.py (+128/-0)
templates/pgsql_engine.tmpl (+1/-1)
tests/10-mysql (+2/-3)
tests/10-postgresql (+2/-3)
tests/11-django-1.8-with-postgresql (+57/-0)
tests/config/django.yaml (+21/-0)
tests/tests.yaml (+10/-0)
To merge this branch: bzr merge lp:~thumper/charms/trusty/python-django/support-1.7
Reviewer Review Type Date Requested Status
Charles Butler (community) Approve
Whit Morriss (community) Needs Fixing
Tim Van Steenburgh Pending
Review via email: mp+260389@code.launchpad.net

This proposal supersedes a proposal from 2015-05-23.

Description of the change

Well, this fixes the last merge I attempted. It was very broken.

In order to make sure I really didn't break it, I added some unit tests, which meant refactoring the hooks module so it had no global state. What I thought would take about and hour at the most became an eight hour marathon.

'make check' now runs the hook unit tests.

Two of the files in the hooks/tests directory were direct copies from gunicorn. I removed one and renamed the other so it doesn't get in the way.

There is more cleanup to do, but I left some obvious things to fix as markers so the diff will show I didn't replace *everything*.

I have also tested manual deployment, and it works with postgresql, gunicorn and the django_settings relations.

To post a comment you must log in.
Revision history for this message
Whit Morriss (whitmo) wrote :

Hi Tim!

I'm guessing by look at the tests that maybe this is classified wrong in the queue ie this is a work in progress. On running the tests as below, I'm seeing test setup errors for the db tests and a failure due to flake8 missing.

'''
docker run -ti --rm --net=host \
   -v /home/whit/.juju:/home/ubuntu/.juju \
   -v /home/whit/proj/rq/trusty:/home/ubuntu/trusty charmbox
workon charm-review
bundletester -vFl DEBUG
'''

https://gist.github.com/whitmo/01f5ece4b0e57c5c2a6b

Bundletester is not picking up the unit-tests either (and test_hooks.py is removed from the discovery path). Using a tests.yaml could add "make check" to the tests bt executes (see README https://github.com/juju-solutions/bundletester )

In general the refactor looks good, I might add a simple decorator to encapsulate what is happening in the first few lines of every hook now.

review: Needs Fixing
Revision history for this message
Tim Penhey (thumper) wrote :

I have spent several hours today trying to get a clean environment to run bundle tester in.

Unfortunately the docs on github are not sufficient to get a working environment.

A clear problem I have is that if I add the check target to a tests.yaml file in the tests dir, there is no easy way to install the 'semantic-version' pip package that the charm does in order to run the unit tests.

What is the best way to do this?

It seems that the virtual environment that the bundletester says it will create is not in force when the make targets are executed because I added one to 'pip install semantic-version', but it wasn't available to the 'make check' target. I'm guessing that it was installed in the .local directory for the user, and not in the default python path.

What is the best way to make sure packages are installed? Please note that these are python packages from pypi, not ubuntu deb packages.

Also worth noting that the deployer requires a .jenv file, and this is going away soon. I have added a card to my team's kanban board to add this.

Also worth noting that the bundletester code doesn't work with bzr shared repositories.

Another thing is that the tests currently in python-django are starting to fail due to changes in juju where we no longer reuse unit names even if the service is destroyed. I'm not sure how many other charms will run up against this problem.

I'll check in with someone later to determine how best to add the python package dependencies.

Revision history for this message
Tim Penhey (thumper) wrote :

OK, running here against 1.24-beta7 has it passing:

python-django
    charm-proof PASS
    make virtualenv PASS
    make lint PASS
    make check PASS
    00-setup PASS
    01-dj13 PASS
    01-dj14 PASS
    01-djdistro PASS
    10-mysql PASS
    10-postgresql PASS
    11-django-1.8-with-postgresql PASS

Revision history for this message
Charles Butler (lazypower) wrote :

Greetings Thumper,

Thanks for the changes here. I've given this branch a review, and everything seems to be in order here. I've gone ahead and merged the branch.

Thanks for your effort on this, we really appreciate it.

If you have any questions/comments/concerns about the review contact us in #juju on irc.freenode.net or email the mailing list <email address hidden>, or ask a question tagged with "juju" on http://askubuntu.com.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.bzrignore'
2--- .bzrignore 2014-11-19 17:34:34 +0000
3+++ .bzrignore 2015-06-06 04:57:38 +0000
4@@ -1,3 +1,4 @@
5+__pycache__
6 *~
7 *.tmp
8 *.py[co]
9
10=== modified file 'Makefile'
11--- Makefile 2014-09-26 21:30:42 +0000
12+++ Makefile 2015-06-06 04:57:38 +0000
13@@ -1,7 +1,7 @@
14 #!/usr/bin/make
15 PYTHON := /usr/bin/env python
16
17-#test: lint integration-test
18+build: virtualenv lint check
19
20 sync-charm-helpers: bin/charm_helpers_sync.py
21 @mkdir -p bin
22@@ -10,11 +10,6 @@
23 bin/charm_helpers_sync.py:
24 @bzr cat lp:charm-helpers/tools/charm_helpers_sync/charm_helpers_sync.py > bin/charm_helpers_sync.py
25
26-lint:
27- @echo "Lint check (flake8)"
28- @flake8 -v --ignore E501 --exclude hooks/charmhelpers hooks
29- @charm-proof .
30-
31 verify-juju-test:
32 @echo "Checking for ... "
33 @echo -n "juju-test: "
34@@ -28,3 +23,24 @@
35 integration-test:
36 juju test --set-e -p SKIP_SLOW_TESTS,DEPLOYER_TARGET,JUJU_HOME,JUJU_ENV -v --timeout 3000s
37
38+virtualenv: .venv/bin/python
39+.venv/bin/python: make-virtual-env
40+.venv/bin/flake8: make-virtual-env
41+make-virtual-env:
42+ sudo apt-get install -y python-virtualenv
43+ virtualenv .venv
44+ .venv/bin/pip install nose flake8 mock semantic-version pyyaml charmhelpers # charm-tools
45+
46+lint: .venv/bin/flake8
47+ @.venv/bin/flake8 -v --ignore E501 --exclude hooks/charmhelpers hooks
48+ @charm proof
49+
50+check: .venv/bin/python
51+ @echo Starting tests...
52+ @CHARM_DIR=. PYTHONPATH=./hooks:./tests/fakepython .venv/bin/nosetests -v --nologcapture --ignore-files=hooks/charmhelpers.* hooks
53+
54+clean:
55+ rm -rf .venv
56+ find -name *.pyc -delete
57+
58+.PHONY: check clean lint virtualenv integration-test verify-juju-test build
59
60=== modified file 'hooks/hooks.py'
61--- hooks/hooks.py 2015-05-22 05:37:28 +0000
62+++ hooks/hooks.py 2015-06-06 04:57:38 +0000
63@@ -41,7 +41,7 @@
64 hooks = Hooks()
65
66 CHARM_DEB_PACKAGES = ["python-pip", "python-jinja2", "mercurial", "git-core",
67- "subversion", "bzr", "gettext"]
68+ "subversion", "bzr", "gettext"]
69
70 CHARM_PIP_PACKAGES = ["semantic_version"]
71
72@@ -85,7 +85,7 @@
73 cmd_line.append('-e')
74 cmd_line.append(package)
75
76- cmd_line.append('--use-mirrors')
77+ log('%s' % cmd_line, DEBUG)
78 return(subprocess.call(cmd_line))
79
80
81@@ -101,7 +101,6 @@
82 cmd_line.append('--upgrade')
83 cmd_line.append('-r')
84 cmd_line.append(path_or_url)
85- cmd_line.append('--use-mirrors')
86 return(subprocess.call(cmd_line))
87
88
89@@ -209,10 +208,11 @@
90 for cmd in ['django-admin.py', 'django-admin']:
91 django_admin_cmd = which(cmd)
92 if django_admin_cmd:
93- p = 'PYTHONPATH=%s' % python_path
94- return '%s %s' % (p, django_admin_cmd)
95+ log('found django admin: %s' % django_admin_cmd, DEBUG)
96+ return django_admin_cmd
97
98 log("No django-admin executable found.", ERROR)
99+ sys.exit(1)
100
101
102 def append_template(template_name, template_vars, path, try_append=False):
103@@ -240,33 +240,38 @@
104 inject_file.write(str(template))
105
106
107-def update_database_schema(exit_on_error=True, swallow_exceptions=False):
108+def get_django_version(django_admin_cmd):
109+ return subprocess.check_output([django_admin_cmd, '--version'])
110+
111+
112+def update_database_schema(unit, exit_on_error=True, swallow_exceptions=False):
113 """Run the appropriate syncdb/migrate calls.
114
115 This will make sure that the database reflects the currently installed
116 applications.
117 """
118- django_admin_cmd = find_django_admin_cmd()
119+ django_migrate = unit.has_modern_django()
120
121- django_version = semantic_version.Version(
122- subprocess.check_output([django_admin_cmd, '--version']),
123- partial=True)
124- django_migrate = django_version >= semantic_version.Version('1.7.0')
125+ # From here we want the fully qualified admin command with the python path
126+ django_admin_cmd = unit.django_admin_cmd_with_path()
127
128 if not django_migrate:
129 try:
130 run("%s syncdb --noinput --settings=%s" %
131- (django_admin_cmd, settings_module),
132- exit_on_error=exit_on_error, cwd=working_dir)
133+ (django_admin_cmd, unit.settings_module),
134+ exit_on_error=exit_on_error, cwd=unit.working_dir)
135 except:
136 if not swallow_exceptions:
137 raise
138
139- if django_migrate or django_south:
140+ if django_migrate or unit.config('django_south'):
141 try:
142- run("%s migrate --settings=%s" %
143- (django_admin_cmd, settings_module),
144- exit_on_error=exit_on_error, cwd=working_dir)
145+ extra_arg = ''
146+ if django_migrate:
147+ extra_arg = '--noinput'
148+ run("%s migrate --settings=%s %s" %
149+ (django_admin_cmd, unit.settings_module, extra_arg),
150+ exit_on_error=exit_on_error, cwd=unit.working_dir)
151 except:
152 if not swallow_exceptions:
153 raise
154@@ -276,188 +281,158 @@
155 # Hook functions
156 ###############################################################################
157 @hooks.hook()
158-def install():
159+def install(unit=None):
160+ if unit is None:
161+ unit = Unit(config())
162+
163 log("Installing {}".format(service_name()))
164+ log('%s' % unit.data, DEBUG)
165 apt_update()
166
167- deb_packages = list(CHARM_DEB_PACKAGES)
168- if extra_deb_pkgs:
169- deb_packages.extend(extra_deb_pkgs.split(','))
170-
171- for retry in range(0, 24):
172- try:
173- apt_install(deb_packages)
174- except subprocess.CalledProcessError as e:
175- log("Error ({}) running {}. Output: {}".format(
176- e.returncode, e.cmd, e.output))
177- time.sleep(10)
178- continue
179-
180- break
181-
182- pip_packages = list(CHARM_PIP_PACKAGES)
183- if extra_pip_pkgs:
184- pip_packages.extend(extra_pip_pkgs.split(','))
185- for package in pip_packages:
186- pip_install(package, upgrade=True)
187-
188- configure_and_install(config_data['django_version'])
189-
190- if django_south:
191- configure_and_install(django_south_version, distro="python-django-south", pip='South')
192-
193+ unit.install_deb_packages()
194+ unit.install_pip_packages()
195+
196+ # There is bug in 14.04 with new requests package which needs an updated pip.
197+ # In order to make subsequent pip installs succeed without having to look
198+ # to see if they use a recent requests package, we install the latest pip using
199+ # easy_install. The verion in trusty isn't good enough.
200+ run('sudo easy_install -U pip')
201+
202+ configure_and_install(unit.config('django_version'))
203+
204+ if unit.config('django_south'):
205+ configure_and_install(unit.config('django_south_version'), distro="python-django-south", pip='South')
206+
207+ vcs = unit.config('vcs')
208+ repos_url = unit.config('repos_url')
209+ # repos_username = unit.config('repos_username')
210+ # repos_password = unit.config('repos_password')
211+ repos_branch = unit.config('repos_branch')
212+
213+ vcs_clone_dir = unit.base_dir
214 if vcs == '' and repos_url == '':
215 log("No version control using django-admin startproject")
216- django_admin_cmd = find_django_admin_cmd()
217+ django_admin_cmd = unit.django_admin_cmd_with_path()
218 cmd = '%s startproject' % django_admin_cmd
219- if project_template_url:
220- cmd = " ".join([cmd, '--template', project_template_url])
221- if project_template_extension:
222- cmd = " ".join([cmd, '--extension', project_template_extension])
223+ template_url = unit.config('project_template_url')
224+ extension = unit.config('project_template_extension')
225+ install_root = unit.config('install_root')
226+ if template_url:
227+ cmd = " ".join([cmd, '--template', template_url])
228+ if extension:
229+ cmd = " ".join([cmd, '--extension', extension])
230 try:
231- run('%s %s %s' % (cmd, sanitized_service_name, install_root), exit_on_error=False)
232+ run('%s %s %s' % (cmd, unit.service_name, install_root), exit_on_error=False)
233 except subprocess.CalledProcessError:
234- run('%s %s' % (cmd, sanitized_service_name), cwd=install_root)
235- elif vcs == 'hg' or vcs == 'mercurial':
236+ run('%s %s' % (cmd, unit.service_name), cwd=install_root)
237+ elif vcs in ('hg', 'mercurial'):
238 run('hg clone %s %s' % (repos_url, vcs_clone_dir))
239- elif vcs == 'git' or vcs == 'git-core':
240+ elif vcs in ('git', 'git-core'):
241 if repos_branch:
242 run('git clone %s -b %s %s' % (repos_url, repos_branch, vcs_clone_dir))
243 else:
244 run('git clone %s %s' % (repos_url, vcs_clone_dir))
245- elif vcs == 'bzr' or vcs == 'bazaar':
246+ elif vcs in ('bzr', 'bazaar'):
247 run('bzr branch %s %s' % (repos_url, vcs_clone_dir))
248- elif vcs == 'svn' or vcs == 'subversion':
249+ elif vcs in ('svn', 'subversion'):
250 run('svn co %s %s' % (repos_url, vcs_clone_dir))
251 else:
252 log("Unknown version control", ERROR)
253 sys.exit(1)
254
255- run('chown -R %s:%s %s' % (wsgi_user, wsgi_group, vcs_clone_dir))
256+ unit.update_file_ownership()
257
258- mkdir(settings_dir_path, owner=wsgi_user, group=wsgi_group, perms=0755)
259- mkdir(urls_dir_path, owner=wsgi_user, group=wsgi_group, perms=0755)
260+ unit.mkdir(unit.settings_dir_path)
261+ unit.mkdir(unit.urls_dir_path)
262
263 # FIXME: Upgrades/pulls will mess those files
264
265- append_template('conf_injection.tmpl', {'dir': settings_dir_name}, settings_py_path)
266- append_template('urls_injection.tmpl', {'dir': urls_dir_name}, urls_py_path)
267-
268- if requirements_pip_files:
269- for req_file in requirements_pip_files.split(','):
270- pip_install_req(os.path.join(working_dir, req_file))
271-
272- wsgi_py_path = os.path.join(working_dir, 'wsgi.py')
273+ append_template('conf_injection.tmpl', {'dir': unit.settings_dir_name}, unit.settings_py_path)
274+ append_template('urls_injection.tmpl', {'dir': unit.urls_dir_name}, unit.urls_py_path)
275+
276+ unit.install_requirements_files()
277+
278+ wsgi_py_path = os.path.join(unit.working_dir, 'wsgi.py')
279 if not os.path.exists(wsgi_py_path):
280- process_template('wsgi.py.tmpl', {'project_name': sanitized_service_name,
281- 'django_settings': settings_module},
282+ process_template('wsgi.py.tmpl', {'project_name': unit.service_name,
283+ 'django_settings': unit.settings_module},
284 wsgi_py_path)
285
286
287 @hooks.hook()
288-def start():
289- if os.path.exists(os.path.join('/etc/init/', sanitized_service_name + '.conf')):
290- service_start(sanitized_service_name)
291-
292-
293-@hooks.hook()
294-def stop():
295- if os.path.exists(os.path.join('/etc/init/', sanitized_service_name + '.conf')):
296- service_stop(sanitized_service_name)
297-
298-
299-@hooks.hook()
300-def config_changed():
301- os.environ['DJANGO_SETTINGS_MODULE'] = settings_module
302-
303- site_secret_key = config_data['site_secret_key']
304- if not site_secret_key:
305- site_secret_key = ''.join([choice('abcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*(-_=+)') for i in range(50)])
306-
307- process_template('secret.tmpl', {'site_secret_key': site_secret_key}, settings_secret_path)
308-
309- dst = os.path.join(settings_dir_path, '30-allowed.py')
310- ip = run('unit-get public-address').strip()
311- allowed = [socket.gethostname(), socket.getfqdn(), ip]
312- if 'django_allowed_hosts' in config_data and \
313- config_data['django_allowed_hosts'].strip() != '':
314- allowed = config_data['django_allowed_hosts'].split(' ')
315- process_template('allowed_hosts.tmpl', {'allowed_hosts': allowed}, dst)
316-
317- dst = os.path.join(settings_dir_path, '40-debug.py')
318- debug = config_data.get('django_debug', False)
319- process_template('debug.tmpl', {'debug': debug}, dst)
320-
321- if 'django_extra_settings' in config_data:
322- dst = os.path.join(settings_dir_path, '50-extra-conf.py')
323- pairs = config_data['django_extra_settings'].split(',')
324- process_template('extra-conf.tmpl', {'pairs': pairs}, dst)
325-
326- # Trigger WSGI reloading
327- for relid in relation_ids('wsgi'):
328- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
329-
330-
331-@hooks.hook()
332-def upgrade():
333-
334- apt_update()
335- for retry in range(0, 24):
336- try:
337- apt_install(CHARM_DEB_PACKAGES)
338- except subprocess.CalledProcessError as e:
339- log("Error ({}) running {}. Output: {}".format(
340- e.returncode, e.cmd, e.output))
341- time.sleep(10)
342- continue
343-
344- break
345-
346+def start(unit=None):
347+ if unit is None:
348+ unit = Unit(config())
349+
350+ if os.path.exists(os.path.join('/etc/init/', unit.service_name + '.conf')):
351+ service_start(unit.service_name)
352+
353+
354+@hooks.hook()
355+def stop(unit=None):
356+ if unit is None:
357+ unit = Unit(config())
358+
359+ if os.path.exists(os.path.join('/etc/init/', unit.service_name + '.conf')):
360+ service_stop(unit.service_name)
361+
362+
363+@hooks.hook()
364+def config_changed(unit=None):
365+ if unit is None:
366+ unit = Unit(config())
367+
368+ # TODO: update Django?
369+ unit.update_settings()
370+ unit.reload_wsgi()
371+
372+
373+@hooks.hook('upgrade-charm')
374+def upgrade(unit=None):
375+ if unit is None:
376+ unit = Unit(config())
377+
378+ # There is bug in 14.04 with new requests package which needs an updated pip.
379+ # In order to make subsequent pip installs succeed without having to look
380+ # to see if they use a recent requests package, we install the latest pip using
381+ # easy_install. The verion in trusty isn't good enough.
382+ run('sudo easy_install -U pip')
383+
384+ # NOTE: worth noting that although the readme says that the updrade hook
385+ # also updates django, this does not appear to be the case.
386+ unit.install_deb_packages()
387+ unit.install_pip_packages()
388 # FIXME: pull new code ?
389-
390- pip_packages = list(CHARM_PIP_PACKAGES)
391- if extra_pip_pkgs:
392- pip_packages.extend(extra_pip_pkgs.split(','))
393- for package in pip_packages:
394- pip_install(package, upgrade=True)
395-
396- if requirements_pip_files:
397- for req_file in requirements_pip_files.split(','):
398- pip_install_req(os.path.join(working_dir, req_file), upgrade=True)
399-
400- run('chown -R %s:%s %s' % (wsgi_user, wsgi_group, vcs_clone_dir))
401-
402- # Trigger WSGI reloading
403- for relid in relation_ids('wsgi'):
404- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
405+ unit.install_requirements_files(upgrade=True)
406+ unit.update_file_ownership()
407+
408+ unit.reload_wsgi()
409
410 for relid in relation_ids('django-settings'):
411 relation_set(relation_settings={'django_settings_timestamp': time.time()}, relation_id=relid)
412
413
414 @hooks.hook('django-settings-relation-joined', 'django-settings-relation-changed')
415-def django_settings_relation_joined_changed():
416- os.environ['DJANGO_SETTINGS_MODULE'] = settings_module
417- django_admin_cmd = find_django_admin_cmd()
418+def django_settings_relation_joined_changed(unit=None):
419+ if unit is None:
420+ unit = Unit(config())
421
422 relation_set(relation_settings={
423- 'settings_dir_path': settings_dir_path,
424- 'settings_module': settings_module,
425- 'urls_dir_path': urls_dir_path,
426- 'working_dir': working_dir,
427- 'django_admin_cmd': django_admin_cmd,
428- 'wsgi_user': wsgi_user,
429- 'wsgi_group': wsgi_group,
430+ 'settings_dir_path': unit.settings_dir_path,
431+ 'settings_module': unit.settings_module,
432+ 'urls_dir_path': unit.urls_dir_path,
433+ 'working_dir': unit.working_dir,
434+ 'django_admin_cmd': unit.django_admin_cmd_with_path(),
435+ 'wsgi_user': unit.config('wsgi_user'),
436+ 'wsgi_group': unit.config('wsgi_group'),
437 })
438
439- run('chown -R %s:%s %s' % (wsgi_user, wsgi_group, vcs_clone_dir))
440+ unit.update_file_ownership()
441
442 # It isn't entirely clear to me why we don't want to exit on an error here.
443- update_database_schema(exit_on_error=False, swallow_exceptions=True)
444+ update_database_schema(unit, exit_on_error=False, swallow_exceptions=True)
445
446- # Trigger WSGI reloading
447- for relid in relation_ids('wsgi'):
448- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
449+ unit.reload_wsgi()
450
451
452 @hooks.hook()
453@@ -466,8 +441,9 @@
454
455
456 @hooks.hook('pgsql-relation-joined', 'pgsql-relation-changed')
457-def pgsql_relation_joined_changed():
458- os.environ['DJANGO_SETTINGS_MODULE'] = settings_module
459+def pgsql_relation_joined_changed(unit=None):
460+ if unit is None:
461+ unit = Unit(config())
462
463 packages = ["python-psycopg2", "postgresql-client"]
464 apt_install(packages, options=['--force-yes'])
465@@ -482,38 +458,43 @@
466 'db_user': relation_get("user"),
467 'db_password': relation_get("password"),
468 'db_host': relation_get("host"),
469+ 'db_options': '',
470 }
471+ if not unit.has_modern_django():
472+ # As of Django 1.6, autocommit defaults to true.
473+ # Django 1.8 removed this as a valid option.
474+ templ_vars['db_options'] = '''"OPTIONS": {'autocommit': True},'''
475
476 process_template('pgsql_engine.tmpl', templ_vars,
477- settings_database_path % {'engine_name': 'pgsql'})
478+ unit.database_path('pgsql'))
479
480- if django_south:
481- south_config_file = os.path.join(settings_dir_path, '50-south.py')
482+ if unit.config('django_south'):
483+ south_config_file = unit.settings_file('50-south.py')
484 process_template('south.tmpl', {}, south_config_file)
485
486- update_database_schema()
487-
488- run('chown -R %s:%s %s' % (wsgi_user, wsgi_group, vcs_clone_dir))
489-
490- # Trigger WSGI reloading
491- for relid in relation_ids('wsgi'):
492- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
493+ update_database_schema(unit)
494+
495+ unit.update_file_ownership()
496+
497+ unit.reload_wsgi()
498
499
500 @hooks.hook('pgsql-relation-broken')
501-def pgsql_relation_broken():
502- run('rm %s' % settings_database_path % {'engine_name': 'pgsql'})
503-
504- # Trigger WSGI reloading
505- for relid in relation_ids('wsgi'):
506- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
507+def pgsql_relation_broken(unit=None):
508+ if unit is None:
509+ unit = Unit(config())
510+
511+ run('rm %s' % unit.database_path('pgsql'))
512+
513+ unit.reload_wsgi()
514
515
516 @hooks.hook('mysql-relation-joined', 'mysql-relation-changed',
517 'mysql-root-relation-joined', 'mysql-root-relation-changed',
518 'mysql-shared-relation-joined', 'mysql-shared-relation-changed')
519-def mysql_relation_joined_changed():
520- os.environ['DJANGO_SETTINGS_MODULE'] = settings_module
521+def mysql_relation_joined_changed(unit=None):
522+ if unit is None:
523+ unit = Unit(config())
524
525 packages = ["python-mysqldb", "mysql-client"]
526 apt_install(packages, options=['--force-yes'])
527@@ -530,33 +511,34 @@
528 'db_host': relation_get("host"),
529 }
530
531- process_template('mysql_engine.tmpl', templ_vars, settings_database_path % {'engine_name': 'mysql'})
532+ process_template('mysql_engine.tmpl', templ_vars, unit.database_path('mysql'))
533
534- if django_south:
535- south_config_file = os.path.join(settings_dir_path, '50-south.py')
536+ if unit.config('django_south'):
537+ south_config_file = unit.settings_file('50-south.py')
538 process_template('south.tmpl', {}, south_config_file)
539
540- update_database_schema()
541-
542- run('chown -R %s:%s %s' % (wsgi_user, wsgi_group, vcs_clone_dir))
543-
544- # Trigger WSGI reloading
545- for relid in relation_ids('wsgi'):
546- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
547+ update_database_schema(unit)
548+
549+ unit.update_file_ownership()
550+ unit.reload_wsgi()
551
552
553 @hooks.hook('mysql-relation-broken', 'mysql-root-relation-broken',
554 'mysql-shared-relation-broken')
555-def mysql_relation_broken():
556- run('rm %s' % settings_database_path % {'engine_name': 'mysql'})
557-
558- # Trigger WSGI reloading
559- for relid in relation_ids('wsgi'):
560- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
561+def mysql_relation_broken(unit=None):
562+ if unit is None:
563+ unit = Unit(config())
564+
565+ run('rm %s' % unit.database_path('mysql'))
566+
567+ unit.reload_wsgi()
568
569
570 @hooks.hook('mongodb-relation-joined', 'mongodb-relation-changed')
571-def mongodb_relation_joined_changed():
572+def mongodb_relation_joined_changed(unit=None):
573+ if unit is None:
574+ unit = Unit(config())
575+
576 packages = ["python-mongoengine"]
577 apt_install(packages, options=['--force-yes'])
578
579@@ -569,26 +551,27 @@
580 'db_port': relation_get("port"),
581 }
582
583- process_template('mongodb_engine.tmpl', templ_vars, settings_database_path % {'engine_name': 'mongodb'})
584-
585- run('chown -R %s:%s %s' % (wsgi_user, wsgi_group, vcs_clone_dir))
586-
587- # Trigger WSGI reloading
588- for relid in relation_ids('wsgi'):
589- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
590+ process_template('mongodb_engine.tmpl', templ_vars, unit.database_path('mongodb'))
591+
592+ unit.update_file_ownership()
593+ unit.reload_wsgi()
594
595
596 @hooks.hook()
597-def mongodb_relation_broken():
598- run('rm %s' % settings_database_path % {'engine_name': 'mongodb'})
599-
600- # Trigger WSGI reloading
601- for relid in relation_ids('wsgi'):
602- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
603+def mongodb_relation_broken(unit=None):
604+ if unit is None:
605+ unit = Unit(config())
606+
607+ run('rm %s' % unit.database_path('mongodb'))
608+
609+ unit.reload_wsgi()
610
611
612 @hooks.hook('redis-relation-joined', 'redis-relation-changed')
613-def redis_relation_joined_changed():
614+def redis_relation_joined_changed(unit=None):
615+ if unit is None:
616+ unit = Unit(config())
617+
618 packages = ["python-redis"]
619 apt_install(packages, options=['--force-yes'])
620
621@@ -601,51 +584,63 @@
622 'db_port': relation_get("port"),
623 }
624
625- process_template('redis_engine.tmpl', templ_vars, settings_database_path % {'engine_name': 'redis'})
626-
627- run('chown -R %s:%s %s' % (wsgi_user, wsgi_group, vcs_clone_dir))
628-
629- # Trigger WSGI reloading
630- for relid in relation_ids('wsgi'):
631- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
632+ process_template('redis_engine.tmpl', templ_vars, unit.database_path('redis'))
633+
634+ unit.update_file_ownership()
635+ unit.reload_wsgi()
636
637
638 @hooks.hook()
639-def redis_relation_broken():
640- run('rm %s' % settings_database_path % {'engine_name': 'redis'})
641-
642- # Trigger WSGI reloading
643- for relid in relation_ids('wsgi'):
644- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
645+def redis_relation_broken(unit=None):
646+ if unit is None:
647+ unit = Unit(config())
648+
649+ run('rm %s' % unit.database_path('redis'))
650+
651+ unit.reload_wsgi()
652
653
654 @hooks.hook('wsgi-relation-joined', 'wsgi-relation-changed')
655-def wsgi_relation_joined_changed():
656- relation_set(relation_settings={'working_dir': working_dir})
657-
658- for var in config_data:
659- if var.startswith('wsgi_') or var in ['listen_ip', 'port']:
660- relation_set(relation_settings={var: config_data[var]})
661-
662- relation_set(relation_settings={'python_path': python_path})
663-
664- open_port(config_data['port'])
665+def wsgi_relation_joined_changed(unit=None):
666+ if unit is None:
667+ unit = Unit(config())
668+
669+ settings = {
670+ 'working_dir': unit.working_dir,
671+ 'python_path': unit.python_path(),
672+ }
673+ unit_config = unit.config_dict()
674+ for key in unit_config:
675+ if key.startswith('wsgi_') or key in ['listen_ip', 'port']:
676+ settings[key] = unit_config[key]
677+
678+ relation_set(relation_settings=settings)
679+ open_port(unit.config('port'))
680
681
682 @hooks.hook()
683-def wsgi_relation_broken():
684- close_port(config_data['port'])
685+def wsgi_relation_broken(unit=None):
686+ if unit is None:
687+ unit = Unit(config())
688+
689+ close_port(unit.config('port'))
690
691
692 @hooks.hook('amqp-relation-joined')
693-def amqp_relation_joined():
694+def amqp_relation_joined(unit=None):
695+ if unit is None:
696+ unit = Unit(config())
697+
698 relation_set(relation_settings={
699- 'username': sanitized_service_name, 'vhost': sanitized_service_name})
700+ 'username': unit.service_name,
701+ 'vhost': unit.service_name,
702+ })
703
704
705 @hooks.hook('amqp-relation-changed')
706-def amqp_relation_changed():
707- os.environ['DJANGO_SETTINGS_MODULE'] = settings_module
708+def amqp_relation_changed(unit=None):
709+ if unit is None:
710+ unit = Unit(config())
711
712 host = relation_get("hostname")
713 if not host:
714@@ -653,33 +648,33 @@
715
716 pip_install('django-celery')
717
718- templ_vars = config_data
719+ templ_vars = unit.config_dict()
720 templ_vars.update({
721- 'username': sanitized_service_name,
722- 'vhost': config_data['celery_amqp_vhost'] or sanitized_service_name
723+ 'username': unit.service_name,
724+ 'vhost': unit.config('celery_amqp_vhost') or unit.service_name
725 })
726-
727 templ_vars.update(relation_get())
728
729- process_template('amqp_celery.tmpl', templ_vars, amqp_path)
730-
731- run('chown -R %s:%s %s' % (wsgi_user, wsgi_group, vcs_clone_dir))
732-
733- update_database_schema()
734-
735- # Trigger WSGI reloading
736- for relid in relation_ids('wsgi'):
737- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
738+ process_template('amqp_celery.tmpl', templ_vars, unit.amqp_path)
739+
740+ unit.update_file_ownership()
741+
742+ update_database_schema(unit)
743+ unit.reload_wsgi()
744
745
746 @hooks.hook()
747-def amqp_relation_broken():
748- run('rm %s' % amqp_path)
749+def amqp_relation_broken(unit=None):
750+ if unit is None:
751+ unit = Unit(config())
752+
753+ run('rm %s' % unit.amqp_path)
754
755
756 @hooks.hook('cache-relation-joined', 'cache-relation-changed')
757-def cache_relation_joined_changed():
758- os.environ['DJANGO_SETTINGS_MODULE'] = settings_module
759+def cache_relation_joined_changed(unit=None):
760+ if unit is None:
761+ unit = Unit(config())
762
763 packages = ["python-memcache"]
764 apt_install(packages, options=['--force-yes'])
765@@ -694,97 +689,211 @@
766 'cache_port': relation_get("port"),
767 }
768
769- process_template('cache.tmpl', templ_vars, settings_database_path % {'engine_name': 'memcache'})
770-
771- run('chown -R %s:%s %s' % (wsgi_user, wsgi_group, vcs_clone_dir))
772-
773- # Trigger WSGI reloading
774- for relid in relation_ids('wsgi'):
775- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
776+ process_template('cache.tmpl', templ_vars, unit.database_path('memcache'))
777+
778+ unit.update_file_ownership()
779+ unit.reload_wsgi()
780
781
782 @hooks.hook()
783-def cache_relation_broken():
784- run('rm %s' % settings_database_path % {'engine_name': 'memcache'})
785-
786- # Trigger WSGI reloading
787- for relid in relation_ids('wsgi'):
788- relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
789+def cache_relation_broken(unit=None):
790+ if unit is None:
791+ unit = Unit(config())
792+
793+ run('rm %s' % unit.database_path('memcache'))
794+
795+ unit.reload_wsgi()
796
797
798 @hooks.hook('website-relation-joined', 'website-relation-changed')
799-def website_relation_joined_changed():
800- relation_set(relation_settings={'port': config_data["port"], 'hostname': unit_private_ip()})
801+def website_relation_joined_changed(unit=None):
802+ if unit is None:
803+ unit = Unit(config())
804+
805+ relation_set(relation_settings={'port': unit.config("port"), 'hostname': unit_private_ip()})
806
807
808 @hooks.hook()
809 def website_relation_broken():
810 pass
811
812-###############################################################################
813-# Global variables
814-###############################################################################
815-config_data = config()
816-log("got config: %s" % str(config_data), DEBUG)
817-
818-vcs = config_data['vcs']
819-repos_url = config_data['repos_url']
820-repos_username = config_data['repos_username']
821-repos_password = config_data['repos_password']
822-repos_branch = config_data['repos_branch']
823-
824-project_template_extension = config_data['project_template_extension']
825-project_template_url = config_data['project_template_url']
826-
827-extra_deb_pkgs = config_data['additional_distro_packages']
828-extra_pip_pkgs = config_data['additional_pip_packages']
829-requirements_pip_files = config_data['requirements_pip_files']
830-wsgi_user = config_data['wsgi_user']
831-wsgi_group = config_data['wsgi_group']
832-install_root = config_data['install_root']
833-application_path = config_data['application_path']
834-django_settings = config_data['django_settings']
835-settings_dir_name = config_data['settings_dir_name']
836-urls_dir_name = config_data['urls_dir_name']
837-django_south = config_data['django_south']
838-django_south_version = config_data['django_south_version']
839-
840-sanitized_service_name = sanitize(service_name())
841-vcs_clone_dir = os.path.join(install_root, sanitized_service_name)
842-if application_path:
843- working_dir = os.path.join(vcs_clone_dir, application_path)
844-else:
845- working_dir = vcs_clone_dir
846-
847-if config_data['python_path']:
848- python_path = os.pathsep.join([config_data['python_path'],
849- os.path.join(working_dir, '../')])
850-else:
851- python_path = os.path.join(working_dir, '../')
852-
853-if django_settings:
854- settings_module = django_settings # andy hack
855-else:
856- if application_path:
857- settings_module = '.'.join([os.path.basename(working_dir), 'settings'])
858- else:
859- settings_module = '.'.join([sanitized_service_name, 'settings'])
860-
861-django_run_dir = os.path.join(working_dir, "run/")
862-django_logs_dir = os.path.join(working_dir, "logs/")
863-
864-settings_injection_path = config_data['settings_injection_path']
865-settings_py_path = os.path.join(working_dir, settings_injection_path)
866-urls_injection_path = config_data['urls_injection_path']
867-urls_py_path = os.path.join(working_dir, urls_injection_path)
868-settings_dir_path = os.path.join(working_dir, os.path.dirname(settings_injection_path), settings_dir_name)
869-urls_dir_path = os.path.join(working_dir, os.path.dirname(urls_injection_path), urls_dir_name)
870-
871-settings_secret_path = os.path.join(settings_dir_path, config_data["settings_secret_key_name"])
872-settings_database_path = os.path.join(settings_dir_path, config_data["settings_database_name"])
873-amqp_path = os.path.join(settings_dir_path, config_data["settings_amqp_name"])
874-
875-hook_name = os.path.basename(sys.argv[0])
876+
877+class Unit(object):
878+
879+ def __init__(self, data, name=None):
880+ """Construct the Unit with something map-like.
881+
882+ During normal hook execution it is the result of the config() function,
883+ in tests, a simple dict also works.
884+ """
885+ self.data = data
886+ if name is None:
887+ name = service_name()
888+ self.service_name = sanitize(name)
889+
890+ install_root = data['install_root']
891+ self.base_dir = os.path.join(install_root, self.service_name)
892+
893+ application_path = data['application_path']
894+ if application_path:
895+ self.working_dir = os.path.join(self.base_dir, application_path)
896+ else:
897+ self.working_dir = self.base_dir
898+
899+ django_settings = data['django_settings']
900+ if django_settings:
901+ self.settings_module = django_settings
902+ else:
903+ if application_path:
904+ self.settings_module = '.'.join([os.path.basename(self.working_dir), 'settings'])
905+ else:
906+ self.settings_module = '.'.join([self.service_name, 'settings'])
907+
908+ settings_injection_path = data['settings_injection_path']
909+ self.settings_dir_name = data['settings_dir_name']
910+ self.settings_dir_path = os.path.join(self.working_dir, os.path.dirname(settings_injection_path), self.settings_dir_name)
911+ self.settings_py_path = os.path.join(self.working_dir, settings_injection_path)
912+
913+ urls_injection_path = data['urls_injection_path']
914+ self.urls_dir_name = data['urls_dir_name']
915+ self.urls_py_path = os.path.join(self.working_dir, urls_injection_path)
916+ self.urls_dir_path = os.path.join(self.working_dir, os.path.dirname(urls_injection_path), self.urls_dir_name)
917+
918+ self.settings_secret_path = self.settings_file(data["settings_secret_key_name"])
919+ self.amqp_path = self.settings_file(data["settings_amqp_name"])
920+
921+ # Cashed property holders
922+ self._django_admin_cmd = None
923+ self._django_version = None
924+
925+ def config(self, field):
926+ return self.data[field]
927+
928+ def config_dict(self):
929+ """Return a copy of the config data dict."""
930+ return dict(self.data.items())
931+
932+ def python_path(self):
933+ # Not entirely convinced that the python path is set correctly by default
934+ # if the configuration includes 'application_path' without 'python_path'.
935+ parent = os.path.join(self.working_dir, '../')
936+ python_path = self.data.get('python_path', None)
937+ if python_path:
938+ return os.pathsep.join([python_path, parent])
939+ return parent
940+
941+ def django_admin_cmd(self):
942+ if self._django_admin_cmd is None:
943+ self._django_admin_cmd = find_django_admin_cmd()
944+ return self._django_admin_cmd
945+
946+ def django_admin_cmd_with_path(self):
947+ admin_cmd = self.django_admin_cmd()
948+ p = 'PYTHONPATH=%s' % self.python_path()
949+ return '%s %s' % (p, admin_cmd)
950+
951+ def django_version(self):
952+ # Since this semantic_version is installed by the install script, we
953+ # import it late.
954+ import semantic_version
955+ if self._django_version is None:
956+ self._django_version = semantic_version.Version(
957+ get_django_version(self.django_admin_cmd()),
958+ partial=True)
959+ return self._django_version
960+
961+ def has_modern_django(self):
962+ """Modern django is 1.7 or later."""
963+ # Since this semantic_version is installed by the install script, we
964+ # import it late.
965+ import semantic_version
966+ return self.django_version() >= semantic_version.Version('1.7.0')
967+
968+ def install_deb_packages(self):
969+ deb_packages = list(CHARM_DEB_PACKAGES)
970+ extra_deb_pkgs = self.data['additional_distro_packages']
971+ if extra_deb_pkgs:
972+ deb_packages.extend(extra_deb_pkgs.split(','))
973+
974+ installed = False
975+ for retry in range(0, 24):
976+ try:
977+ apt_install(deb_packages)
978+ except subprocess.CalledProcessError as e:
979+ log("Error ({}) running {}. Output: {}".format(
980+ e.returncode, e.cmd, e.output))
981+ time.sleep(10)
982+ continue
983+ installed = True
984+ break
985+
986+ if not installed:
987+ log("Too many install failures", ERROR)
988+
989+ def install_pip_packages(self):
990+ pip_packages = list(CHARM_PIP_PACKAGES)
991+ extra_pip_pkgs = self.data['additional_pip_packages']
992+ if extra_pip_pkgs:
993+ pip_packages.extend(extra_pip_pkgs.split(','))
994+ for package in pip_packages:
995+ pip_install(package, upgrade=True)
996+
997+ def install_requirements_files(self, upgrade=False):
998+ requirements_pip_files = self.config('requirements_pip_files')
999+ if requirements_pip_files:
1000+ for req_file in requirements_pip_files.split(','):
1001+ pip_install_req(os.path.join(self.working_dir, req_file), upgrade=upgrade)
1002+
1003+ def settings_file(self, filename):
1004+ """Return the full path of the filename in the settings directory."""
1005+ return os.path.join(self.settings_dir_path, filename)
1006+
1007+ def database_path(self, engine_name):
1008+ db_filename_template = self.data["settings_database_name"]
1009+ db_filename = db_filename_template % {'engine_name': engine_name}
1010+ return self.settings_file(db_filename)
1011+
1012+ def update_file_ownership(self):
1013+ user = self.data['wsgi_user']
1014+ group = self.data['wsgi_group']
1015+ run('chown -R %s:%s %s' % (user, group, self.base_dir))
1016+
1017+ def mkdir(self, path):
1018+ user = self.data['wsgi_user']
1019+ group = self.data['wsgi_group']
1020+ mkdir(path, owner=user, group=group, perms=0755)
1021+
1022+ def reload_wsgi(self):
1023+ # Trigger WSGI reloading
1024+ for relid in relation_ids('wsgi'):
1025+ relation_set(relation_settings={'wsgi_timestamp': time.time()}, relation_id=relid)
1026+
1027+ def update_settings(self):
1028+ site_secret_key = self.config('site_secret_key')
1029+ if not site_secret_key:
1030+ site_secret_key = ''.join([choice('abcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*(-_=+)') for i in range(50)])
1031+ # config-set the secret key?
1032+ process_template('secret.tmpl', {'site_secret_key': site_secret_key}, self.settings_secret_path)
1033+
1034+ dst = self.settings_file('30-allowed.py')
1035+ ip = run('unit-get public-address').strip()
1036+ allowed = [socket.gethostname(), socket.getfqdn(), ip]
1037+ config_allowed_hosts = self.data.get('django_allowed_hosts', '').strip()
1038+ if config_allowed_hosts != '':
1039+ # NOTE: should this replace the defined allowed above or append?
1040+ allowed = config_allowed_hosts.split(' ')
1041+ process_template('allowed_hosts.tmpl', {'allowed_hosts': allowed}, dst)
1042+
1043+ dst = self.settings_file('40-debug.py')
1044+ debug = self.data.get('django_debug', False)
1045+ process_template('debug.tmpl', {'debug': debug}, dst)
1046+
1047+ extra_settings = self.data.get('django_extra_settings', '').strip()
1048+ if extra_settings:
1049+ dst = self.settings_file('50-extra-conf.py')
1050+ pairs = extra_settings.split(',')
1051+ process_template('extra-conf.tmpl', {'pairs': pairs}, dst)
1052+
1053
1054 if __name__ == "__main__":
1055 hooks.execute(sys.argv)
1056
1057=== renamed file 'hooks/tests/test_hooks.py' => 'hooks/tests/test_hooks.py.to-fix'
1058=== removed file 'hooks/tests/test_template.py'
1059--- hooks/tests/test_template.py 2014-07-14 20:04:56 +0000
1060+++ hooks/tests/test_template.py 1970-01-01 00:00:00 +0000
1061@@ -1,125 +0,0 @@
1062-import os
1063-from unittest import TestCase
1064-from mock import patch, MagicMock
1065-
1066-import hooks
1067-
1068-EXPECTED = """
1069-#--------------------------------------------------------------
1070-# This file is managed by Juju; ANY CHANGES WILL BE OVERWRITTEN
1071-#--------------------------------------------------------------
1072-
1073-description "Gunicorn daemon for the PROJECT_NAME project"
1074-
1075-start on (local-filesystems and net-device-up IFACE=eth0)
1076-stop on runlevel [!12345]
1077-
1078-# If the process quits unexpectadly trigger a respawn
1079-respawn
1080-respawn limit 10 5
1081-
1082-setuid WSGI_USER
1083-setgid WSGI_GROUP
1084-chdir WORKING_DIR
1085-
1086-# This line can be removed and replace with the --pythonpath PYTHON_PATH \\
1087-# option with Gunicorn>1.17
1088-env PYTHONPATH=PYTHON_PATH
1089-env A="1"
1090-env B="1 2"
1091-
1092-
1093-exec gunicorn \\
1094- --name=PROJECT_NAME \\
1095- --workers=WSGI_WORKERS \\
1096- --worker-class=WSGI_WORKER_CLASS \\
1097- --worker-connections=WSGI_WORKER_CONNECTIONS \\
1098- --max-requests=WSGI_MAX_REQUESTS \\
1099- --backlog=WSGI_BACKLOG \\
1100- --timeout=WSGI_TIMEOUT \\
1101- --keep-alive=WSGI_KEEP_ALIVE \\
1102- --umask=WSGI_UMASK \\
1103- --bind=LISTEN_IP:PORT \\
1104- --log-file=WSGI_LOG_FILE \\
1105- --log-level=WSGI_LOG_LEVEL \\
1106- --access-logfile=WSGI_ACCESS_LOGFILE \\
1107- --access-logformat=WSGI_ACCESS_LOGFORMAT \\
1108- WSGI_EXTRA \\
1109- WSGI_WSGI_FILE
1110-""".strip()
1111-
1112-
1113-class TemplateTestCase(TestCase):
1114- maxDiff = None
1115-
1116- def setUp(self):
1117- super(TemplateTestCase, self).setUp()
1118- patch_open = patch('hooks.open', create=True)
1119- self.open = patch_open.start()
1120- self.addCleanup(patch_open.stop)
1121-
1122- self.open.return_value = MagicMock(spec=file)
1123- self.file = self.open.return_value.__enter__.return_value
1124-
1125- patch_environ = patch.dict(os.environ, CHARM_DIR='.')
1126- patch_environ.start()
1127- self.addCleanup(patch_environ.stop)
1128-
1129- patch_hookenv = patch('hooks.hookenv')
1130- patch_hookenv.start()
1131- self.addCleanup(patch_hookenv.stop)
1132-
1133- def get_test_context(self):
1134- keys = [
1135- 'project_name',
1136- 'wsgi_user',
1137- 'wsgi_group',
1138- 'working_dir',
1139- 'python_path',
1140- 'wsgi_workers',
1141- 'wsgi_worker_class',
1142- 'wsgi_worker_connections',
1143- 'wsgi_max_requests',
1144- 'wsgi_backlog',
1145- 'wsgi_timeout',
1146- 'wsgi_keep_alive',
1147- 'wsgi_umask',
1148- 'wsgi_log_file',
1149- 'wsgi_log_level',
1150- 'wsgi_access_logfile',
1151- 'wsgi_access_logformat',
1152- 'listen_ip',
1153- 'port',
1154- 'wsgi_extra',
1155- 'wsgi_wsgi_file',
1156- ]
1157- ctx = dict((k, k.upper()) for k in keys)
1158- ctx['env_extra'] = [["A", "1"], ["B", "1 2"]]
1159- return ctx
1160-
1161- def test_template(self):
1162-
1163- ctx = self.get_test_context()
1164-
1165- hooks.process_template('upstart.tmpl', ctx, 'path')
1166- output = self.file.write.call_args[0][0]
1167-
1168- self.assertMultiLineEqual(EXPECTED, output)
1169-
1170- def test_no_access_logfile(self):
1171- ctx = self.get_test_context()
1172- ctx['wsgi_access_logfile'] = ""
1173-
1174- hooks.process_template('upstart.tmpl', ctx, 'path')
1175- output = self.file.write.call_args[0][0]
1176-
1177- self.assertNotIn('--access-logfile', output)
1178-
1179- def test_no_access_logformat(self):
1180- ctx = self.get_test_context()
1181- ctx['wsgi_access_logformat'] = ""
1182-
1183- hooks.process_template('upstart.tmpl', ctx, 'path')
1184- output = self.file.write.call_args[0][0]
1185-
1186- self.assertNotIn('--access-logformat', output)
1187
1188=== added file 'hooks/tests/test_unit.py'
1189--- hooks/tests/test_unit.py 1970-01-01 00:00:00 +0000
1190+++ hooks/tests/test_unit.py 2015-06-06 04:57:38 +0000
1191@@ -0,0 +1,128 @@
1192+from unittest import TestCase
1193+
1194+import hooks
1195+
1196+
1197+class UnitTestCase(TestCase):
1198+
1199+ test_config = {
1200+ 'install_root': '/srv/',
1201+ 'application_path': '',
1202+ 'django_settings': '',
1203+ 'settings_dir_name': 'juju_settings',
1204+ 'settings_injection_path': 'settings.py',
1205+ 'settings_database_name': '60-%(engine_name)s.py',
1206+ 'urls_injection_path': 'urls.py',
1207+ 'urls_dir_name': 'juju_urls',
1208+ 'settings_secret_key_name': '60-secret.py',
1209+ 'settings_amqp_name': '60-amqp.py',
1210+ }
1211+
1212+ def test_unit_name(self):
1213+ unit = hooks.Unit(self.test_config, 'something-special')
1214+ self.assertEqual(unit.service_name, "something_special")
1215+
1216+ def test_database_path(self):
1217+ unit = hooks.Unit(self.test_config, 'magic')
1218+ self.assertEqual(unit.database_path('oracle'), "/srv/magic/juju_settings/60-oracle.py")
1219+
1220+ def test_database_path_settings(self):
1221+ config = dict(self.test_config.items())
1222+ config['install_root'] = '/some-root'
1223+ config['settings_dir_name'] = 'settings'
1224+ config['settings_database_name'] = '20-db-%(engine_name)s.py'
1225+
1226+ unit = hooks.Unit(config, 'magic')
1227+ self.assertEqual(unit.database_path('oracle'), "/some-root/magic/settings/20-db-oracle.py")
1228+
1229+ def setup_version(self, django_version):
1230+
1231+ run_calls = []
1232+
1233+ def get_version(admin_cmd):
1234+ return django_version
1235+
1236+ def run(cmd, **kwargs):
1237+ run_calls.append(cmd)
1238+
1239+ def log(*args, **kwargs):
1240+ pass
1241+
1242+ def django_admin():
1243+ return "/some/admin/location"
1244+
1245+ version_func = hooks.get_django_version
1246+ run_func = hooks.run
1247+ log_func = hooks.log
1248+ find_admin = hooks.find_django_admin_cmd
1249+
1250+ def restore_func():
1251+ hooks.get_django_version = version_func
1252+ hooks.run = run_func
1253+ hooks.log = log_func
1254+ hooks.find_django_admin_cmd = find_admin
1255+
1256+ self.addCleanup(restore_func)
1257+
1258+ hooks.get_django_version = get_version
1259+ hooks.run = run
1260+ hooks.log = log
1261+ hooks.find_django_admin_cmd = django_admin
1262+ return run_calls
1263+
1264+ def test_version_1_6_no_south(self):
1265+ run_calls = self.setup_version('1.6.0')
1266+ config = dict(self.test_config.items())
1267+ config['django_south'] = False
1268+ unit = hooks.Unit(config, 'magic')
1269+
1270+ hooks.update_database_schema(unit)
1271+
1272+ self.assertEqual(
1273+ run_calls,
1274+ [
1275+ 'PYTHONPATH=/srv/magic/../ /some/admin/location syncdb --noinput --settings=magic.settings',
1276+ ])
1277+
1278+ def test_version_1_6_with_south(self):
1279+ run_calls = self.setup_version('1.6.0')
1280+ config = dict(self.test_config.items())
1281+ config['django_south'] = True
1282+ unit = hooks.Unit(config, 'magic')
1283+
1284+ hooks.update_database_schema(unit)
1285+
1286+ self.assertEqual(
1287+ run_calls,
1288+ [
1289+ 'PYTHONPATH=/srv/magic/../ /some/admin/location syncdb --noinput --settings=magic.settings',
1290+ 'PYTHONPATH=/srv/magic/../ /some/admin/location migrate --settings=magic.settings ',
1291+ ])
1292+
1293+ def test_version_1_7_no_south(self):
1294+ run_calls = self.setup_version('1.7.0')
1295+ config = dict(self.test_config.items())
1296+ config['django_south'] = False
1297+ unit = hooks.Unit(config, 'magic')
1298+
1299+ hooks.update_database_schema(unit)
1300+
1301+ self.assertEqual(
1302+ run_calls,
1303+ [
1304+ 'PYTHONPATH=/srv/magic/../ /some/admin/location migrate --settings=magic.settings --noinput',
1305+ ])
1306+
1307+ def test_version_1_7_with_south_crazy_as_it_is(self):
1308+ run_calls = self.setup_version('1.7.0')
1309+ config = dict(self.test_config.items())
1310+ config['django_south'] = True
1311+ unit = hooks.Unit(config, 'magic')
1312+
1313+ hooks.update_database_schema(unit)
1314+
1315+ self.assertEqual(
1316+ run_calls,
1317+ [
1318+ 'PYTHONPATH=/srv/magic/../ /some/admin/location migrate --settings=magic.settings --noinput',
1319+ ])
1320
1321=== modified file 'templates/pgsql_engine.tmpl'
1322--- templates/pgsql_engine.tmpl 2013-08-05 17:08:28 +0000
1323+++ templates/pgsql_engine.tmpl 2015-06-06 04:57:38 +0000
1324@@ -10,7 +10,7 @@
1325 "PASSWORD": '{{ db_password }}',
1326 "HOST": '{{ db_host }}',
1327 "PORT": '',
1328- "OPTIONS": {'autocommit': True},
1329+ {{ db_options }}
1330 }
1331 }
1332
1333
1334=== modified file 'tests/10-mysql'
1335--- tests/10-mysql 2014-07-14 20:40:34 +0000
1336+++ tests/10-mysql 2015-06-06 04:57:38 +0000
1337@@ -45,9 +45,8 @@
1338
1339 def test_ssh(self):
1340 good_content = "mysql"
1341- output = check_output(["juju", "ssh", "python-django/0",
1342- "sudo", "-u", "www-data",
1343- "cat", "/srv/python_django/juju_settings/60-mysql.py"],
1344+ output = check_output(["juju", "run", "--service=python-django",
1345+ "sudo -u www-data cat /srv/python_django/juju_settings/60-mysql.py"],
1346 stderr=STDOUT).decode("utf-8")
1347 self.assertIn(good_content, output, msg=output)
1348
1349
1350=== modified file 'tests/10-postgresql'
1351--- tests/10-postgresql 2014-07-14 20:40:34 +0000
1352+++ tests/10-postgresql 2015-06-06 04:57:38 +0000
1353@@ -45,9 +45,8 @@
1354
1355 def test_ssh(self):
1356 good_content = "psycopg2"
1357- output = check_output(["juju", "ssh", "python-django/0",
1358- "sudo", "-u", "www-data",
1359- "cat", "/srv/python_django/juju_settings/60-pgsql.py"],
1360+ output = check_output(["juju", "run", "--service=python-django",
1361+ "sudo -u www-data cat /srv/python_django/juju_settings/60-pgsql.py"],
1362 stderr=STDOUT).decode("utf-8")
1363 self.assertIn(good_content, output, msg=output)
1364
1365
1366=== added file 'tests/11-django-1.8-with-postgresql'
1367--- tests/11-django-1.8-with-postgresql 1970-01-01 00:00:00 +0000
1368+++ tests/11-django-1.8-with-postgresql 2015-06-06 04:57:38 +0000
1369@@ -0,0 +1,57 @@
1370+#!/usr/bin/python3
1371+"""
1372+This test creates a real deployment, and runs some checks against it.
1373+
1374+FIXME: revert to using ssh -q, stderr=STDOUT instead of 2>&1, stderr=PIPE once
1375+ lp:1281577 is addressed.
1376+"""
1377+
1378+import logging
1379+import unittest
1380+import jujulib.deployer
1381+
1382+from os import getenv
1383+from os.path import dirname, abspath, join
1384+from subprocess import check_output, STDOUT
1385+
1386+from helpers import (check_url, juju_status, get_service_config,
1387+ find_address, get_service_conf, BaseTests)
1388+
1389+log = logging.getLogger(__file__)
1390+
1391+
1392+def setUpModule():
1393+ """Deploys django and postgresql via the charm. All the tests use this deployment."""
1394+ deployer = jujulib.deployer.Deployer()
1395+ config_file = join(
1396+ dirname(dirname(abspath(__file__))),
1397+ "tests", "config", "django.yaml")
1398+ deployer.deploy(getenv("DEPLOYER_TARGET", "django18-postgresql"), [config_file],
1399+ timeout=2000)
1400+
1401+ frontend = find_address(juju_status(), "python-django")
1402+ good_content = "Welcome to Django"
1403+ log.info("Polling. Waiting for app server: {}".format(frontend))
1404+ check_url("http://{}:8080/".format(frontend), good_content, interval=30,
1405+ attempts=10, retry_unavailable=True)
1406+
1407+
1408+class PostgresqlServiceTests(BaseTests):
1409+ @classmethod
1410+ def setUpClass(cls):
1411+ """Prepares juju_status which many tests use."""
1412+ cls.juju_status = juju_status()
1413+ cls.frontend = find_address(cls.juju_status, "python-django")
1414+
1415+ def test_ssh(self):
1416+ good_content = "psycopg2"
1417+ output = check_output(["juju", "run", "--service=python-django",
1418+ "sudo -u www-data cat /srv/python_django/juju_settings/60-pgsql.py"],
1419+ stderr=STDOUT).decode("utf-8")
1420+ self.assertIn(good_content, output, msg=output)
1421+
1422+
1423+if __name__ == "__main__":
1424+ logging.basicConfig(
1425+ level='DEBUG', format='%(asctime)s %(levelname)s %(message)s')
1426+ unittest.main(verbosity=2)
1427
1428=== modified file 'tests/config/django.yaml'
1429--- tests/config/django.yaml 2015-04-03 19:49:12 +0000
1430+++ tests/config/django.yaml 2015-06-06 04:57:38 +0000
1431@@ -96,3 +96,24 @@
1432 relations:
1433 - - "django14:wsgi"
1434 - "gunicorn:wsgi-file"
1435+
1436+django18-postgresql:
1437+ series: trusty
1438+ services:
1439+ postgresql:
1440+ charm: "cs:trusty/postgresql"
1441+ gunicorn:
1442+ charm: "cs:trusty/gunicorn"
1443+ python-django:
1444+ branch: "lp:charms/trysty/python-django"
1445+ charm: python-django
1446+ num_units: 1
1447+ options:
1448+ django_version: 'django>=1.8,<1.9'
1449+ django_debug: True
1450+ expose: true
1451+ relations:
1452+ - - "python-django:wsgi"
1453+ - "gunicorn:wsgi-file"
1454+ - - "python-django"
1455+ - "postgresql:db"
1456
1457=== added directory 'tests/fakepython'
1458=== added directory 'tests/fakepython/apt_pkg'
1459=== added file 'tests/fakepython/apt_pkg/__init__.py'
1460=== added file 'tests/tests.yaml'
1461--- tests/tests.yaml 1970-01-01 00:00:00 +0000
1462+++ tests/tests.yaml 2015-06-06 04:57:38 +0000
1463@@ -0,0 +1,10 @@
1464+virtualenv: true
1465+packages:
1466+ - amulet
1467+ - python-requests
1468+ - python-pip
1469+ - python-jinja2
1470+makefile:
1471+ - virtualenv
1472+ - lint
1473+ - check

Subscribers

People subscribed via source and target branches