Juju Charms Collection
python-django package

Merge lp:~thumper/charms/trusty/python-django/clean-contrib into lp:charms/python-django

Proposed by Tim Penhey on 2015-05-27

Status:	Merged
Merged at revision:	39
Proposed branch:	lp:~thumper/charms/trusty/python-django/clean-contrib
Merge into:	lp:charms/python-django
Diff against target:	5075 lines (+0/-4900) 29 files modified charm-helpers.yaml (+0/-1) hooks/charmhelpers/contrib/ansible/__init__.py (+0/-165) hooks/charmhelpers/contrib/charmhelpers/__init__.py (+0/-184) hooks/charmhelpers/contrib/charmsupport/nrpe.py (+0/-216) hooks/charmhelpers/contrib/charmsupport/volumes.py (+0/-156) hooks/charmhelpers/contrib/hahelpers/apache.py (+0/-59) hooks/charmhelpers/contrib/hahelpers/cluster.py (+0/-183) hooks/charmhelpers/contrib/jujugui/utils.py (+0/-602) hooks/charmhelpers/contrib/network/ip.py (+0/-69) hooks/charmhelpers/contrib/network/ovs/__init__.py (+0/-75) hooks/charmhelpers/contrib/openstack/alternatives.py (+0/-17) hooks/charmhelpers/contrib/openstack/context.py (+0/-700) hooks/charmhelpers/contrib/openstack/neutron.py (+0/-171) hooks/charmhelpers/contrib/openstack/templates/__init__.py (+0/-2) hooks/charmhelpers/contrib/openstack/templating.py (+0/-280) hooks/charmhelpers/contrib/openstack/utils.py (+0/-450) hooks/charmhelpers/contrib/peerstorage/__init__.py (+0/-83) hooks/charmhelpers/contrib/python/packages.py (+0/-76) hooks/charmhelpers/contrib/python/version.py (+0/-18) hooks/charmhelpers/contrib/saltstack/__init__.py (+0/-102) hooks/charmhelpers/contrib/ssl/__init__.py (+0/-78) hooks/charmhelpers/contrib/ssl/service.py (+0/-267) hooks/charmhelpers/contrib/storage/linux/ceph.py (+0/-387) hooks/charmhelpers/contrib/storage/linux/loopback.py (+0/-62) hooks/charmhelpers/contrib/storage/linux/lvm.py (+0/-88) hooks/charmhelpers/contrib/storage/linux/utils.py (+0/-35) hooks/charmhelpers/contrib/templating/contexts.py (+0/-104) hooks/charmhelpers/contrib/templating/pyformat.py (+0/-13) hooks/charmhelpers/contrib/unison/__init__.py (+0/-257)
To merge this branch:	bzr merge lp:~thumper/charms/trusty/python-django/clean-contrib
Related bugs:	Link a bug report

Reviewer	Review Type	Date Requested	Status
Tim Van Steenburgh (community)		2015-05-27	Approve on 2015-06-12
Review via email: mp+260388@code.launchpad.net

Description of the change

This branch just removes the charmhelpers/contrib package.

It isn't used anywhere.

Revision history for this message

Tim Van Steenburgh (tvansteenburgh) wrote on 2015-06-12:

+1 LGTM.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk

The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches

to all changes:

Sheila Miguez

Tim Penhey

charmers

 === modified file 'charm-helpers.yaml'
 --- charm-helpers.yaml	2013-11-26 17:12:54 +0000
 +++ charm-helpers.yaml	2015-05-27 22:03:53 +0000
@@ -3,4 +3,3 @@
  include:
  - core
  - fetch
-- - contrib
 === removed directory 'hooks/charmhelpers/contrib'
 === removed file 'hooks/charmhelpers/contrib/__init__.py'
 === removed directory 'hooks/charmhelpers/contrib/ansible'
 === removed file 'hooks/charmhelpers/contrib/ansible/__init__.py'
 --- hooks/charmhelpers/contrib/ansible/__init__.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/ansible/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,165 +0,0 @@
--# Copyright 2013 Canonical Ltd.
--#
--# Authors:
--# Charm Helpers Developers <juju@lists.ubuntu.com>
--"""Charm Helpers ansible - declare the state of your machines.
--
--This helper enables you to declare your machine state, rather than
--program it procedurally (and have to test each change to your procedures).
--Your install hook can be as simple as:
--
--{{{
--import charmhelpers.contrib.ansible
--
--
--def install():
-- charmhelpers.contrib.ansible.install_ansible_support()
-- charmhelpers.contrib.ansible.apply_playbook('playbooks/install.yaml')
--}}}
--
--and won't need to change (nor will its tests) when you change the machine
--state.
--
--All of your juju config and relation-data are available as template
--variables within your playbooks and templates. An install playbook looks
--something like:
--
--{{{
-----
--- hosts: localhost
-- user: root
--
-- tasks:
-- - name: Add private repositories.
-- template:
-- src: ../templates/private-repositories.list.jinja2
-- dest: /etc/apt/sources.list.d/private.list
--
-- - name: Update the cache.
-- apt: update_cache=yes
--
-- - name: Install dependencies.
-- apt: pkg={{ item }}
-- with_items:
-- - python-mimeparse
-- - python-webob
-- - sunburnt
--
-- - name: Setup groups.
-- group: name={{ item.name }} gid={{ item.gid }}
-- with_items:
-- - { name: 'deploy_user', gid: 1800 }
-- - { name: 'service_user', gid: 1500 }
--
-- ...
--}}}
--
--Read more online about playbooks[1] and standard ansible modules[2].
--
--[1] http://www.ansibleworks.com/docs/playbooks.html
--[2] http://www.ansibleworks.com/docs/modules.html
--"""
--import os
--import subprocess
--
--import charmhelpers.contrib.templating.contexts
--import charmhelpers.core.host
--import charmhelpers.core.hookenv
--import charmhelpers.fetch
--
--
--charm_dir = os.environ.get('CHARM_DIR', '')
--ansible_hosts_path = '/etc/ansible/hosts'
--# Ansible will automatically include any vars in the following
--# file in its inventory when run locally.
--ansible_vars_path = '/etc/ansible/host_vars/localhost'
--
--
--def install_ansible_support(from_ppa=True):
-- """Installs the ansible package.
--
-- By default it is installed from the PPA [1] linked from
-- the ansible website [2].
--
-- [1] https://launchpad.net/~rquillo/+archive/ansible
-- [2] http://www.ansibleworks.com/docs/gettingstarted.html#ubuntu-and-debian
--
-- If from_ppa is false, you must ensure that the package is available
-- from a configured repository.
-- """
-- if from_ppa:
-- charmhelpers.fetch.add_source('ppa:rquillo/ansible')
-- charmhelpers.fetch.apt_update(fatal=True)
-- charmhelpers.fetch.apt_install('ansible')
-- with open(ansible_hosts_path, 'w+') as hosts_file:
-- hosts_file.write('localhost ansible_connection=local')
--
--
--def apply_playbook(playbook, tags=None):
-- tags = tags or []
-- tags = ",".join(tags)
-- charmhelpers.contrib.templating.contexts.juju_state_to_yaml(
-- ansible_vars_path, namespace_separator='__',
-- allow_hyphens_in_keys=False)
-- call = [
-- 'ansible-playbook',
-- '-c',
-- 'local',
-- playbook,
-- ]
-- if tags:
-- call.extend(['--tags', '{}'.format(tags)])
-- subprocess.check_call(call)
--
--
--class AnsibleHooks(charmhelpers.core.hookenv.Hooks):
-- """Run a playbook with the hook-name as the tag.
--
-- This helper builds on the standard hookenv.Hooks helper,
-- but additionally runs the playbook with the hook-name specified
-- using --tags (ie. running all the tasks tagged with the hook-name).
--
-- Example:
-- hooks = AnsibleHooks(playbook_path='playbooks/my_machine_state.yaml')
--
-- # All the tasks within my_machine_state.yaml tagged with 'install'
-- # will be run automatically after do_custom_work()
-- @hooks.hook()
-- def install():
-- do_custom_work()
--
-- # For most of your hooks, you won't need to do anything other
-- # than run the tagged tasks for the hook:
-- @hooks.hook('config-changed', 'start', 'stop')
-- def just_use_playbook():
-- pass
--
-- # As a convenience, you can avoid the above noop function by specifying
-- # the hooks which are handled by ansible-only and they'll be registered
-- # for you:
-- # hooks = AnsibleHooks(
-- # 'playbooks/my_machine_state.yaml',
-- # default_hooks=['config-changed', 'start', 'stop'])
--
-- if __name__ == "__main__":
-- # execute a hook based on the name the program is called by
-- hooks.execute(sys.argv)
-- """
--
-- def __init__(self, playbook_path, default_hooks=None):
-- """Register any hooks handled by ansible."""
-- super(AnsibleHooks, self).__init__()
--
-- self.playbook_path = playbook_path
--
-- default_hooks = default_hooks or []
-- noop = lambda *args, **kwargs: None
-- for hook in default_hooks:
-- self.register(hook, noop)
--
-- def execute(self, args):
-- """Execute the hook followed by the playbook using the hook as tag."""
-- super(AnsibleHooks, self).execute(args)
-- hook_name = os.path.basename(args[0])
-- charmhelpers.contrib.ansible.apply_playbook(
-- self.playbook_path, tags=[hook_name])
 === removed directory 'hooks/charmhelpers/contrib/charmhelpers'
 === removed file 'hooks/charmhelpers/contrib/charmhelpers/__init__.py'
 --- hooks/charmhelpers/contrib/charmhelpers/__init__.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/charmhelpers/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,184 +0,0 @@
--# Copyright 2012 Canonical Ltd. This software is licensed under the
--# GNU Affero General Public License version 3 (see the file LICENSE).
--
--import warnings
--warnings.warn("contrib.charmhelpers is deprecated", DeprecationWarning)
--
--"""Helper functions for writing Juju charms in Python."""
--
--__metaclass__ = type
--__all__ = [
-- #'get_config', # core.hookenv.config()
-- #'log', # core.hookenv.log()
-- #'log_entry', # core.hookenv.log()
-- #'log_exit', # core.hookenv.log()
-- #'relation_get', # core.hookenv.relation_get()
-- #'relation_set', # core.hookenv.relation_set()
-- #'relation_ids', # core.hookenv.relation_ids()
-- #'relation_list', # core.hookenv.relation_units()
-- #'config_get', # core.hookenv.config()
-- #'unit_get', # core.hookenv.unit_get()
-- #'open_port', # core.hookenv.open_port()
-- #'close_port', # core.hookenv.close_port()
-- #'service_control', # core.host.service()
-- 'unit_info', # client-side, NOT IMPLEMENTED
-- 'wait_for_machine', # client-side, NOT IMPLEMENTED
-- 'wait_for_page_contents', # client-side, NOT IMPLEMENTED
-- 'wait_for_relation', # client-side, NOT IMPLEMENTED
-- 'wait_for_unit', # client-side, NOT IMPLEMENTED
--]
--
--import operator
--from shelltoolbox import (
-- command,
--)
--import tempfile
--import time
--import urllib2
--import yaml
--
--SLEEP_AMOUNT = 0.1
--# We create a juju_status Command here because it makes testing much,
--# much easier.
--juju_status = lambda: command('juju')('status')
--
--# re-implemented as charmhelpers.fetch.configure_sources()
--#def configure_source(update=False):
--# source = config_get('source')
--# if ((source.startswith('ppa:') or
--# source.startswith('cloud:') or
--# source.startswith('http:'))):
--# run('add-apt-repository', source)
--# if source.startswith("http:"):
--# run('apt-key', 'import', config_get('key'))
--# if update:
--# run('apt-get', 'update')
--
--
--# DEPRECATED: client-side only
--def make_charm_config_file(charm_config):
-- charm_config_file = tempfile.NamedTemporaryFile()
-- charm_config_file.write(yaml.dump(charm_config))
-- charm_config_file.flush()
-- # The NamedTemporaryFile instance is returned instead of just the name
-- # because we want to take advantage of garbage collection-triggered
-- # deletion of the temp file when it goes out of scope in the caller.
-- return charm_config_file
--
--
--# DEPRECATED: client-side only
--def unit_info(service_name, item_name, data=None, unit=None):
-- if data is None:
-- data = yaml.safe_load(juju_status())
-- service = data['services'].get(service_name)
-- if service is None:
-- # XXX 2012-02-08 gmb:
-- # This allows us to cope with the race condition that we
-- # have between deploying a service and having it come up in
-- # `juju status`. We could probably do with cleaning it up so
-- # that it fails a bit more noisily after a while.
-- return ''
-- units = service['units']
-- if unit is not None:
-- item = units[unit][item_name]
-- else:
-- # It might seem odd to sort the units here, but we do it to
-- # ensure that when no unit is specified, the first unit for the
-- # service (or at least the one with the lowest number) is the
-- # one whose data gets returned.
-- sorted_unit_names = sorted(units.keys())
-- item = units[sorted_unit_names[0]][item_name]
-- return item
--
--
--# DEPRECATED: client-side only
--def get_machine_data():
-- return yaml.safe_load(juju_status())['machines']
--
--
--# DEPRECATED: client-side only
--def wait_for_machine(num_machines=1, timeout=300):
-- """Wait `timeout` seconds for `num_machines` machines to come up.
--
-- This wait_for... function can be called by other wait_for functions
-- whose timeouts might be too short in situations where only a bare
-- Juju setup has been bootstrapped.
--
-- :return: A tuple of (num_machines, time_taken). This is used for
-- testing.
-- """
-- # You may think this is a hack, and you'd be right. The easiest way
-- # to tell what environment we're working in (LXC vs EC2) is to check
-- # the dns-name of the first machine. If it's localhost we're in LXC
-- # and we can just return here.
-- if get_machine_data()[0]['dns-name'] == 'localhost':
-- return 1, 0
-- start_time = time.time()
-- while True:
-- # Drop the first machine, since it's the Zookeeper and that's
-- # not a machine that we need to wait for. This will only work
-- # for EC2 environments, which is why we return early above if
-- # we're in LXC.
-- machine_data = get_machine_data()
-- non_zookeeper_machines = [
-- machine_data[key] for key in machine_data.keys()[1:]]
-- if len(non_zookeeper_machines) >= num_machines:
-- all_machines_running = True
-- for machine in non_zookeeper_machines:
-- if machine.get('instance-state') != 'running':
-- all_machines_running = False
-- break
-- if all_machines_running:
-- break
-- if time.time() - start_time >= timeout:
-- raise RuntimeError('timeout waiting for service to start')
-- time.sleep(SLEEP_AMOUNT)
-- return num_machines, time.time() - start_time
--
--
--# DEPRECATED: client-side only
--def wait_for_unit(service_name, timeout=480):
-- """Wait `timeout` seconds for a given service name to come up."""
-- wait_for_machine(num_machines=1)
-- start_time = time.time()
-- while True:
-- state = unit_info(service_name, 'agent-state')
-- if 'error' in state or state == 'started':
-- break
-- if time.time() - start_time >= timeout:
-- raise RuntimeError('timeout waiting for service to start')
-- time.sleep(SLEEP_AMOUNT)
-- if state != 'started':
-- raise RuntimeError('unit did not start, agent-state: ' + state)
--
--
--# DEPRECATED: client-side only
--def wait_for_relation(service_name, relation_name, timeout=120):
-- """Wait `timeout` seconds for a given relation to come up."""
-- start_time = time.time()
-- while True:
-- relation = unit_info(service_name, 'relations').get(relation_name)
-- if relation is not None and relation['state'] == 'up':
-- break
-- if time.time() - start_time >= timeout:
-- raise RuntimeError('timeout waiting for relation to be up')
-- time.sleep(SLEEP_AMOUNT)
--
--
--# DEPRECATED: client-side only
--def wait_for_page_contents(url, contents, timeout=120, validate=None):
-- if validate is None:
-- validate = operator.contains
-- start_time = time.time()
-- while True:
-- try:
-- stream = urllib2.urlopen(url)
-- except (urllib2.HTTPError, urllib2.URLError):
-- pass
-- else:
-- page = stream.read()
-- if validate(page, contents):
-- return page
-- if time.time() - start_time >= timeout:
-- raise RuntimeError('timeout waiting for contents of ' + url)
-- time.sleep(SLEEP_AMOUNT)
 === removed directory 'hooks/charmhelpers/contrib/charmsupport'
 === removed file 'hooks/charmhelpers/contrib/charmsupport/__init__.py'
 === removed file 'hooks/charmhelpers/contrib/charmsupport/nrpe.py'
 --- hooks/charmhelpers/contrib/charmsupport/nrpe.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/charmsupport/nrpe.py	1970-01-01 00:00:00 +0000
@@ -1,216 +0,0 @@
--"""Compatibility with the nrpe-external-master charm"""
--# Copyright 2012 Canonical Ltd.
--#
--# Authors:
--# Matthew Wedgwood <matthew.wedgwood@canonical.com>
--
--import subprocess
--import pwd
--import grp
--import os
--import re
--import shlex
--import yaml
--
--from charmhelpers.core.hookenv import (
-- config,
-- local_unit,
-- log,
-- relation_ids,
-- relation_set,
--)
--
--from charmhelpers.core.host import service
--
--# This module adds compatibility with the nrpe-external-master and plain nrpe
--# subordinate charms. To use it in your charm:
--#
--# 1. Update metadata.yaml
--#
--# provides:
--# (...)
--# nrpe-external-master:
--# interface: nrpe-external-master
--# scope: container
--#
--# and/or
--#
--# provides:
--# (...)
--# local-monitors:
--# interface: local-monitors
--# scope: container
--
--#
--# 2. Add the following to config.yaml
--#
--# nagios_context:
--# default: "juju"
--# type: string
--# description: |
--# Used by the nrpe subordinate charms.
--# A string that will be prepended to instance name to set the host name
--# in nagios. So for instance the hostname would be something like:
--# juju-myservice-0
--# If you're running multiple environments with the same services in them
--# this allows you to differentiate between them.
--#
--# 3. Add custom checks (Nagios plugins) to files/nrpe-external-master
--#
--# 4. Update your hooks.py with something like this:
--#
--# from charmsupport.nrpe import NRPE
--# (...)
--# def update_nrpe_config():
--# nrpe_compat = NRPE()
--# nrpe_compat.add_check(
--# shortname = "myservice",
--# description = "Check MyService",
--# check_cmd = "check_http -w 2 -c 10 http://localhost"
--# )
--# nrpe_compat.add_check(
--# "myservice_other",
--# "Check for widget failures",
--# check_cmd = "/srv/myapp/scripts/widget_check"
--# )
--# nrpe_compat.write()
--#
--# def config_changed():
--# (...)
--# update_nrpe_config()
--#
--# def nrpe_external_master_relation_changed():
--# update_nrpe_config()
--#
--# def local_monitors_relation_changed():
--# update_nrpe_config()
--#
--# 5. ln -s hooks.py nrpe-external-master-relation-changed
--# ln -s hooks.py local-monitors-relation-changed
--
--
--class CheckException(Exception):
-- pass
--
--
--class Check(object):
-- shortname_re = '[A-Za-z0-9-_]+$'
-- service_template = ("""
--#---------------------------------------------------
--# This file is Juju managed
--#---------------------------------------------------
--define service {{
-- use active-service
-- host_name {nagios_hostname}
-- service_description {nagios_hostname}[{shortname}] """
-- """{description}
-- check_command check_nrpe!{command}
-- servicegroups {nagios_servicegroup}
--}}
--""")
--
-- def __init__(self, shortname, description, check_cmd):
-- super(Check, self).__init__()
-- # XXX: could be better to calculate this from the service name
-- if not re.match(self.shortname_re, shortname):
-- raise CheckException("shortname must match {}".format(
-- Check.shortname_re))
-- self.shortname = shortname
-- self.command = "check_{}".format(shortname)
-- # Note: a set of invalid characters is defined by the
-- # Nagios server config
-- # The default is: illegal_object_name_chars=`~!$%^&*"|'<>?,()=
-- self.description = description
-- self.check_cmd = self._locate_cmd(check_cmd)
--
-- def _locate_cmd(self, check_cmd):
-- search_path = (
-- '/usr/lib/nagios/plugins',
-- '/usr/local/lib/nagios/plugins',
-- )
-- parts = shlex.split(check_cmd)
-- for path in search_path:
-- if os.path.exists(os.path.join(path, parts[0])):
-- command = os.path.join(path, parts[0])
-- if len(parts) > 1:
-- command += " " + " ".join(parts[1:])
-- return command
-- log('Check command not found: {}'.format(parts[0]))
-- return ''
--
-- def write(self, nagios_context, hostname):
-- nrpe_check_file = '/etc/nagios/nrpe.d/{}.cfg'.format(
-- self.command)
-- with open(nrpe_check_file, 'w') as nrpe_check_config:
-- nrpe_check_config.write("# check {}\n".format(self.shortname))
-- nrpe_check_config.write("command[{}]={}\n".format(
-- self.command, self.check_cmd))
--
-- if not os.path.exists(NRPE.nagios_exportdir):
-- log('Not writing service config as {} is not accessible'.format(
-- NRPE.nagios_exportdir))
-- else:
-- self.write_service_config(nagios_context, hostname)
--
-- def write_service_config(self, nagios_context, hostname):
-- for f in os.listdir(NRPE.nagios_exportdir):
-- if re.search('.*{}.cfg'.format(self.command), f):
-- os.remove(os.path.join(NRPE.nagios_exportdir, f))
--
-- templ_vars = {
-- 'nagios_hostname': hostname,
-- 'nagios_servicegroup': nagios_context,
-- 'description': self.description,
-- 'shortname': self.shortname,
-- 'command': self.command,
-- }
-- nrpe_service_text = Check.service_template.format(**templ_vars)
-- nrpe_service_file = '{}/service__{}_{}.cfg'.format(
-- NRPE.nagios_exportdir, hostname, self.command)
-- with open(nrpe_service_file, 'w') as nrpe_service_config:
-- nrpe_service_config.write(str(nrpe_service_text))
--
-- def run(self):
-- subprocess.call(self.check_cmd)
--
--
--class NRPE(object):
-- nagios_logdir = '/var/log/nagios'
-- nagios_exportdir = '/var/lib/nagios/export'
-- nrpe_confdir = '/etc/nagios/nrpe.d'
--
-- def __init__(self):
-- super(NRPE, self).__init__()
-- self.config = config()
-- self.nagios_context = self.config['nagios_context']
-- self.unit_name = local_unit().replace('/', '-')
-- self.hostname = "{}-{}".format(self.nagios_context, self.unit_name)
-- self.checks = []
--
-- def add_check(self, *args, **kwargs):
-- self.checks.append(Check(*args, **kwargs))
--
-- def write(self):
-- try:
-- nagios_uid = pwd.getpwnam('nagios').pw_uid
-- nagios_gid = grp.getgrnam('nagios').gr_gid
-- except:
-- log("Nagios user not set up, nrpe checks not updated")
-- return
--
-- if not os.path.exists(NRPE.nagios_logdir):
-- os.mkdir(NRPE.nagios_logdir)
-- os.chown(NRPE.nagios_logdir, nagios_uid, nagios_gid)
--
-- nrpe_monitors = {}
-- monitors = {"monitors": {"remote": {"nrpe": nrpe_monitors}}}
-- for nrpecheck in self.checks:
-- nrpecheck.write(self.nagios_context, self.hostname)
-- nrpe_monitors[nrpecheck.shortname] = {
-- "command": nrpecheck.command,
-- }
--
-- service('restart', 'nagios-nrpe-server')
--
-- for rid in relation_ids("local-monitors"):
-- relation_set(relation_id=rid, monitors=yaml.dump(monitors))
 === removed file 'hooks/charmhelpers/contrib/charmsupport/volumes.py'
 --- hooks/charmhelpers/contrib/charmsupport/volumes.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/charmsupport/volumes.py	1970-01-01 00:00:00 +0000
@@ -1,156 +0,0 @@
--'''
--Functions for managing volumes in juju units. One volume is supported per unit.
--Subordinates may have their own storage, provided it is on its own partition.
--
--Configuration stanzas:
-- volume-ephemeral:
-- type: boolean
-- default: true
-- description: >
-- If false, a volume is mounted as sepecified in "volume-map"
-- If true, ephemeral storage will be used, meaning that log data
-- will only exist as long as the machine. YOU HAVE BEEN WARNED.
-- volume-map:
-- type: string
-- default: {}
-- description: >
-- YAML map of units to device names, e.g:
-- "{ rsyslog/0: /dev/vdb, rsyslog/1: /dev/vdb }"
-- Service units will raise a configure-error if volume-ephemeral
-- is 'true' and no volume-map value is set. Use 'juju set' to set a
-- value and 'juju resolved' to complete configuration.
--
--Usage:
-- from charmsupport.volumes import configure_volume, VolumeConfigurationError
-- from charmsupport.hookenv import log, ERROR
-- def post_mount_hook():
-- stop_service('myservice')
-- def post_mount_hook():
-- start_service('myservice')
--
-- if __name__ == '__main__':
-- try:
-- configure_volume(before_change=pre_mount_hook,
-- after_change=post_mount_hook)
-- except VolumeConfigurationError:
-- log('Storage could not be configured', ERROR)
--'''
--
--# XXX: Known limitations
--# - fstab is neither consulted nor updated
--
--import os
--from charmhelpers.core import hookenv
--from charmhelpers.core import host
--import yaml
--
--
--MOUNT_BASE = '/srv/juju/volumes'
--
--
--class VolumeConfigurationError(Exception):
-- '''Volume configuration data is missing or invalid'''
-- pass
--
--
--def get_config():
-- '''Gather and sanity-check volume configuration data'''
-- volume_config = {}
-- config = hookenv.config()
--
-- errors = False
--
-- if config.get('volume-ephemeral') in (True, 'True', 'true', 'Yes', 'yes'):
-- volume_config['ephemeral'] = True
-- else:
-- volume_config['ephemeral'] = False
--
-- try:
-- volume_map = yaml.safe_load(config.get('volume-map', '{}'))
-- except yaml.YAMLError as e:
-- hookenv.log("Error parsing YAML volume-map: {}".format(e),
-- hookenv.ERROR)
-- errors = True
-- if volume_map is None:
-- # probably an empty string
-- volume_map = {}
-- elif not isinstance(volume_map, dict):
-- hookenv.log("Volume-map should be a dictionary, not {}".format(
-- type(volume_map)))
-- errors = True
--
-- volume_config['device'] = volume_map.get(os.environ['JUJU_UNIT_NAME'])
-- if volume_config['device'] and volume_config['ephemeral']:
-- # asked for ephemeral storage but also defined a volume ID
-- hookenv.log('A volume is defined for this unit, but ephemeral '
-- 'storage was requested', hookenv.ERROR)
-- errors = True
-- elif not volume_config['device'] and not volume_config['ephemeral']:
-- # asked for permanent storage but did not define volume ID
-- hookenv.log('Ephemeral storage was requested, but there is no volume '
-- 'defined for this unit.', hookenv.ERROR)
-- errors = True
--
-- unit_mount_name = hookenv.local_unit().replace('/', '-')
-- volume_config['mountpoint'] = os.path.join(MOUNT_BASE, unit_mount_name)
--
-- if errors:
-- return None
-- return volume_config
--
--
--def mount_volume(config):
-- if os.path.exists(config['mountpoint']):
-- if not os.path.isdir(config['mountpoint']):
-- hookenv.log('Not a directory: {}'.format(config['mountpoint']))
-- raise VolumeConfigurationError()
-- else:
-- host.mkdir(config['mountpoint'])
-- if os.path.ismount(config['mountpoint']):
-- unmount_volume(config)
-- if not host.mount(config['device'], config['mountpoint'], persist=True):
-- raise VolumeConfigurationError()
--
--
--def unmount_volume(config):
-- if os.path.ismount(config['mountpoint']):
-- if not host.umount(config['mountpoint'], persist=True):
-- raise VolumeConfigurationError()
--
--
--def managed_mounts():
-- '''List of all mounted managed volumes'''
-- return filter(lambda mount: mount[0].startswith(MOUNT_BASE), host.mounts())
--
--
--def configure_volume(before_change=lambda: None, after_change=lambda: None):
-- '''Set up storage (or don't) according to the charm's volume configuration.
-- Returns the mount point or "ephemeral". before_change and after_change
-- are optional functions to be called if the volume configuration changes.
-- '''
--
-- config = get_config()
-- if not config:
-- hookenv.log('Failed to read volume configuration', hookenv.CRITICAL)
-- raise VolumeConfigurationError()
--
-- if config['ephemeral']:
-- if os.path.ismount(config['mountpoint']):
-- before_change()
-- unmount_volume(config)
-- after_change()
-- return 'ephemeral'
-- else:
-- # persistent storage
-- if os.path.ismount(config['mountpoint']):
-- mounts = dict(managed_mounts())
-- if mounts.get(config['mountpoint']) != config['device']:
-- before_change()
-- unmount_volume(config)
-- mount_volume(config)
-- after_change()
-- else:
-- before_change()
-- mount_volume(config)
-- after_change()
-- return config['mountpoint']
 === removed directory 'hooks/charmhelpers/contrib/hahelpers'
 === removed file 'hooks/charmhelpers/contrib/hahelpers/__init__.py'
 === removed file 'hooks/charmhelpers/contrib/hahelpers/apache.py'
 --- hooks/charmhelpers/contrib/hahelpers/apache.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/hahelpers/apache.py	1970-01-01 00:00:00 +0000
@@ -1,59 +0,0 @@
--#
--# Copyright 2012 Canonical Ltd.
--#
--# This file is sourced from lp:openstack-charm-helpers
--#
--# Authors:
--# James Page <james.page@ubuntu.com>
--# Adam Gandelman <adamg@ubuntu.com>
--#
--
--import subprocess
--
--from charmhelpers.core.hookenv import (
-- config as config_get,
-- relation_get,
-- relation_ids,
-- related_units as relation_list,
-- log,
-- INFO,
--)
--
--
--def get_cert():
-- cert = config_get('ssl_cert')
-- key = config_get('ssl_key')
-- if not (cert and key):
-- log("Inspecting identity-service relations for SSL certificate.",
-- level=INFO)
-- cert = key = None
-- for r_id in relation_ids('identity-service'):
-- for unit in relation_list(r_id):
-- if not cert:
-- cert = relation_get('ssl_cert',
-- rid=r_id, unit=unit)
-- if not key:
-- key = relation_get('ssl_key',
-- rid=r_id, unit=unit)
-- return (cert, key)
--
--
--def get_ca_cert():
-- ca_cert = config_get('ssl_ca')
-- if ca_cert is None:
-- log("Inspecting identity-service relations for CA SSL certificate.",
-- level=INFO)
-- for r_id in relation_ids('identity-service'):
-- for unit in relation_list(r_id):
-- if ca_cert is None:
-- ca_cert = relation_get('ca_cert',
-- rid=r_id, unit=unit)
-- return ca_cert
--
--
--def install_ca_cert(ca_cert):
-- if ca_cert:
-- with open('/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt',
-- 'w') as crt:
-- crt.write(ca_cert)
-- subprocess.check_call(['update-ca-certificates', '--fresh'])
 === removed file 'hooks/charmhelpers/contrib/hahelpers/cluster.py'
 --- hooks/charmhelpers/contrib/hahelpers/cluster.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/hahelpers/cluster.py	1970-01-01 00:00:00 +0000
@@ -1,183 +0,0 @@
--#
--# Copyright 2012 Canonical Ltd.
--#
--# Authors:
--# James Page <james.page@ubuntu.com>
--# Adam Gandelman <adamg@ubuntu.com>
--#
--
--import subprocess
--import os
--
--from socket import gethostname as get_unit_hostname
--
--from charmhelpers.core.hookenv import (
-- log,
-- relation_ids,
-- related_units as relation_list,
-- relation_get,
-- config as config_get,
-- INFO,
-- ERROR,
-- unit_get,
--)
--
--
--class HAIncompleteConfig(Exception):
-- pass
--
--
--def is_clustered():
-- for r_id in (relation_ids('ha') or []):
-- for unit in (relation_list(r_id) or []):
-- clustered = relation_get('clustered',
-- rid=r_id,
-- unit=unit)
-- if clustered:
-- return True
-- return False
--
--
--def is_leader(resource):
-- cmd = [
-- "crm", "resource",
-- "show", resource
-- ]
-- try:
-- status = subprocess.check_output(cmd)
-- except subprocess.CalledProcessError:
-- return False
-- else:
-- if get_unit_hostname() in status:
-- return True
-- else:
-- return False
--
--
--def peer_units():
-- peers = []
-- for r_id in (relation_ids('cluster') or []):
-- for unit in (relation_list(r_id) or []):
-- peers.append(unit)
-- return peers
--
--
--def oldest_peer(peers):
-- local_unit_no = int(os.getenv('JUJU_UNIT_NAME').split('/')[1])
-- for peer in peers:
-- remote_unit_no = int(peer.split('/')[1])
-- if remote_unit_no < local_unit_no:
-- return False
-- return True
--
--
--def eligible_leader(resource):
-- if is_clustered():
-- if not is_leader(resource):
-- log('Deferring action to CRM leader.', level=INFO)
-- return False
-- else:
-- peers = peer_units()
-- if peers and not oldest_peer(peers):
-- log('Deferring action to oldest service unit.', level=INFO)
-- return False
-- return True
--
--
--def https():
-- '''
-- Determines whether enough data has been provided in configuration
-- or relation data to configure HTTPS
-- .
-- returns: boolean
-- '''
-- if config_get('use-https') == "yes":
-- return True
-- if config_get('ssl_cert') and config_get('ssl_key'):
-- return True
-- for r_id in relation_ids('identity-service'):
-- for unit in relation_list(r_id):
-- rel_state = [
-- relation_get('https_keystone', rid=r_id, unit=unit),
-- relation_get('ssl_cert', rid=r_id, unit=unit),
-- relation_get('ssl_key', rid=r_id, unit=unit),
-- relation_get('ca_cert', rid=r_id, unit=unit),
-- ]
-- # NOTE: works around (LP: #1203241)
-- if (None not in rel_state) and ('' not in rel_state):
-- return True
-- return False
--
--
--def determine_api_port(public_port):
-- '''
-- Determine correct API server listening port based on
-- existence of HTTPS reverse proxy and/or haproxy.
--
-- public_port: int: standard public port for given service
--
-- returns: int: the correct listening port for the API service
-- '''
-- i = 0
-- if len(peer_units()) > 0 or is_clustered():
-- i += 1
-- if https():
-- i += 1
-- return public_port - (i * 10)
--
--
--def determine_apache_port(public_port):
-- '''
-- Description: Determine correct apache listening port based on public IP +
-- state of the cluster.
--
-- public_port: int: standard public port for given service
--
-- returns: int: the correct listening port for the HAProxy service
-- '''
-- i = 0
-- if len(peer_units()) > 0 or is_clustered():
-- i += 1
-- return public_port - (i * 10)
--
--
--def get_hacluster_config():
-- '''
-- Obtains all relevant configuration from charm configuration required
-- for initiating a relation to hacluster:
--
-- ha-bindiface, ha-mcastport, vip, vip_iface, vip_cidr
--
-- returns: dict: A dict containing settings keyed by setting name.
-- raises: HAIncompleteConfig if settings are missing.
-- '''
-- settings = ['ha-bindiface', 'ha-mcastport', 'vip', 'vip_iface', 'vip_cidr']
-- conf = {}
-- for setting in settings:
-- conf[setting] = config_get(setting)
-- missing = []
-- [missing.append(s) for s, v in conf.iteritems() if v is None]
-- if missing:
-- log('Insufficient config data to configure hacluster.', level=ERROR)
-- raise HAIncompleteConfig
-- return conf
--
--
--def canonical_url(configs, vip_setting='vip'):
-- '''
-- Returns the correct HTTP URL to this host given the state of HTTPS
-- configuration and hacluster.
--
-- :configs : OSTemplateRenderer: A config tempating object to inspect for
-- a complete https context.
-- :vip_setting: str: Setting in charm config that specifies
-- VIP address.
-- '''
-- scheme = 'http'
-- if 'https' in configs.complete_contexts():
-- scheme = 'https'
-- if is_clustered():
-- addr = config_get(vip_setting)
-- else:
-- addr = unit_get('private-address')
-- return '%s://%s' % (scheme, addr)
 === removed directory 'hooks/charmhelpers/contrib/jujugui'
 === removed file 'hooks/charmhelpers/contrib/jujugui/__init__.py'
 === removed file 'hooks/charmhelpers/contrib/jujugui/utils.py'
 --- hooks/charmhelpers/contrib/jujugui/utils.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/jujugui/utils.py	1970-01-01 00:00:00 +0000
@@ -1,602 +0,0 @@
--"""Juju GUI charm utilities."""
--
--__all__ = [
-- 'AGENT',
-- 'APACHE',
-- 'API_PORT',
-- 'CURRENT_DIR',
-- 'HAPROXY',
-- 'IMPROV',
-- 'JUJU_DIR',
-- 'JUJU_GUI_DIR',
-- 'JUJU_GUI_SITE',
-- 'JUJU_PEM',
-- 'WEB_PORT',
-- 'bzr_checkout',
-- 'chain',
-- 'cmd_log',
-- 'fetch_api',
-- 'fetch_gui',
-- 'find_missing_packages',
-- 'first_path_in_dir',
-- 'get_api_address',
-- 'get_npm_cache_archive_url',
-- 'get_release_file_url',
-- 'get_staging_dependencies',
-- 'get_zookeeper_address',
-- 'legacy_juju',
-- 'log_hook',
-- 'merge',
-- 'parse_source',
-- 'prime_npm_cache',
-- 'render_to_file',
-- 'save_or_create_certificates',
-- 'setup_apache',
-- 'setup_gui',
-- 'start_agent',
-- 'start_gui',
-- 'start_improv',
-- 'write_apache_config',
--]
--
--from contextlib import contextmanager
--import errno
--import json
--import os
--import logging
--import shutil
--from subprocess import CalledProcessError
--import tempfile
--from urlparse import urlparse
--
--import apt
--import tempita
--
--from launchpadlib.launchpad import Launchpad
--from shelltoolbox import (
-- Serializer,
-- apt_get_install,
-- command,
-- environ,
-- install_extra_repositories,
-- run,
-- script_name,
-- search_file,
-- su,
--)
--from charmhelpers.core.host import (
-- service_start,
--)
--from charmhelpers.core.hookenv import (
-- log,
-- config,
-- unit_get,
--)
--
--
--AGENT = 'juju-api-agent'
--APACHE = 'apache2'
--IMPROV = 'juju-api-improv'
--HAPROXY = 'haproxy'
--
--API_PORT = 8080
--WEB_PORT = 8000
--
--CURRENT_DIR = os.getcwd()
--JUJU_DIR = os.path.join(CURRENT_DIR, 'juju')
--JUJU_GUI_DIR = os.path.join(CURRENT_DIR, 'juju-gui')
--JUJU_GUI_SITE = '/etc/apache2/sites-available/juju-gui'
--JUJU_GUI_PORTS = '/etc/apache2/ports.conf'
--JUJU_PEM = 'juju.includes-private-key.pem'
--BUILD_REPOSITORIES = ('ppa:chris-lea/node.js-legacy',)
--DEB_BUILD_DEPENDENCIES = (
-- 'bzr', 'imagemagick', 'make', 'nodejs', 'npm',
--)
--DEB_STAGE_DEPENDENCIES = (
-- 'zookeeper',
--)
--
--
--# Store the configuration from on invocation to the next.
--config_json = Serializer('/tmp/config.json')
--# Bazaar checkout command.
--bzr_checkout = command('bzr', 'co', '--lightweight')
--# Whether or not the charm is deployed using juju-core.
--# If juju-core has been used to deploy the charm, an agent.conf file must
--# be present in the charm parent directory.
--legacy_juju = lambda: not os.path.exists(
-- os.path.join(CURRENT_DIR, '..', 'agent.conf'))
--
--
--def _get_build_dependencies():
-- """Install deb dependencies for building."""
-- log('Installing build dependencies.')
-- cmd_log(install_extra_repositories(*BUILD_REPOSITORIES))
-- cmd_log(apt_get_install(*DEB_BUILD_DEPENDENCIES))
--
--
--def get_api_address(unit_dir):
-- """Return the Juju API address stored in the uniter agent.conf file."""
-- import yaml # python-yaml is only installed if juju-core is used.
-- # XXX 2013-03-27 frankban bug=1161443:
-- # currently the uniter agent.conf file does not include the API
-- # address. For now retrieve it from the machine agent file.
-- base_dir = os.path.abspath(os.path.join(unit_dir, '..'))
-- for dirname in os.listdir(base_dir):
-- if dirname.startswith('machine-'):
-- agent_conf = os.path.join(base_dir, dirname, 'agent.conf')
-- break
-- else:
-- raise IOError('Juju agent configuration file not found.')
-- contents = yaml.load(open(agent_conf))
-- return contents['apiinfo']['addrs'][0]
--
--
--def get_staging_dependencies():
-- """Install deb dependencies for the stage (improv) environment."""
-- log('Installing stage dependencies.')
-- cmd_log(apt_get_install(*DEB_STAGE_DEPENDENCIES))
--
--
--def first_path_in_dir(directory):
-- """Return the full path of the first file/dir in *directory*."""
-- return os.path.join(directory, os.listdir(directory)[0])
--
--
--def _get_by_attr(collection, attr, value):
-- """Return the first item in collection having attr == value.
--
-- Return None if the item is not found.
-- """
-- for item in collection:
-- if getattr(item, attr) == value:
-- return item
--
--
--def get_release_file_url(project, series_name, release_version):
-- """Return the URL of the release file hosted in Launchpad.
--
-- The returned URL points to a release file for the given project, series
-- name and release version.
-- The argument *project* is a project object as returned by launchpadlib.
-- The arguments *series_name* and *release_version* are strings. If
-- *release_version* is None, the URL of the latest release will be returned.
-- """
-- series = _get_by_attr(project.series, 'name', series_name)
-- if series is None:
-- raise ValueError('%r: series not found' % series_name)
-- # Releases are returned by Launchpad in reverse date order.
-- releases = list(series.releases)
-- if not releases:
-- raise ValueError('%r: series does not contain releases' % series_name)
-- if release_version is not None:
-- release = _get_by_attr(releases, 'version', release_version)
-- if release is None:
-- raise ValueError('%r: release not found' % release_version)
-- releases = [release]
-- for release in releases:
-- for file_ in release.files:
-- if str(file_).endswith('.tgz'):
-- return file_.file_link
-- raise ValueError('%r: file not found' % release_version)
--
--
--def get_zookeeper_address(agent_file_path):
-- """Retrieve the Zookeeper address contained in the given *agent_file_path*.
--
-- The *agent_file_path* is a path to a file containing a line similar to the
-- following::
--
-- env JUJU_ZOOKEEPER="address"
-- """
-- line = search_file('JUJU_ZOOKEEPER', agent_file_path).strip()
-- return line.split('=')[1].strip('"')
--
--
--@contextmanager
--def log_hook():
-- """Log when a hook starts and stops its execution.
--
-- Also log to stdout possible CalledProcessError exceptions raised executing
-- the hook.
-- """
-- script = script_name()
-- log(">>> Entering {}".format(script))
-- try:
-- yield
-- except CalledProcessError as err:
-- log('Exception caught:')
-- log(err.output)
-- raise
-- finally:
-- log("<<< Exiting {}".format(script))
--
--
--def parse_source(source):
-- """Parse the ``juju-gui-source`` option.
--
-- Return a tuple of two elements representing info on how to deploy Juju GUI.
-- Examples:
-- - ('stable', None): latest stable release;
-- - ('stable', '0.1.0'): stable release v0.1.0;
-- - ('trunk', None): latest trunk release;
-- - ('trunk', '0.1.0+build.1'): trunk release v0.1.0 bzr revision 1;
-- - ('branch', 'lp:juju-gui'): release is made from a branch;
-- - ('url', 'http://example.com/gui'): release from a downloaded file.
-- """
-- if source.startswith('url:'):
-- source = source[4:]
-- # Support file paths, including relative paths.
-- if urlparse(source).scheme == '':
-- if not source.startswith('/'):
-- source = os.path.join(os.path.abspath(CURRENT_DIR), source)
-- source = "file://%s" % source
-- return 'url', source
-- if source in ('stable', 'trunk'):
-- return source, None
-- if source.startswith('lp:') or source.startswith('http://'):
-- return 'branch', source
-- if 'build' in source:
-- return 'trunk', source
-- return 'stable', source
--
--
--def render_to_file(template_name, context, destination):
-- """Render the given *template_name* into *destination* using *context*.
--
-- The tempita template language is used to render contents
-- (see http://pythonpaste.org/tempita/).
-- The argument *template_name* is the name or path of the template file:
-- it may be either a path relative to ``../config`` or an absolute path.
-- The argument *destination* is a file path.
-- The argument *context* is a dict-like object.
-- """
-- template_path = os.path.abspath(template_name)
-- template = tempita.Template.from_filename(template_path)
-- with open(destination, 'w') as stream:
-- stream.write(template.substitute(context))
--
--
--results_log = None
--
--
--def _setupLogging():
-- global results_log
-- if results_log is not None:
-- return
-- cfg = config()
-- logging.basicConfig(
-- filename=cfg['command-log-file'],
-- level=logging.INFO,
-- format="%(asctime)s: %(name)s@%(levelname)s %(message)s")
-- results_log = logging.getLogger('juju-gui')
--
--
--def cmd_log(results):
-- global results_log
-- if not results:
-- return
-- if results_log is None:
-- _setupLogging()
-- # Since 'results' may be multi-line output, start it on a separate line
-- # from the logger timestamp, etc.
-- results_log.info('\n' + results)
--
--
--def start_improv(staging_env, ssl_cert_path,
-- config_path='/etc/init/juju-api-improv.conf'):
-- """Start a simulated juju environment using ``improv.py``."""
-- log('Setting up staging start up script.')
-- context = {
-- 'juju_dir': JUJU_DIR,
-- 'keys': ssl_cert_path,
-- 'port': API_PORT,
-- 'staging_env': staging_env,
-- }
-- render_to_file('config/juju-api-improv.conf.template', context, config_path)
-- log('Starting the staging backend.')
-- with su('root'):
-- service_start(IMPROV)
--
--
--def start_agent(
-- ssl_cert_path, config_path='/etc/init/juju-api-agent.conf',
-- read_only=False):
-- """Start the Juju agent and connect to the current environment."""
-- # Retrieve the Zookeeper address from the start up script.
-- unit_dir = os.path.realpath(os.path.join(CURRENT_DIR, '..'))
-- agent_file = '/etc/init/juju-{0}.conf'.format(os.path.basename(unit_dir))
-- zookeeper = get_zookeeper_address(agent_file)
-- log('Setting up API agent start up script.')
-- context = {
-- 'juju_dir': JUJU_DIR,
-- 'keys': ssl_cert_path,
-- 'port': API_PORT,
-- 'zookeeper': zookeeper,
-- 'read_only': read_only
-- }
-- render_to_file('config/juju-api-agent.conf.template', context, config_path)
-- log('Starting API agent.')
-- with su('root'):
-- service_start(AGENT)
--
--
--def start_gui(
-- console_enabled, login_help, readonly, in_staging, ssl_cert_path,
-- charmworld_url, serve_tests, haproxy_path='/etc/haproxy/haproxy.cfg',
-- config_js_path=None, secure=True, sandbox=False):
-- """Set up and start the Juju GUI server."""
-- with su('root'):
-- run('chown', '-R', 'ubuntu:', JUJU_GUI_DIR)
-- # XXX 2013-02-05 frankban bug=1116320:
-- # External insecure resources are still loaded when testing in the
-- # debug environment. For now, switch to the production environment if
-- # the charm is configured to serve tests.
-- if in_staging and not serve_tests:
-- build_dirname = 'build-debug'
-- else:
-- build_dirname = 'build-prod'
-- build_dir = os.path.join(JUJU_GUI_DIR, build_dirname)
-- log('Generating the Juju GUI configuration file.')
-- is_legacy_juju = legacy_juju()
-- user, password = None, None
-- if (is_legacy_juju and in_staging) or sandbox:
-- user, password = 'admin', 'admin'
-- else:
-- user, password = None, None
--
-- api_backend = 'python' if is_legacy_juju else 'go'
-- if secure:
-- protocol = 'wss'
-- else:
-- log('Running in insecure mode! Port 80 will serve unencrypted.')
-- protocol = 'ws'
--
-- context = {
-- 'raw_protocol': protocol,
-- 'address': unit_get('public-address'),
-- 'console_enabled': json.dumps(console_enabled),
-- 'login_help': json.dumps(login_help),
-- 'password': json.dumps(password),
-- 'api_backend': json.dumps(api_backend),
-- 'readonly': json.dumps(readonly),
-- 'user': json.dumps(user),
-- 'protocol': json.dumps(protocol),
-- 'sandbox': json.dumps(sandbox),
-- 'charmworld_url': json.dumps(charmworld_url),
-- }
-- if config_js_path is None:
-- config_js_path = os.path.join(
-- build_dir, 'juju-ui', 'assets', 'config.js')
-- render_to_file('config/config.js.template', context, config_js_path)
--
-- write_apache_config(build_dir, serve_tests)
--
-- log('Generating haproxy configuration file.')
-- if is_legacy_juju:
-- # The PyJuju API agent is listening on localhost.
-- api_address = '127.0.0.1:{0}'.format(API_PORT)
-- else:
-- # Retrieve the juju-core API server address.
-- api_address = get_api_address(os.path.join(CURRENT_DIR, '..'))
-- context = {
-- 'api_address': api_address,
-- 'api_pem': JUJU_PEM,
-- 'legacy_juju': is_legacy_juju,
-- 'ssl_cert_path': ssl_cert_path,
-- # In PyJuju environments, use the same certificate for both HTTPS and
-- # WebSocket connections. In juju-core the system already has the proper
-- # certificate installed.
-- 'web_pem': JUJU_PEM,
-- 'web_port': WEB_PORT,
-- 'secure': secure
-- }
-- render_to_file('config/haproxy.cfg.template', context, haproxy_path)
-- log('Starting Juju GUI.')
--
--
--def write_apache_config(build_dir, serve_tests=False):
-- log('Generating the apache site configuration file.')
-- context = {
-- 'port': WEB_PORT,
-- 'serve_tests': serve_tests,
-- 'server_root': build_dir,
-- 'tests_root': os.path.join(JUJU_GUI_DIR, 'test', ''),
-- }
-- render_to_file('config/apache-ports.template', context, JUJU_GUI_PORTS)
-- render_to_file('config/apache-site.template', context, JUJU_GUI_SITE)
--
--
--def get_npm_cache_archive_url(Launchpad=Launchpad):
-- """Figure out the URL of the most recent NPM cache archive on Launchpad."""
-- launchpad = Launchpad.login_anonymously('Juju GUI charm', 'production')
-- project = launchpad.projects['juju-gui']
-- # Find the URL of the most recently created NPM cache archive.
-- npm_cache_url = get_release_file_url(project, 'npm-cache', None)
-- return npm_cache_url
--
--
--def prime_npm_cache(npm_cache_url):
-- """Download NPM cache archive and prime the NPM cache with it."""
-- # Download the cache archive and then uncompress it into the NPM cache.
-- npm_cache_archive = os.path.join(CURRENT_DIR, 'npm-cache.tgz')
-- cmd_log(run('curl', '-L', '-o', npm_cache_archive, npm_cache_url))
-- npm_cache_dir = os.path.expanduser('~/.npm')
-- # The NPM cache directory probably does not exist, so make it if not.
-- try:
-- os.mkdir(npm_cache_dir)
-- except OSError, e:
-- # If the directory already exists then ignore the error.
-- if e.errno != errno.EEXIST: # File exists.
-- raise
-- uncompress = command('tar', '-x', '-z', '-C', npm_cache_dir, '-f')
-- cmd_log(uncompress(npm_cache_archive))
--
--
--def fetch_gui(juju_gui_source, logpath):
-- """Retrieve the Juju GUI release/branch."""
-- # Retrieve a Juju GUI release.
-- origin, version_or_branch = parse_source(juju_gui_source)
-- if origin == 'branch':
-- # Make sure we have the dependencies necessary for us to actually make
-- # a build.
-- _get_build_dependencies()
-- # Create a release starting from a branch.
-- juju_gui_source_dir = os.path.join(CURRENT_DIR, 'juju-gui-source')
-- log('Retrieving Juju GUI source checkout from %s.' % version_or_branch)
-- cmd_log(run('rm', '-rf', juju_gui_source_dir))
-- cmd_log(bzr_checkout(version_or_branch, juju_gui_source_dir))
-- log('Preparing a Juju GUI release.')
-- logdir = os.path.dirname(logpath)
-- fd, name = tempfile.mkstemp(prefix='make-distfile-', dir=logdir)
-- log('Output from "make distfile" sent to %s' % name)
-- with environ(NO_BZR='1'):
-- run('make', '-C', juju_gui_source_dir, 'distfile',
-- stdout=fd, stderr=fd)
-- release_tarball = first_path_in_dir(
-- os.path.join(juju_gui_source_dir, 'releases'))
-- else:
-- log('Retrieving Juju GUI release.')
-- if origin == 'url':
-- file_url = version_or_branch
-- else:
-- # Retrieve a release from Launchpad.
-- launchpad = Launchpad.login_anonymously(
-- 'Juju GUI charm', 'production')
-- project = launchpad.projects['juju-gui']
-- file_url = get_release_file_url(project, origin, version_or_branch)
-- log('Downloading release file from %s.' % file_url)
-- release_tarball = os.path.join(CURRENT_DIR, 'release.tgz')
-- cmd_log(run('curl', '-L', '-o', release_tarball, file_url))
-- return release_tarball
--
--
--def fetch_api(juju_api_branch):
-- """Retrieve the Juju branch."""
-- # Retrieve Juju API source checkout.
-- log('Retrieving Juju API source checkout.')
-- cmd_log(run('rm', '-rf', JUJU_DIR))
-- cmd_log(bzr_checkout(juju_api_branch, JUJU_DIR))
--
--
--def setup_gui(release_tarball):
-- """Set up Juju GUI."""
-- # Uncompress the release tarball.
-- log('Installing Juju GUI.')
-- release_dir = os.path.join(CURRENT_DIR, 'release')
-- cmd_log(run('rm', '-rf', release_dir))
-- os.mkdir(release_dir)
-- uncompress = command('tar', '-x', '-z', '-C', release_dir, '-f')
-- cmd_log(uncompress(release_tarball))
-- # Link the Juju GUI dir to the contents of the release tarball.
-- cmd_log(run('ln', '-sf', first_path_in_dir(release_dir), JUJU_GUI_DIR))
--
--
--def setup_apache():
-- """Set up apache."""
-- log('Setting up apache.')
-- if not os.path.exists(JUJU_GUI_SITE):
-- cmd_log(run('touch', JUJU_GUI_SITE))
-- cmd_log(run('chown', 'ubuntu:', JUJU_GUI_SITE))
-- cmd_log(
-- run('ln', '-s', JUJU_GUI_SITE,
-- '/etc/apache2/sites-enabled/juju-gui'))
--
-- if not os.path.exists(JUJU_GUI_PORTS):
-- cmd_log(run('touch', JUJU_GUI_PORTS))
-- cmd_log(run('chown', 'ubuntu:', JUJU_GUI_PORTS))
--
-- with su('root'):
-- run('a2dissite', 'default')
-- run('a2ensite', 'juju-gui')
--
--
--def save_or_create_certificates(
-- ssl_cert_path, ssl_cert_contents, ssl_key_contents):
-- """Generate the SSL certificates.
--
-- If both *ssl_cert_contents* and *ssl_key_contents* are provided, use them
-- as certificates; otherwise, generate them.
--
-- Also create a pem file, suitable for use in the haproxy configuration,
-- concatenating the key and the certificate files.
-- """
-- crt_path = os.path.join(ssl_cert_path, 'juju.crt')
-- key_path = os.path.join(ssl_cert_path, 'juju.key')
-- if not os.path.exists(ssl_cert_path):
-- os.makedirs(ssl_cert_path)
-- if ssl_cert_contents and ssl_key_contents:
-- # Save the provided certificates.
-- with open(crt_path, 'w') as cert_file:
-- cert_file.write(ssl_cert_contents)
-- with open(key_path, 'w') as key_file:
-- key_file.write(ssl_key_contents)
-- else:
-- # Generate certificates.
-- # See http://superuser.com/questions/226192/openssl-without-prompt
-- cmd_log(run(
-- 'openssl', 'req', '-new', '-newkey', 'rsa:4096',
-- '-days', '365', '-nodes', '-x509', '-subj',
-- # These are arbitrary test values for the certificate.
-- '/C=GB/ST=Juju/L=GUI/O=Ubuntu/CN=juju.ubuntu.com',
-- '-keyout', key_path, '-out', crt_path))
-- # Generate the pem file.
-- pem_path = os.path.join(ssl_cert_path, JUJU_PEM)
-- if os.path.exists(pem_path):
-- os.remove(pem_path)
-- with open(pem_path, 'w') as pem_file:
-- shutil.copyfileobj(open(key_path), pem_file)
-- shutil.copyfileobj(open(crt_path), pem_file)
--
--
--def find_missing_packages(*packages):
-- """Given a list of packages, return the packages which are not installed.
-- """
-- cache = apt.Cache()
-- missing = set()
-- for pkg_name in packages:
-- try:
-- pkg = cache[pkg_name]
-- except KeyError:
-- missing.add(pkg_name)
-- continue
-- if pkg.is_installed:
-- continue
-- missing.add(pkg_name)
-- return missing
--
--
--## Backend support decorators
--
--def chain(name):
-- """Helper method to compose a set of mixin objects into a callable.
--
-- Each method is called in the context of its mixin instance, and its
-- argument is the Backend instance.
-- """
-- # Chain method calls through all implementing mixins.
-- def method(self):
-- for mixin in self.mixins:
-- a_callable = getattr(type(mixin), name, None)
-- if a_callable:
-- a_callable(mixin, self)
--
-- method.__name__ = name
-- return method
--
--
--def merge(name):
-- """Helper to merge a property from a set of strategy objects
-- into a unified set.
-- """
-- # Return merged property from every providing mixin as a set.
-- @property
-- def method(self):
-- result = set()
-- for mixin in self.mixins:
-- segment = getattr(type(mixin), name, None)
-- if segment and isinstance(segment, (list, tuple, set)):
-- result |= set(segment)
--
-- return result
-- return method
 === removed directory 'hooks/charmhelpers/contrib/network'
 === removed file 'hooks/charmhelpers/contrib/network/__init__.py'
 === removed file 'hooks/charmhelpers/contrib/network/ip.py'
 --- hooks/charmhelpers/contrib/network/ip.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/network/ip.py	1970-01-01 00:00:00 +0000
@@ -1,69 +0,0 @@
--import sys
--
--from charmhelpers.fetch import apt_install
--from charmhelpers.core.hookenv import (
-- ERROR, log,
--)
--
--try:
-- import netifaces
--except ImportError:
-- apt_install('python-netifaces')
-- import netifaces
--
--try:
-- import netaddr
--except ImportError:
-- apt_install('python-netaddr')
-- import netaddr
--
--
--def _validate_cidr(network):
-- try:
-- netaddr.IPNetwork(network)
-- except (netaddr.core.AddrFormatError, ValueError):
-- raise ValueError("Network (%s) is not in CIDR presentation format" %
-- network)
--
--
--def get_address_in_network(network, fallback=None, fatal=False):
-- """
-- Get an IPv4 address within the network from the host.
--
-- Args:
-- network (str): CIDR presentation format. For example,
-- '192.168.1.0/24'.
-- fallback (str): If no address is found, return fallback.
-- fatal (boolean): If no address is found, fallback is not
-- set and fatal is True then exit(1).
-- """
--
-- def not_found_error_out():
-- log("No IP address found in network: %s" % network,
-- level=ERROR)
-- sys.exit(1)
--
-- if network is None:
-- if fallback is not None:
-- return fallback
-- else:
-- if fatal:
-- not_found_error_out()
--
-- _validate_cidr(network)
-- for iface in netifaces.interfaces():
-- addresses = netifaces.ifaddresses(iface)
-- if netifaces.AF_INET in addresses:
-- addr = addresses[netifaces.AF_INET][0]['addr']
-- netmask = addresses[netifaces.AF_INET][0]['netmask']
-- cidr = netaddr.IPNetwork("%s/%s" % (addr, netmask))
-- if cidr in netaddr.IPNetwork(network):
-- return str(cidr.ip)
--
-- if fallback is not None:
-- return fallback
--
-- if fatal:
-- not_found_error_out()
--
-- return None
 === removed directory 'hooks/charmhelpers/contrib/network/ovs'
 === removed file 'hooks/charmhelpers/contrib/network/ovs/__init__.py'
 --- hooks/charmhelpers/contrib/network/ovs/__init__.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/network/ovs/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,75 +0,0 @@
--''' Helpers for interacting with OpenvSwitch '''
--import subprocess
--import os
--from charmhelpers.core.hookenv import (
-- log, WARNING
--)
--from charmhelpers.core.host import (
-- service
--)
--
--
--def add_bridge(name):
-- ''' Add the named bridge to openvswitch '''
-- log('Creating bridge {}'.format(name))
-- subprocess.check_call(["ovs-vsctl", "--", "--may-exist", "add-br", name])
--
--
--def del_bridge(name):
-- ''' Delete the named bridge from openvswitch '''
-- log('Deleting bridge {}'.format(name))
-- subprocess.check_call(["ovs-vsctl", "--", "--if-exists", "del-br", name])
--
--
--def add_bridge_port(name, port):
-- ''' Add a port to the named openvswitch bridge '''
-- log('Adding port {} to bridge {}'.format(port, name))
-- subprocess.check_call(["ovs-vsctl", "--", "--may-exist", "add-port",
-- name, port])
-- subprocess.check_call(["ip", "link", "set", port, "up"])
--
--
--def del_bridge_port(name, port):
-- ''' Delete a port from the named openvswitch bridge '''
-- log('Deleting port {} from bridge {}'.format(port, name))
-- subprocess.check_call(["ovs-vsctl", "--", "--if-exists", "del-port",
-- name, port])
-- subprocess.check_call(["ip", "link", "set", port, "down"])
--
--
--def set_manager(manager):
-- ''' Set the controller for the local openvswitch '''
-- log('Setting manager for local ovs to {}'.format(manager))
-- subprocess.check_call(['ovs-vsctl', 'set-manager',
-- 'ssl:{}'.format(manager)])
--
--
--CERT_PATH = '/etc/openvswitch/ovsclient-cert.pem'
--
--
--def get_certificate():
-- ''' Read openvswitch certificate from disk '''
-- if os.path.exists(CERT_PATH):
-- log('Reading ovs certificate from {}'.format(CERT_PATH))
-- with open(CERT_PATH, 'r') as cert:
-- full_cert = cert.read()
-- begin_marker = "-----BEGIN CERTIFICATE-----"
-- end_marker = "-----END CERTIFICATE-----"
-- begin_index = full_cert.find(begin_marker)
-- end_index = full_cert.rfind(end_marker)
-- if end_index == -1 or begin_index == -1:
-- raise RuntimeError("Certificate does not contain valid begin"
-- " and end markers.")
-- full_cert = full_cert[begin_index:(end_index + len(end_marker))]
-- return full_cert
-- else:
-- log('Certificate not found', level=WARNING)
-- return None
--
--
--def full_restart():
-- ''' Full restart and reload of openvswitch '''
-- if os.path.exists('/etc/init/openvswitch-force-reload-kmod.conf'):
-- service('start', 'openvswitch-force-reload-kmod')
-- else:
-- service('force-reload-kmod', 'openvswitch-switch')
 === removed directory 'hooks/charmhelpers/contrib/openstack'
 === removed file 'hooks/charmhelpers/contrib/openstack/__init__.py'
 === removed file 'hooks/charmhelpers/contrib/openstack/alternatives.py'
 --- hooks/charmhelpers/contrib/openstack/alternatives.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/openstack/alternatives.py	1970-01-01 00:00:00 +0000
@@ -1,17 +0,0 @@
--''' Helper for managing alternatives for file conflict resolution '''
--
--import subprocess
--import shutil
--import os
--
--
--def install_alternative(name, target, source, priority=50):
-- ''' Install alternative configuration '''
-- if (os.path.exists(target) and not os.path.islink(target)):
-- # Move existing file/directory away before installing
-- shutil.move(target, '{}.bak'.format(target))
-- cmd = [
-- 'update-alternatives', '--force', '--install',
-- target, name, source, str(priority)
-- ]
-- subprocess.check_call(cmd)
 === removed file 'hooks/charmhelpers/contrib/openstack/context.py'
 --- hooks/charmhelpers/contrib/openstack/context.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/openstack/context.py	1970-01-01 00:00:00 +0000
@@ -1,700 +0,0 @@
--import json
--import os
--import time
--
--from base64 import b64decode
--
--from subprocess import (
-- check_call
--)
--
--
--from charmhelpers.fetch import (
-- apt_install,
-- filter_installed_packages,
--)
--
--from charmhelpers.core.hookenv import (
-- config,
-- local_unit,
-- log,
-- relation_get,
-- relation_ids,
-- related_units,
-- unit_get,
-- unit_private_ip,
-- ERROR,
--)
--
--from charmhelpers.contrib.hahelpers.cluster import (
-- determine_apache_port,
-- determine_api_port,
-- https,
-- is_clustered
--)
--
--from charmhelpers.contrib.hahelpers.apache import (
-- get_cert,
-- get_ca_cert,
--)
--
--from charmhelpers.contrib.openstack.neutron import (
-- neutron_plugin_attribute,
--)
--
--CA_CERT_PATH = '/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt'
--
--
--class OSContextError(Exception):
-- pass
--
--
--def ensure_packages(packages):
-- '''Install but do not upgrade required plugin packages'''
-- required = filter_installed_packages(packages)
-- if required:
-- apt_install(required, fatal=True)
--
--
--def context_complete(ctxt):
-- _missing = []
-- for k, v in ctxt.iteritems():
-- if v is None or v == '':
-- _missing.append(k)
-- if _missing:
-- log('Missing required data: %s' % ' '.join(_missing), level='INFO')
-- return False
-- return True
--
--
--def config_flags_parser(config_flags):
-- if config_flags.find('==') >= 0:
-- log("config_flags is not in expected format (key=value)",
-- level=ERROR)
-- raise OSContextError
-- # strip the following from each value.
-- post_strippers = ' ,'
-- # we strip any leading/trailing '=' or ' ' from the string then
-- # split on '='.
-- split = config_flags.strip(' =').split('=')
-- limit = len(split)
-- flags = {}
-- for i in xrange(0, limit - 1):
-- current = split[i]
-- next = split[i + 1]
-- vindex = next.rfind(',')
-- if (i == limit - 2) or (vindex < 0):
-- value = next
-- else:
-- value = next[:vindex]
--
-- if i == 0:
-- key = current
-- else:
-- # if this not the first entry, expect an embedded key.
-- index = current.rfind(',')
-- if index < 0:
-- log("invalid config value(s) at index %s" % (i),
-- level=ERROR)
-- raise OSContextError
-- key = current[index + 1:]
--
-- # Add to collection.
-- flags[key.strip(post_strippers)] = value.rstrip(post_strippers)
-- return flags
--
--
--class OSContextGenerator(object):
-- interfaces = []
--
-- def __call__(self):
-- raise NotImplementedError
--
--
--class SharedDBContext(OSContextGenerator):
-- interfaces = ['shared-db']
--
-- def __init__(self,
-- database=None, user=None, relation_prefix=None, ssl_dir=None):
-- '''
-- Allows inspecting relation for settings prefixed with relation_prefix.
-- This is useful for parsing access for multiple databases returned via
-- the shared-db interface (eg, nova_password, quantum_password)
-- '''
-- self.relation_prefix = relation_prefix
-- self.database = database
-- self.user = user
-- self.ssl_dir = ssl_dir
--
-- def __call__(self):
-- self.database = self.database or config('database')
-- self.user = self.user or config('database-user')
-- if None in [self.database, self.user]:
-- log('Could not generate shared_db context. '
-- 'Missing required charm config options. '
-- '(database name and user)')
-- raise OSContextError
-- ctxt = {}
--
-- password_setting = 'password'
-- if self.relation_prefix:
-- password_setting = self.relation_prefix + '_password'
--
-- for rid in relation_ids('shared-db'):
-- for unit in related_units(rid):
-- rdata = relation_get(rid=rid, unit=unit)
-- ctxt = {
-- 'database_host': rdata.get('db_host'),
-- 'database': self.database,
-- 'database_user': self.user,
-- 'database_password': rdata.get(password_setting),
-- 'database_type': 'mysql'
-- }
-- if context_complete(ctxt):
-- db_ssl(rdata, ctxt, self.ssl_dir)
-- return ctxt
-- return {}
--
--
--class PostgresqlDBContext(OSContextGenerator):
-- interfaces = ['pgsql-db']
--
-- def __init__(self, database=None):
-- self.database = database
--
-- def __call__(self):
-- self.database = self.database or config('database')
-- if self.database is None:
-- log('Could not generate postgresql_db context. '
-- 'Missing required charm config options. '
-- '(database name)')
-- raise OSContextError
-- ctxt = {}
--
-- for rid in relation_ids(self.interfaces[0]):
-- for unit in related_units(rid):
-- ctxt = {
-- 'database_host': relation_get('host', rid=rid, unit=unit),
-- 'database': self.database,
-- 'database_user': relation_get('user', rid=rid, unit=unit),
-- 'database_password': relation_get('password', rid=rid, unit=unit),
-- 'database_type': 'postgresql',
-- }
-- if context_complete(ctxt):
-- return ctxt
-- return {}
--
--
--def db_ssl(rdata, ctxt, ssl_dir):
-- if 'ssl_ca' in rdata and ssl_dir:
-- ca_path = os.path.join(ssl_dir, 'db-client.ca')
-- with open(ca_path, 'w') as fh:
-- fh.write(b64decode(rdata['ssl_ca']))
-- ctxt['database_ssl_ca'] = ca_path
-- elif 'ssl_ca' in rdata:
-- log("Charm not setup for ssl support but ssl ca found")
-- return ctxt
-- if 'ssl_cert' in rdata:
-- cert_path = os.path.join(
-- ssl_dir, 'db-client.cert')
-- if not os.path.exists(cert_path):
-- log("Waiting 1m for ssl client cert validity")
-- time.sleep(60)
-- with open(cert_path, 'w') as fh:
-- fh.write(b64decode(rdata['ssl_cert']))
-- ctxt['database_ssl_cert'] = cert_path
-- key_path = os.path.join(ssl_dir, 'db-client.key')
-- with open(key_path, 'w') as fh:
-- fh.write(b64decode(rdata['ssl_key']))
-- ctxt['database_ssl_key'] = key_path
-- return ctxt
--
--
--class IdentityServiceContext(OSContextGenerator):
-- interfaces = ['identity-service']
--
-- def __call__(self):
-- log('Generating template context for identity-service')
-- ctxt = {}
--
-- for rid in relation_ids('identity-service'):
-- for unit in related_units(rid):
-- rdata = relation_get(rid=rid, unit=unit)
-- ctxt = {
-- 'service_port': rdata.get('service_port'),
-- 'service_host': rdata.get('service_host'),
-- 'auth_host': rdata.get('auth_host'),
-- 'auth_port': rdata.get('auth_port'),
-- 'admin_tenant_name': rdata.get('service_tenant'),
-- 'admin_user': rdata.get('service_username'),
-- 'admin_password': rdata.get('service_password'),
-- 'service_protocol':
-- rdata.get('service_protocol') or 'http',
-- 'auth_protocol':
-- rdata.get('auth_protocol') or 'http',
-- }
-- if context_complete(ctxt):
-- # NOTE(jamespage) this is required for >= icehouse
-- # so a missing value just indicates keystone needs
-- # upgrading
-- ctxt['admin_tenant_id'] = rdata.get('service_tenant_id')
-- return ctxt
-- return {}
--
--
--class AMQPContext(OSContextGenerator):
-- interfaces = ['amqp']
--
-- def __init__(self, ssl_dir=None):
-- self.ssl_dir = ssl_dir
--
-- def __call__(self):
-- log('Generating template context for amqp')
-- conf = config()
-- try:
-- username = conf['rabbit-user']
-- vhost = conf['rabbit-vhost']
-- except KeyError as e:
-- log('Could not generate shared_db context. '
-- 'Missing required charm config options: %s.' % e)
-- raise OSContextError
-- ctxt = {}
-- for rid in relation_ids('amqp'):
-- ha_vip_only = False
-- for unit in related_units(rid):
-- if relation_get('clustered', rid=rid, unit=unit):
-- ctxt['clustered'] = True
-- ctxt['rabbitmq_host'] = relation_get('vip', rid=rid,
-- unit=unit)
-- else:
-- ctxt['rabbitmq_host'] = relation_get('private-address',
-- rid=rid, unit=unit)
-- ctxt.update({
-- 'rabbitmq_user': username,
-- 'rabbitmq_password': relation_get('password', rid=rid,
-- unit=unit),
-- 'rabbitmq_virtual_host': vhost,
-- })
--
-- ssl_port = relation_get('ssl_port', rid=rid, unit=unit)
-- if ssl_port:
-- ctxt['rabbit_ssl_port'] = ssl_port
-- ssl_ca = relation_get('ssl_ca', rid=rid, unit=unit)
-- if ssl_ca:
-- ctxt['rabbit_ssl_ca'] = ssl_ca
--
-- if relation_get('ha_queues', rid=rid, unit=unit) is not None:
-- ctxt['rabbitmq_ha_queues'] = True
--
-- ha_vip_only = relation_get('ha-vip-only',
-- rid=rid, unit=unit) is not None
--
-- if context_complete(ctxt):
-- if 'rabbit_ssl_ca' in ctxt:
-- if not self.ssl_dir:
-- log(("Charm not setup for ssl support "
-- "but ssl ca found"))
-- break
-- ca_path = os.path.join(
-- self.ssl_dir, 'rabbit-client-ca.pem')
-- with open(ca_path, 'w') as fh:
-- fh.write(b64decode(ctxt['rabbit_ssl_ca']))
-- ctxt['rabbit_ssl_ca'] = ca_path
-- # Sufficient information found = break out!
-- break
-- # Used for active/active rabbitmq >= grizzly
-- if ('clustered' not in ctxt or ha_vip_only) \
-- and len(related_units(rid)) > 1:
-- rabbitmq_hosts = []
-- for unit in related_units(rid):
-- rabbitmq_hosts.append(relation_get('private-address',
-- rid=rid, unit=unit))
-- ctxt['rabbitmq_hosts'] = ','.join(rabbitmq_hosts)
-- if not context_complete(ctxt):
-- return {}
-- else:
-- return ctxt
--
--
--class CephContext(OSContextGenerator):
-- interfaces = ['ceph']
--
-- def __call__(self):
-- '''This generates context for /etc/ceph/ceph.conf templates'''
-- if not relation_ids('ceph'):
-- return {}
--
-- log('Generating template context for ceph')
--
-- mon_hosts = []
-- auth = None
-- key = None
-- use_syslog = str(config('use-syslog')).lower()
-- for rid in relation_ids('ceph'):
-- for unit in related_units(rid):
-- mon_hosts.append(relation_get('private-address', rid=rid,
-- unit=unit))
-- auth = relation_get('auth', rid=rid, unit=unit)
-- key = relation_get('key', rid=rid, unit=unit)
--
-- ctxt = {
-- 'mon_hosts': ' '.join(mon_hosts),
-- 'auth': auth,
-- 'key': key,
-- 'use_syslog': use_syslog
-- }
--
-- if not os.path.isdir('/etc/ceph'):
-- os.mkdir('/etc/ceph')
--
-- if not context_complete(ctxt):
-- return {}
--
-- ensure_packages(['ceph-common'])
--
-- return ctxt
--
--
--class HAProxyContext(OSContextGenerator):
-- interfaces = ['cluster']
--
-- def __call__(self):
-- '''
-- Builds half a context for the haproxy template, which describes
-- all peers to be included in the cluster. Each charm needs to include
-- its own context generator that describes the port mapping.
-- '''
-- if not relation_ids('cluster'):
-- return {}
--
-- cluster_hosts = {}
-- l_unit = local_unit().replace('/', '-')
-- cluster_hosts[l_unit] = unit_get('private-address')
--
-- for rid in relation_ids('cluster'):
-- for unit in related_units(rid):
-- _unit = unit.replace('/', '-')
-- addr = relation_get('private-address', rid=rid, unit=unit)
-- cluster_hosts[_unit] = addr
--
-- ctxt = {
-- 'units': cluster_hosts,
-- }
-- if len(cluster_hosts.keys()) > 1:
-- # Enable haproxy when we have enough peers.
-- log('Ensuring haproxy enabled in /etc/default/haproxy.')
-- with open('/etc/default/haproxy', 'w') as out:
-- out.write('ENABLED=1\n')
-- return ctxt
-- log('HAProxy context is incomplete, this unit has no peers.')
-- return {}
--
--
--class ImageServiceContext(OSContextGenerator):
-- interfaces = ['image-service']
--
-- def __call__(self):
-- '''
-- Obtains the glance API server from the image-service relation. Useful
-- in nova and cinder (currently).
-- '''
-- log('Generating template context for image-service.')
-- rids = relation_ids('image-service')
-- if not rids:
-- return {}
-- for rid in rids:
-- for unit in related_units(rid):
-- api_server = relation_get('glance-api-server',
-- rid=rid, unit=unit)
-- if api_server:
-- return {'glance_api_servers': api_server}
-- log('ImageService context is incomplete. '
-- 'Missing required relation data.')
-- return {}
--
--
--class ApacheSSLContext(OSContextGenerator):
--
-- """
-- Generates a context for an apache vhost configuration that configures
-- HTTPS reverse proxying for one or many endpoints. Generated context
-- looks something like:
-- {
-- 'namespace': 'cinder',
-- 'private_address': 'iscsi.mycinderhost.com',
-- 'endpoints': [(8776, 8766), (8777, 8767)]
-- }
--
-- The endpoints list consists of a tuples mapping external ports
-- to internal ports.
-- """
-- interfaces = ['https']
--
-- # charms should inherit this context and set external ports
-- # and service namespace accordingly.
-- external_ports = []
-- service_namespace = None
--
-- def enable_modules(self):
-- cmd = ['a2enmod', 'ssl', 'proxy', 'proxy_http']
-- check_call(cmd)
--
-- def configure_cert(self):
-- if not os.path.isdir('/etc/apache2/ssl'):
-- os.mkdir('/etc/apache2/ssl')
-- ssl_dir = os.path.join('/etc/apache2/ssl/', self.service_namespace)
-- if not os.path.isdir(ssl_dir):
-- os.mkdir(ssl_dir)
-- cert, key = get_cert()
-- with open(os.path.join(ssl_dir, 'cert'), 'w') as cert_out:
-- cert_out.write(b64decode(cert))
-- with open(os.path.join(ssl_dir, 'key'), 'w') as key_out:
-- key_out.write(b64decode(key))
-- ca_cert = get_ca_cert()
-- if ca_cert:
-- with open(CA_CERT_PATH, 'w') as ca_out:
-- ca_out.write(b64decode(ca_cert))
-- check_call(['update-ca-certificates'])
--
-- def __call__(self):
-- if isinstance(self.external_ports, basestring):
-- self.external_ports = [self.external_ports]
-- if (not self.external_ports or not https()):
-- return {}
--
-- self.configure_cert()
-- self.enable_modules()
--
-- ctxt = {
-- 'namespace': self.service_namespace,
-- 'private_address': unit_get('private-address'),
-- 'endpoints': []
-- }
-- if is_clustered():
-- ctxt['private_address'] = config('vip')
-- for api_port in self.external_ports:
-- ext_port = determine_apache_port(api_port)
-- int_port = determine_api_port(api_port)
-- portmap = (int(ext_port), int(int_port))
-- ctxt['endpoints'].append(portmap)
-- return ctxt
--
--
--class NeutronContext(OSContextGenerator):
-- interfaces = []
--
-- @property
-- def plugin(self):
-- return None
--
-- @property
-- def network_manager(self):
-- return None
--
-- @property
-- def packages(self):
-- return neutron_plugin_attribute(
-- self.plugin, 'packages', self.network_manager)
--
-- @property
-- def neutron_security_groups(self):
-- return None
--
-- def _ensure_packages(self):
-- [ensure_packages(pkgs) for pkgs in self.packages]
--
-- def _save_flag_file(self):
-- if self.network_manager == 'quantum':
-- _file = '/etc/nova/quantum_plugin.conf'
-- else:
-- _file = '/etc/nova/neutron_plugin.conf'
-- with open(_file, 'wb') as out:
-- out.write(self.plugin + '\n')
--
-- def ovs_ctxt(self):
-- driver = neutron_plugin_attribute(self.plugin, 'driver',
-- self.network_manager)
-- config = neutron_plugin_attribute(self.plugin, 'config',
-- self.network_manager)
-- ovs_ctxt = {
-- 'core_plugin': driver,
-- 'neutron_plugin': 'ovs',
-- 'neutron_security_groups': self.neutron_security_groups,
-- 'local_ip': unit_private_ip(),
-- 'config': config
-- }
--
-- return ovs_ctxt
--
-- def nvp_ctxt(self):
-- driver = neutron_plugin_attribute(self.plugin, 'driver',
-- self.network_manager)
-- config = neutron_plugin_attribute(self.plugin, 'config',
-- self.network_manager)
-- nvp_ctxt = {
-- 'core_plugin': driver,
-- 'neutron_plugin': 'nvp',
-- 'neutron_security_groups': self.neutron_security_groups,
-- 'local_ip': unit_private_ip(),
-- 'config': config
-- }
--
-- return nvp_ctxt
--
-- def neutron_ctxt(self):
-- if https():
-- proto = 'https'
-- else:
-- proto = 'http'
-- if is_clustered():
-- host = config('vip')
-- else:
-- host = unit_get('private-address')
-- url = '%s://%s:%s' % (proto, host, '9696')
-- ctxt = {
-- 'network_manager': self.network_manager,
-- 'neutron_url': url,
-- }
-- return ctxt
--
-- def __call__(self):
-- self._ensure_packages()
--
-- if self.network_manager not in ['quantum', 'neutron']:
-- return {}
--
-- if not self.plugin:
-- return {}
--
-- ctxt = self.neutron_ctxt()
--
-- if self.plugin == 'ovs':
-- ctxt.update(self.ovs_ctxt())
-- elif self.plugin == 'nvp':
-- ctxt.update(self.nvp_ctxt())
--
-- alchemy_flags = config('neutron-alchemy-flags')
-- if alchemy_flags:
-- flags = config_flags_parser(alchemy_flags)
-- ctxt['neutron_alchemy_flags'] = flags
--
-- self._save_flag_file()
-- return ctxt
--
--
--class OSConfigFlagContext(OSContextGenerator):
--
-- """
-- Responsible for adding user-defined config-flags in charm config to a
-- template context.
--
-- NOTE: the value of config-flags may be a comma-separated list of
-- key=value pairs and some Openstack config files support
-- comma-separated lists as values.
-- """
--
-- def __call__(self):
-- config_flags = config('config-flags')
-- if not config_flags:
-- return {}
--
-- flags = config_flags_parser(config_flags)
-- return {'user_config_flags': flags}
--
--
--class SubordinateConfigContext(OSContextGenerator):
--
-- """
-- Responsible for inspecting relations to subordinates that
-- may be exporting required config via a json blob.
--
-- The subordinate interface allows subordinates to export their
-- configuration requirements to the principle for multiple config
-- files and multiple serivces. Ie, a subordinate that has interfaces
-- to both glance and nova may export to following yaml blob as json:
--
-- glance:
-- /etc/glance/glance-api.conf:
-- sections:
-- DEFAULT:
-- - [key1, value1]
-- /etc/glance/glance-registry.conf:
-- MYSECTION:
-- - [key2, value2]
-- nova:
-- /etc/nova/nova.conf:
-- sections:
-- DEFAULT:
-- - [key3, value3]
--
--
-- It is then up to the principle charms to subscribe this context to
-- the service+config file it is interestd in. Configuration data will
-- be available in the template context, in glance's case, as:
-- ctxt = {
-- ... other context ...
-- 'subordinate_config': {
-- 'DEFAULT': {
-- 'key1': 'value1',
-- },
-- 'MYSECTION': {
-- 'key2': 'value2',
-- },
-- }
-- }
--
-- """
--
-- def __init__(self, service, config_file, interface):
-- """
-- :param service : Service name key to query in any subordinate
-- data found
-- :param config_file : Service's config file to query sections
-- :param interface : Subordinate interface to inspect
-- """
-- self.service = service
-- self.config_file = config_file
-- self.interface = interface
--
-- def __call__(self):
-- ctxt = {}
-- for rid in relation_ids(self.interface):
-- for unit in related_units(rid):
-- sub_config = relation_get('subordinate_configuration',
-- rid=rid, unit=unit)
-- if sub_config and sub_config != '':
-- try:
-- sub_config = json.loads(sub_config)
-- except:
-- log('Could not parse JSON from subordinate_config '
-- 'setting from %s' % rid, level=ERROR)
-- continue
--
-- if self.service not in sub_config:
-- log('Found subordinate_config on %s but it contained'
-- 'nothing for %s service' % (rid, self.service))
-- continue
--
-- sub_config = sub_config[self.service]
-- if self.config_file not in sub_config:
-- log('Found subordinate_config on %s but it contained'
-- 'nothing for %s' % (rid, self.config_file))
-- continue
--
-- sub_config = sub_config[self.config_file]
-- for k, v in sub_config.iteritems():
-- ctxt[k] = v
--
-- if not ctxt:
-- ctxt['sections'] = {}
--
-- return ctxt
--
--
--class SyslogContext(OSContextGenerator):
--
-- def __call__(self):
-- ctxt = {
-- 'use_syslog': config('use-syslog')
-- }
-- return ctxt
 === removed file 'hooks/charmhelpers/contrib/openstack/neutron.py'
 --- hooks/charmhelpers/contrib/openstack/neutron.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/openstack/neutron.py	1970-01-01 00:00:00 +0000
@@ -1,171 +0,0 @@
--# Various utilies for dealing with Neutron and the renaming from Quantum.
--
--from subprocess import check_output
--
--from charmhelpers.core.hookenv import (
-- config,
-- log,
-- ERROR,
--)
--
--from charmhelpers.contrib.openstack.utils import os_release
--
--
--def headers_package():
-- """Ensures correct linux-headers for running kernel are installed,
-- for building DKMS package"""
-- kver = check_output(['uname', '-r']).strip()
-- return 'linux-headers-%s' % kver
--
--QUANTUM_CONF_DIR = '/etc/quantum'
--
--
--def kernel_version():
-- """ Retrieve the current major kernel version as a tuple e.g. (3, 13) """
-- kver = check_output(['uname', '-r']).strip()
-- kver = kver.split('.')
-- return (int(kver[0]), int(kver[1]))
--
--
--def determine_dkms_package():
-- """ Determine which DKMS package should be used based on kernel version """
-- # NOTE: 3.13 kernels have support for GRE and VXLAN native
-- if kernel_version() >= (3, 13):
-- return []
-- else:
-- return ['openvswitch-datapath-dkms']
--
--
--# legacy
--
--
--def quantum_plugins():
-- from charmhelpers.contrib.openstack import context
-- return {
-- 'ovs': {
-- 'config': '/etc/quantum/plugins/openvswitch/'
-- 'ovs_quantum_plugin.ini',
-- 'driver': 'quantum.plugins.openvswitch.ovs_quantum_plugin.'
-- 'OVSQuantumPluginV2',
-- 'contexts': [
-- context.SharedDBContext(user=config('neutron-database-user'),
-- database=config('neutron-database'),
-- relation_prefix='neutron',
-- ssl_dir=QUANTUM_CONF_DIR)],
-- 'services': ['quantum-plugin-openvswitch-agent'],
-- 'packages': [[headers_package()] + determine_dkms_package(),
-- ['quantum-plugin-openvswitch-agent']],
-- 'server_packages': ['quantum-server',
-- 'quantum-plugin-openvswitch'],
-- 'server_services': ['quantum-server']
-- },
-- 'nvp': {
-- 'config': '/etc/quantum/plugins/nicira/nvp.ini',
-- 'driver': 'quantum.plugins.nicira.nicira_nvp_plugin.'
-- 'QuantumPlugin.NvpPluginV2',
-- 'contexts': [
-- context.SharedDBContext(user=config('neutron-database-user'),
-- database=config('neutron-database'),
-- relation_prefix='neutron',
-- ssl_dir=QUANTUM_CONF_DIR)],
-- 'services': [],
-- 'packages': [],
-- 'server_packages': ['quantum-server',
-- 'quantum-plugin-nicira'],
-- 'server_services': ['quantum-server']
-- }
-- }
--
--NEUTRON_CONF_DIR = '/etc/neutron'
--
--
--def neutron_plugins():
-- from charmhelpers.contrib.openstack import context
-- release = os_release('nova-common')
-- plugins = {
-- 'ovs': {
-- 'config': '/etc/neutron/plugins/openvswitch/'
-- 'ovs_neutron_plugin.ini',
-- 'driver': 'neutron.plugins.openvswitch.ovs_neutron_plugin.'
-- 'OVSNeutronPluginV2',
-- 'contexts': [
-- context.SharedDBContext(user=config('neutron-database-user'),
-- database=config('neutron-database'),
-- relation_prefix='neutron',
-- ssl_dir=NEUTRON_CONF_DIR)],
-- 'services': ['neutron-plugin-openvswitch-agent'],
-- 'packages': [[headers_package()] + determine_dkms_package(),
-- ['neutron-plugin-openvswitch-agent']],
-- 'server_packages': ['neutron-server',
-- 'neutron-plugin-openvswitch'],
-- 'server_services': ['neutron-server']
-- },
-- 'nvp': {
-- 'config': '/etc/neutron/plugins/nicira/nvp.ini',
-- 'driver': 'neutron.plugins.nicira.nicira_nvp_plugin.'
-- 'NeutronPlugin.NvpPluginV2',
-- 'contexts': [
-- context.SharedDBContext(user=config('neutron-database-user'),
-- database=config('neutron-database'),
-- relation_prefix='neutron',
-- ssl_dir=NEUTRON_CONF_DIR)],
-- 'services': [],
-- 'packages': [],
-- 'server_packages': ['neutron-server',
-- 'neutron-plugin-nicira'],
-- 'server_services': ['neutron-server']
-- }
-- }
-- # NOTE: patch in ml2 plugin for icehouse onwards
-- if release >= 'icehouse':
-- plugins['ovs']['config'] = '/etc/neutron/plugins/ml2/ml2_conf.ini'
-- plugins['ovs']['driver'] = 'neutron.plugins.ml2.plugin.Ml2Plugin'
-- plugins['ovs']['server_packages'] = ['neutron-server',
-- 'neutron-plugin-ml2']
-- return plugins
--
--
--def neutron_plugin_attribute(plugin, attr, net_manager=None):
-- manager = net_manager or network_manager()
-- if manager == 'quantum':
-- plugins = quantum_plugins()
-- elif manager == 'neutron':
-- plugins = neutron_plugins()
-- else:
-- log('Error: Network manager does not support plugins.')
-- raise Exception
--
-- try:
-- _plugin = plugins[plugin]
-- except KeyError:
-- log('Unrecognised plugin for %s: %s' % (manager, plugin), level=ERROR)
-- raise Exception
--
-- try:
-- return _plugin[attr]
-- except KeyError:
-- return None
--
--
--def network_manager():
-- '''
-- Deals with the renaming of Quantum to Neutron in H and any situations
-- that require compatability (eg, deploying H with network-manager=quantum,
-- upgrading from G).
-- '''
-- release = os_release('nova-common')
-- manager = config('network-manager').lower()
--
-- if manager not in ['quantum', 'neutron']:
-- return manager
--
-- if release in ['essex']:
-- # E does not support neutron
-- log('Neutron networking not supported in Essex.', level=ERROR)
-- raise Exception
-- elif release in ['folsom', 'grizzly']:
-- # neutron is named quantum in F and G
-- return 'quantum'
-- else:
-- # ensure accurate naming for all releases post-H
-- return 'neutron'
 === removed directory 'hooks/charmhelpers/contrib/openstack/templates'
 === removed file 'hooks/charmhelpers/contrib/openstack/templates/__init__.py'
 --- hooks/charmhelpers/contrib/openstack/templates/__init__.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/openstack/templates/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,2 +0,0 @@
--# dummy __init__.py to fool syncer into thinking this is a syncable python
--# module
 === removed file 'hooks/charmhelpers/contrib/openstack/templating.py'
 --- hooks/charmhelpers/contrib/openstack/templating.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/openstack/templating.py	1970-01-01 00:00:00 +0000
@@ -1,280 +0,0 @@
--import os
--
--from charmhelpers.fetch import apt_install
--
--from charmhelpers.core.hookenv import (
-- log,
-- ERROR,
-- INFO
--)
--
--from charmhelpers.contrib.openstack.utils import OPENSTACK_CODENAMES
--
--try:
-- from jinja2 import FileSystemLoader, ChoiceLoader, Environment, exceptions
--except ImportError:
-- # python-jinja2 may not be installed yet, or we're running unittests.
-- FileSystemLoader = ChoiceLoader = Environment = exceptions = None
--
--
--class OSConfigException(Exception):
-- pass
--
--
--def get_loader(templates_dir, os_release):
-- """
-- Create a jinja2.ChoiceLoader containing template dirs up to
-- and including os_release. If directory template directory
-- is missing at templates_dir, it will be omitted from the loader.
-- templates_dir is added to the bottom of the search list as a base
-- loading dir.
--
-- A charm may also ship a templates dir with this module
-- and it will be appended to the bottom of the search list, eg:
-- hooks/charmhelpers/contrib/openstack/templates.
--
-- :param templates_dir: str: Base template directory containing release
-- sub-directories.
-- :param os_release : str: OpenStack release codename to construct template
-- loader.
--
-- :returns : jinja2.ChoiceLoader constructed with a list of
-- jinja2.FilesystemLoaders, ordered in descending
-- order by OpenStack release.
-- """
-- tmpl_dirs = [(rel, os.path.join(templates_dir, rel))
-- for rel in OPENSTACK_CODENAMES.itervalues()]
--
-- if not os.path.isdir(templates_dir):
-- log('Templates directory not found @ %s.' % templates_dir,
-- level=ERROR)
-- raise OSConfigException
--
-- # the bottom contains tempaltes_dir and possibly a common templates dir
-- # shipped with the helper.
-- loaders = [FileSystemLoader(templates_dir)]
-- helper_templates = os.path.join(os.path.dirname(__file__), 'templates')
-- if os.path.isdir(helper_templates):
-- loaders.append(FileSystemLoader(helper_templates))
--
-- for rel, tmpl_dir in tmpl_dirs:
-- if os.path.isdir(tmpl_dir):
-- loaders.insert(0, FileSystemLoader(tmpl_dir))
-- if rel == os_release:
-- break
-- log('Creating choice loader with dirs: %s' %
-- [l.searchpath for l in loaders], level=INFO)
-- return ChoiceLoader(loaders)
--
--
--class OSConfigTemplate(object):
-- """
-- Associates a config file template with a list of context generators.
-- Responsible for constructing a template context based on those generators.
-- """
-- def __init__(self, config_file, contexts):
-- self.config_file = config_file
--
-- if hasattr(contexts, '__call__'):
-- self.contexts = [contexts]
-- else:
-- self.contexts = contexts
--
-- self._complete_contexts = []
--
-- def context(self):
-- ctxt = {}
-- for context in self.contexts:
-- _ctxt = context()
-- if _ctxt:
-- ctxt.update(_ctxt)
-- # track interfaces for every complete context.
-- [self._complete_contexts.append(interface)
-- for interface in context.interfaces
-- if interface not in self._complete_contexts]
-- return ctxt
--
-- def complete_contexts(self):
-- '''
-- Return a list of interfaces that have atisfied contexts.
-- '''
-- if self._complete_contexts:
-- return self._complete_contexts
-- self.context()
-- return self._complete_contexts
--
--
--class OSConfigRenderer(object):
-- """
-- This class provides a common templating system to be used by OpenStack
-- charms. It is intended to help charms share common code and templates,
-- and ease the burden of managing config templates across multiple OpenStack
-- releases.
--
-- Basic usage:
-- # import some common context generates from charmhelpers
-- from charmhelpers.contrib.openstack import context
--
-- # Create a renderer object for a specific OS release.
-- configs = OSConfigRenderer(templates_dir='/tmp/templates',
-- openstack_release='folsom')
-- # register some config files with context generators.
-- configs.register(config_file='/etc/nova/nova.conf',
-- contexts=[context.SharedDBContext(),
-- context.AMQPContext()])
-- configs.register(config_file='/etc/nova/api-paste.ini',
-- contexts=[context.IdentityServiceContext()])
-- configs.register(config_file='/etc/haproxy/haproxy.conf',
-- contexts=[context.HAProxyContext()])
-- # write out a single config
-- configs.write('/etc/nova/nova.conf')
-- # write out all registered configs
-- configs.write_all()
--
-- Details:
--
-- OpenStack Releases and template loading
-- ---------------------------------------
-- When the object is instantiated, it is associated with a specific OS
-- release. This dictates how the template loader will be constructed.
--
-- The constructed loader attempts to load the template from several places
-- in the following order:
-- - from the most recent OS release-specific template dir (if one exists)
-- - the base templates_dir
-- - a template directory shipped in the charm with this helper file.
--
--
-- For the example above, '/tmp/templates' contains the following structure:
-- /tmp/templates/nova.conf
-- /tmp/templates/api-paste.ini
-- /tmp/templates/grizzly/api-paste.ini
-- /tmp/templates/havana/api-paste.ini
--
-- Since it was registered with the grizzly release, it first seraches
-- the grizzly directory for nova.conf, then the templates dir.
--
-- When writing api-paste.ini, it will find the template in the grizzly
-- directory.
--
-- If the object were created with folsom, it would fall back to the
-- base templates dir for its api-paste.ini template.
--
-- This system should help manage changes in config files through
-- openstack releases, allowing charms to fall back to the most recently
-- updated config template for a given release
--
-- The haproxy.conf, since it is not shipped in the templates dir, will
-- be loaded from the module directory's template directory, eg
-- $CHARM/hooks/charmhelpers/contrib/openstack/templates. This allows
-- us to ship common templates (haproxy, apache) with the helpers.
--
-- Context generators
-- ---------------------------------------
-- Context generators are used to generate template contexts during hook
-- execution. Doing so may require inspecting service relations, charm
-- config, etc. When registered, a config file is associated with a list
-- of generators. When a template is rendered and written, all context
-- generates are called in a chain to generate the context dictionary
-- passed to the jinja2 template. See context.py for more info.
-- """
-- def __init__(self, templates_dir, openstack_release):
-- if not os.path.isdir(templates_dir):
-- log('Could not locate templates dir %s' % templates_dir,
-- level=ERROR)
-- raise OSConfigException
--
-- self.templates_dir = templates_dir
-- self.openstack_release = openstack_release
-- self.templates = {}
-- self._tmpl_env = None
--
-- if None in [Environment, ChoiceLoader, FileSystemLoader]:
-- # if this code is running, the object is created pre-install hook.
-- # jinja2 shouldn't get touched until the module is reloaded on next
-- # hook execution, with proper jinja2 bits successfully imported.
-- apt_install('python-jinja2')
--
-- def register(self, config_file, contexts):
-- """
-- Register a config file with a list of context generators to be called
-- during rendering.
-- """
-- self.templates[config_file] = OSConfigTemplate(config_file=config_file,
-- contexts=contexts)
-- log('Registered config file: %s' % config_file, level=INFO)
--
-- def _get_tmpl_env(self):
-- if not self._tmpl_env:
-- loader = get_loader(self.templates_dir, self.openstack_release)
-- self._tmpl_env = Environment(loader=loader)
--
-- def _get_template(self, template):
-- self._get_tmpl_env()
-- template = self._tmpl_env.get_template(template)
-- log('Loaded template from %s' % template.filename, level=INFO)
-- return template
--
-- def render(self, config_file):
-- if config_file not in self.templates:
-- log('Config not registered: %s' % config_file, level=ERROR)
-- raise OSConfigException
-- ctxt = self.templates[config_file].context()
--
-- _tmpl = os.path.basename(config_file)
-- try:
-- template = self._get_template(_tmpl)
-- except exceptions.TemplateNotFound:
-- # if no template is found with basename, try looking for it
-- # using a munged full path, eg:
-- # /etc/apache2/apache2.conf -> etc_apache2_apache2.conf
-- _tmpl = '_'.join(config_file.split('/')[1:])
-- try:
-- template = self._get_template(_tmpl)
-- except exceptions.TemplateNotFound as e:
-- log('Could not load template from %s by %s or %s.' %
-- (self.templates_dir, os.path.basename(config_file), _tmpl),
-- level=ERROR)
-- raise e
--
-- log('Rendering from template: %s' % _tmpl, level=INFO)
-- return template.render(ctxt)
--
-- def write(self, config_file):
-- """
-- Write a single config file, raises if config file is not registered.
-- """
-- if config_file not in self.templates:
-- log('Config not registered: %s' % config_file, level=ERROR)
-- raise OSConfigException
--
-- _out = self.render(config_file)
--
-- with open(config_file, 'wb') as out:
-- out.write(_out)
--
-- log('Wrote template %s.' % config_file, level=INFO)
--
-- def write_all(self):
-- """
-- Write out all registered config files.
-- """
-- [self.write(k) for k in self.templates.iterkeys()]
--
-- def set_release(self, openstack_release):
-- """
-- Resets the template environment and generates a new template loader
-- based on a the new openstack release.
-- """
-- self._tmpl_env = None
-- self.openstack_release = openstack_release
-- self._get_tmpl_env()
--
-- def complete_contexts(self):
-- '''
-- Returns a list of context interfaces that yield a complete context.
-- '''
-- interfaces = []
-- [interfaces.extend(i.complete_contexts())
-- for i in self.templates.itervalues()]
-- return interfaces
 === removed file 'hooks/charmhelpers/contrib/openstack/utils.py'
 --- hooks/charmhelpers/contrib/openstack/utils.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/openstack/utils.py	1970-01-01 00:00:00 +0000
@@ -1,450 +0,0 @@
--#!/usr/bin/python
--
--# Common python helper functions used for OpenStack charms.
--from collections import OrderedDict
--
--import apt_pkg as apt
--import subprocess
--import os
--import socket
--import sys
--
--from charmhelpers.core.hookenv import (
-- config,
-- log as juju_log,
-- charm_dir,
-- ERROR,
-- INFO
--)
--
--from charmhelpers.contrib.storage.linux.lvm import (
-- deactivate_lvm_volume_group,
-- is_lvm_physical_volume,
-- remove_lvm_physical_volume,
--)
--
--from charmhelpers.core.host import lsb_release, mounts, umount
--from charmhelpers.fetch import apt_install
--from charmhelpers.contrib.storage.linux.utils import is_block_device, zap_disk
--from charmhelpers.contrib.storage.linux.loopback import ensure_loopback_device
--
--CLOUD_ARCHIVE_URL = "http://ubuntu-cloud.archive.canonical.com/ubuntu"
--CLOUD_ARCHIVE_KEY_ID = '5EDB1B62EC4926EA'
--
--DISTRO_PROPOSED = ('deb http://archive.ubuntu.com/ubuntu/ %s-proposed '
-- 'restricted main multiverse universe')
--
--
--UBUNTU_OPENSTACK_RELEASE = OrderedDict([
-- ('oneiric', 'diablo'),
-- ('precise', 'essex'),
-- ('quantal', 'folsom'),
-- ('raring', 'grizzly'),
-- ('saucy', 'havana'),
-- ('trusty', 'icehouse')
--])
--
--
--OPENSTACK_CODENAMES = OrderedDict([
-- ('2011.2', 'diablo'),
-- ('2012.1', 'essex'),
-- ('2012.2', 'folsom'),
-- ('2013.1', 'grizzly'),
-- ('2013.2', 'havana'),
-- ('2014.1', 'icehouse'),
--])
--
--# The ugly duckling
--SWIFT_CODENAMES = OrderedDict([
-- ('1.4.3', 'diablo'),
-- ('1.4.8', 'essex'),
-- ('1.7.4', 'folsom'),
-- ('1.8.0', 'grizzly'),
-- ('1.7.7', 'grizzly'),
-- ('1.7.6', 'grizzly'),
-- ('1.10.0', 'havana'),
-- ('1.9.1', 'havana'),
-- ('1.9.0', 'havana'),
-- ('1.13.1', 'icehouse'),
-- ('1.13.0', 'icehouse'),
-- ('1.12.0', 'icehouse'),
-- ('1.11.0', 'icehouse'),
--])
--
--DEFAULT_LOOPBACK_SIZE = '5G'
--
--
--def error_out(msg):
-- juju_log("FATAL ERROR: %s" % msg, level='ERROR')
-- sys.exit(1)
--
--
--def get_os_codename_install_source(src):
-- '''Derive OpenStack release codename from a given installation source.'''
-- ubuntu_rel = lsb_release()['DISTRIB_CODENAME']
-- rel = ''
-- if src in ['distro', 'distro-proposed']:
-- try:
-- rel = UBUNTU_OPENSTACK_RELEASE[ubuntu_rel]
-- except KeyError:
-- e = 'Could not derive openstack release for '\
-- 'this Ubuntu release: %s' % ubuntu_rel
-- error_out(e)
-- return rel
--
-- if src.startswith('cloud:'):
-- ca_rel = src.split(':')[1]
-- ca_rel = ca_rel.split('%s-' % ubuntu_rel)[1].split('/')[0]
-- return ca_rel
--
-- # Best guess match based on deb string provided
-- if src.startswith('deb') or src.startswith('ppa'):
-- for k, v in OPENSTACK_CODENAMES.iteritems():
-- if v in src:
-- return v
--
--
--def get_os_version_install_source(src):
-- codename = get_os_codename_install_source(src)
-- return get_os_version_codename(codename)
--
--
--def get_os_codename_version(vers):
-- '''Determine OpenStack codename from version number.'''
-- try:
-- return OPENSTACK_CODENAMES[vers]
-- except KeyError:
-- e = 'Could not determine OpenStack codename for version %s' % vers
-- error_out(e)
--
--
--def get_os_version_codename(codename):
-- '''Determine OpenStack version number from codename.'''
-- for k, v in OPENSTACK_CODENAMES.iteritems():
-- if v == codename:
-- return k
-- e = 'Could not derive OpenStack version for '\
-- 'codename: %s' % codename
-- error_out(e)
--
--
--def get_os_codename_package(package, fatal=True):
-- '''Derive OpenStack release codename from an installed package.'''
-- apt.init()
-- cache = apt.Cache()
--
-- try:
-- pkg = cache[package]
-- except:
-- if not fatal:
-- return None
-- # the package is unknown to the current apt cache.
-- e = 'Could not determine version of package with no installation '\
-- 'candidate: %s' % package
-- error_out(e)
--
-- if not pkg.current_ver:
-- if not fatal:
-- return None
-- # package is known, but no version is currently installed.
-- e = 'Could not determine version of uninstalled package: %s' % package
-- error_out(e)
--
-- vers = apt.upstream_version(pkg.current_ver.ver_str)
--
-- try:
-- if 'swift' in pkg.name:
-- swift_vers = vers[:5]
-- if swift_vers not in SWIFT_CODENAMES:
-- # Deal with 1.10.0 upward
-- swift_vers = vers[:6]
-- return SWIFT_CODENAMES[swift_vers]
-- else:
-- vers = vers[:6]
-- return OPENSTACK_CODENAMES[vers]
-- except KeyError:
-- e = 'Could not determine OpenStack codename for version %s' % vers
-- error_out(e)
--
--
--def get_os_version_package(pkg, fatal=True):
-- '''Derive OpenStack version number from an installed package.'''
-- codename = get_os_codename_package(pkg, fatal=fatal)
--
-- if not codename:
-- return None
--
-- if 'swift' in pkg:
-- vers_map = SWIFT_CODENAMES
-- else:
-- vers_map = OPENSTACK_CODENAMES
--
-- for version, cname in vers_map.iteritems():
-- if cname == codename:
-- return version
-- #e = "Could not determine OpenStack version for package: %s" % pkg
-- #error_out(e)
--
--
--os_rel = None
--
--
--def os_release(package, base='essex'):
-- '''
-- Returns OpenStack release codename from a cached global.
-- If the codename can not be determined from either an installed package or
-- the installation source, the earliest release supported by the charm should
-- be returned.
-- '''
-- global os_rel
-- if os_rel:
-- return os_rel
-- os_rel = (get_os_codename_package(package, fatal=False) or
-- get_os_codename_install_source(config('openstack-origin')) or
-- base)
-- return os_rel
--
--
--def import_key(keyid):
-- cmd = "apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 " \
-- "--recv-keys %s" % keyid
-- try:
-- subprocess.check_call(cmd.split(' '))
-- except subprocess.CalledProcessError:
-- error_out("Error importing repo key %s" % keyid)
--
--
--def configure_installation_source(rel):
-- '''Configure apt installation source.'''
-- if rel == 'distro':
-- return
-- elif rel == 'distro-proposed':
-- ubuntu_rel = lsb_release()['DISTRIB_CODENAME']
-- with open('/etc/apt/sources.list.d/juju_deb.list', 'w') as f:
-- f.write(DISTRO_PROPOSED % ubuntu_rel)
-- elif rel[:4] == "ppa:":
-- src = rel
-- subprocess.check_call(["add-apt-repository", "-y", src])
-- elif rel[:3] == "deb":
-- l = len(rel.split('|'))
-- if l == 2:
-- src, key = rel.split('|')
-- juju_log("Importing PPA key from keyserver for %s" % src)
-- import_key(key)
-- elif l == 1:
-- src = rel
-- with open('/etc/apt/sources.list.d/juju_deb.list', 'w') as f:
-- f.write(src)
-- elif rel[:6] == 'cloud:':
-- ubuntu_rel = lsb_release()['DISTRIB_CODENAME']
-- rel = rel.split(':')[1]
-- u_rel = rel.split('-')[0]
-- ca_rel = rel.split('-')[1]
--
-- if u_rel != ubuntu_rel:
-- e = 'Cannot install from Cloud Archive pocket %s on this Ubuntu '\
-- 'version (%s)' % (ca_rel, ubuntu_rel)
-- error_out(e)
--
-- if 'staging' in ca_rel:
-- # staging is just a regular PPA.
-- os_rel = ca_rel.split('/')[0]
-- ppa = 'ppa:ubuntu-cloud-archive/%s-staging' % os_rel
-- cmd = 'add-apt-repository -y %s' % ppa
-- subprocess.check_call(cmd.split(' '))
-- return
--
-- # map charm config options to actual archive pockets.
-- pockets = {
-- 'folsom': 'precise-updates/folsom',
-- 'folsom/updates': 'precise-updates/folsom',
-- 'folsom/proposed': 'precise-proposed/folsom',
-- 'grizzly': 'precise-updates/grizzly',
-- 'grizzly/updates': 'precise-updates/grizzly',
-- 'grizzly/proposed': 'precise-proposed/grizzly',
-- 'havana': 'precise-updates/havana',
-- 'havana/updates': 'precise-updates/havana',
-- 'havana/proposed': 'precise-proposed/havana',
-- 'icehouse': 'precise-updates/icehouse',
-- 'icehouse/updates': 'precise-updates/icehouse',
-- 'icehouse/proposed': 'precise-proposed/icehouse',
-- }
--
-- try:
-- pocket = pockets[ca_rel]
-- except KeyError:
-- e = 'Invalid Cloud Archive release specified: %s' % rel
-- error_out(e)
--
-- src = "deb %s %s main" % (CLOUD_ARCHIVE_URL, pocket)
-- apt_install('ubuntu-cloud-keyring', fatal=True)
--
-- with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as f:
-- f.write(src)
-- else:
-- error_out("Invalid openstack-release specified: %s" % rel)
--
--
--def save_script_rc(script_path="scripts/scriptrc", **env_vars):
-- """
-- Write an rc file in the charm-delivered directory containing
-- exported environment variables provided by env_vars. Any charm scripts run
-- outside the juju hook environment can source this scriptrc to obtain
-- updated config information necessary to perform health checks or
-- service changes.
-- """
-- juju_rc_path = "%s/%s" % (charm_dir(), script_path)
-- if not os.path.exists(os.path.dirname(juju_rc_path)):
-- os.mkdir(os.path.dirname(juju_rc_path))
-- with open(juju_rc_path, 'wb') as rc_script:
-- rc_script.write(
-- "#!/bin/bash\n")
-- [rc_script.write('export %s=%s\n' % (u, p))
-- for u, p in env_vars.iteritems() if u != "script_path"]
--
--
--def openstack_upgrade_available(package):
-- """
-- Determines if an OpenStack upgrade is available from installation
-- source, based on version of installed package.
--
-- :param package: str: Name of installed package.
--
-- :returns: bool: : Returns True if configured installation source offers
-- a newer version of package.
--
-- """
--
-- src = config('openstack-origin')
-- cur_vers = get_os_version_package(package)
-- available_vers = get_os_version_install_source(src)
-- apt.init()
-- return apt.version_compare(available_vers, cur_vers) == 1
--
--
--def ensure_block_device(block_device):
-- '''
-- Confirm block_device, create as loopback if necessary.
--
-- :param block_device: str: Full path of block device to ensure.
--
-- :returns: str: Full path of ensured block device.
-- '''
-- _none = ['None', 'none', None]
-- if (block_device in _none):
-- error_out('prepare_storage(): Missing required input: '
-- 'block_device=%s.' % block_device, level=ERROR)
--
-- if block_device.startswith('/dev/'):
-- bdev = block_device
-- elif block_device.startswith('/'):
-- _bd = block_device.split('|')
-- if len(_bd) == 2:
-- bdev, size = _bd
-- else:
-- bdev = block_device
-- size = DEFAULT_LOOPBACK_SIZE
-- bdev = ensure_loopback_device(bdev, size)
-- else:
-- bdev = '/dev/%s' % block_device
--
-- if not is_block_device(bdev):
-- error_out('Failed to locate valid block device at %s' % bdev,
-- level=ERROR)
--
-- return bdev
--
--
--def clean_storage(block_device):
-- '''
-- Ensures a block device is clean. That is:
-- - unmounted
-- - any lvm volume groups are deactivated
-- - any lvm physical device signatures removed
-- - partition table wiped
--
-- :param block_device: str: Full path to block device to clean.
-- '''
-- for mp, d in mounts():
-- if d == block_device:
-- juju_log('clean_storage(): %s is mounted @ %s, unmounting.' %
-- (d, mp), level=INFO)
-- umount(mp, persist=True)
--
-- if is_lvm_physical_volume(block_device):
-- deactivate_lvm_volume_group(block_device)
-- remove_lvm_physical_volume(block_device)
-- else:
-- zap_disk(block_device)
--
--
--def is_ip(address):
-- """
-- Returns True if address is a valid IP address.
-- """
-- try:
-- # Test to see if already an IPv4 address
-- socket.inet_aton(address)
-- return True
-- except socket.error:
-- return False
--
--
--def ns_query(address):
-- try:
-- import dns.resolver
-- except ImportError:
-- apt_install('python-dnspython')
-- import dns.resolver
--
-- if isinstance(address, dns.name.Name):
-- rtype = 'PTR'
-- elif isinstance(address, basestring):
-- rtype = 'A'
-- else:
-- return None
--
-- answers = dns.resolver.query(address, rtype)
-- if answers:
-- return str(answers[0])
-- return None
--
--
--def get_host_ip(hostname):
-- """
-- Resolves the IP for a given hostname, or returns
-- the input if it is already an IP.
-- """
-- if is_ip(hostname):
-- return hostname
--
-- return ns_query(hostname)
--
--
--def get_hostname(address, fqdn=True):
-- """
-- Resolves hostname for given IP, or returns the input
-- if it is already a hostname.
-- """
-- if is_ip(address):
-- try:
-- import dns.reversename
-- except ImportError:
-- apt_install('python-dnspython')
-- import dns.reversename
--
-- rev = dns.reversename.from_address(address)
-- result = ns_query(rev)
-- if not result:
-- return None
-- else:
-- result = address
--
-- if fqdn:
-- # strip trailing .
-- if result.endswith('.'):
-- return result[:-1]
-- else:
-- return result
-- else:
-- return result.split('.')[0]
 === removed directory 'hooks/charmhelpers/contrib/peerstorage'
 === removed file 'hooks/charmhelpers/contrib/peerstorage/__init__.py'
 --- hooks/charmhelpers/contrib/peerstorage/__init__.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/peerstorage/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,83 +0,0 @@
--from charmhelpers.core.hookenv import (
-- relation_ids,
-- relation_get,
-- local_unit,
-- relation_set,
--)
--
--"""
--This helper provides functions to support use of a peer relation
--for basic key/value storage, with the added benefit that all storage
--can be replicated across peer units, so this is really useful for
--services that issue usernames/passwords to remote services.
--
--def shared_db_changed()
-- # Only the lead unit should create passwords
-- if not is_leader():
-- return
-- username = relation_get('username')
-- key = '{}.password'.format(username)
-- # Attempt to retrieve any existing password for this user
-- password = peer_retrieve(key)
-- if password is None:
-- # New user, create password and store
-- password = pwgen(length=64)
-- peer_store(key, password)
-- create_access(username, password)
-- relation_set(password=password)
--
--
--def cluster_changed()
-- # Echo any relation data other that *-address
-- # back onto the peer relation so all units have
-- # all *.password keys stored on their local relation
-- # for later retrieval.
-- peer_echo()
--
--"""
--
--
--def peer_retrieve(key, relation_name='cluster'):
-- """ Retrieve a named key from peer relation relation_name """
-- cluster_rels = relation_ids(relation_name)
-- if len(cluster_rels) > 0:
-- cluster_rid = cluster_rels[0]
-- return relation_get(attribute=key, rid=cluster_rid,
-- unit=local_unit())
-- else:
-- raise ValueError('Unable to detect'
-- 'peer relation {}'.format(relation_name))
--
--
--def peer_store(key, value, relation_name='cluster'):
-- """ Store the key/value pair on the named peer relation relation_name """
-- cluster_rels = relation_ids(relation_name)
-- if len(cluster_rels) > 0:
-- cluster_rid = cluster_rels[0]
-- relation_set(relation_id=cluster_rid,
-- relation_settings={key: value})
-- else:
-- raise ValueError('Unable to detect '
-- 'peer relation {}'.format(relation_name))
--
--
--def peer_echo(includes=None):
-- """Echo filtered attributes back onto the same relation for storage
--
-- Note that this helper must only be called within a peer relation
-- changed hook
-- """
-- rdata = relation_get()
-- echo_data = {}
-- if includes is None:
-- echo_data = rdata.copy()
-- for ex in ['private-address', 'public-address']:
-- if ex in echo_data:
-- echo_data.pop(ex)
-- else:
-- for attribute, value in rdata.iteritems():
-- for include in includes:
-- if include in attribute:
-- echo_data[attribute] = value
-- if len(echo_data) > 0:
-- relation_set(relation_settings=echo_data)
 === removed directory 'hooks/charmhelpers/contrib/python'
 === removed file 'hooks/charmhelpers/contrib/python/__init__.py'
 === removed file 'hooks/charmhelpers/contrib/python/packages.py'
 --- hooks/charmhelpers/contrib/python/packages.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/python/packages.py	1970-01-01 00:00:00 +0000
@@ -1,76 +0,0 @@
--#!/usr/bin/env python
--# coding: utf-8
--
--__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
--
--from charmhelpers.fetch import apt_install
--from charmhelpers.core.hookenv import log
--
--try:
-- from pip import main as pip_execute
--except ImportError:
-- apt_install('python-pip')
-- from pip import main as pip_execute
--
--
--def parse_options(given, available):
-- """Given a set of options, check if available"""
-- for key, value in given.items():
-- if key in available:
-- yield "--{0}={1}".format(key, value)
--
--
--def pip_install_requirements(requirements, **options):
-- """Install a requirements file """
-- command = ["install"]
--
-- available_options = ('proxy', 'src', 'log', )
-- for option in parse_options(options, available_options):
-- command.append(option)
--
-- command.append("-r {0}".format(requirements))
-- log("Installing from file: {} with options: {}".format(requirements,
-- command))
-- pip_execute(command)
--
--
--def pip_install(package, fatal=False, **options):
-- """Install a python package"""
-- command = ["install"]
--
-- available_options = ('proxy', 'src', 'log', "index-url", )
-- for option in parse_options(options, available_options):
-- command.append(option)
--
-- if isinstance(package, list):
-- command.extend(package)
-- else:
-- command.append(package)
--
-- log("Installing {} package with options: {}".format(package,
-- command))
-- pip_execute(command)
--
--
--def pip_uninstall(package, **options):
-- """Uninstall a python package"""
-- command = ["uninstall", "-q", "-y"]
--
-- available_options = ('proxy', 'log', )
-- for option in parse_options(options, available_options):
-- command.append(option)
--
-- if isinstance(package, list):
-- command.extend(package)
-- else:
-- command.append(package)
--
-- log("Uninstalling {} package with options: {}".format(package,
-- command))
-- pip_execute(command)
--
--
--def pip_list():
-- """Returns the list of current python installed packages
-- """
-- return pip_execute(["list"])
 === removed file 'hooks/charmhelpers/contrib/python/version.py'
 --- hooks/charmhelpers/contrib/python/version.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/python/version.py	1970-01-01 00:00:00 +0000
@@ -1,18 +0,0 @@
--#!/usr/bin/env python
--# coding: utf-8
--
--__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
--
--import sys
--
--
--def current_version():
-- """Current system python version"""
-- return sys.version_info
--
--
--def current_version_string():
-- """Current system python version as string major.minor.micro"""
-- return "{0}.{1}.{2}".format(sys.version_info.major,
-- sys.version_info.minor,
-- sys.version_info.micro)
 === removed directory 'hooks/charmhelpers/contrib/saltstack'
 === removed file 'hooks/charmhelpers/contrib/saltstack/__init__.py'
 --- hooks/charmhelpers/contrib/saltstack/__init__.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/saltstack/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,102 +0,0 @@
--"""Charm Helpers saltstack - declare the state of your machines.
--
--This helper enables you to declare your machine state, rather than
--program it procedurally (and have to test each change to your procedures).
--Your install hook can be as simple as:
--
--{{{
--from charmhelpers.contrib.saltstack import (
-- install_salt_support,
-- update_machine_state,
--)
--
--
--def install():
-- install_salt_support()
-- update_machine_state('machine_states/dependencies.yaml')
-- update_machine_state('machine_states/installed.yaml')
--}}}
--
--and won't need to change (nor will its tests) when you change the machine
--state.
--
--It's using a python package called salt-minion which allows various formats for
--specifying resources, such as:
--
--{{{
--/srv/{{ basedir }}:
-- file.directory:
-- - group: ubunet
-- - user: ubunet
-- - require:
-- - user: ubunet
-- - recurse:
-- - user
-- - group
--
--ubunet:
-- group.present:
-- - gid: 1500
-- user.present:
-- - uid: 1500
-- - gid: 1500
-- - createhome: False
-- - require:
-- - group: ubunet
--}}}
--
--The docs for all the different state definitions are at:
-- http://docs.saltstack.com/ref/states/all/
--
--
--TODO:
-- * Add test helpers which will ensure that machine state definitions
-- are functionally (but not necessarily logically) correct (ie. getting
-- salt to parse all state defs.
-- * Add a link to a public bootstrap charm example / blogpost.
-- * Find a way to obviate the need to use the grains['charm_dir'] syntax
-- in templates.
--"""
--# Copyright 2013 Canonical Ltd.
--#
--# Authors:
--# Charm Helpers Developers <juju@lists.ubuntu.com>
--import subprocess
--
--import charmhelpers.contrib.templating.contexts
--import charmhelpers.core.host
--import charmhelpers.core.hookenv
--
--
--salt_grains_path = '/etc/salt/grains'
--
--
--def install_salt_support(from_ppa=True):
-- """Installs the salt-minion helper for machine state.
--
-- By default the salt-minion package is installed from
-- the saltstack PPA. If from_ppa is False you must ensure
-- that the salt-minion package is available in the apt cache.
-- """
-- if from_ppa:
-- subprocess.check_call([
-- '/usr/bin/add-apt-repository',
-- '--yes',
-- 'ppa:saltstack/salt',
-- ])
-- subprocess.check_call(['/usr/bin/apt-get', 'update'])
-- # We install salt-common as salt-minion would run the salt-minion
-- # daemon.
-- charmhelpers.fetch.apt_install('salt-common')
--
--
--def update_machine_state(state_path):
-- """Update the machine state using the provided state declaration."""
-- charmhelpers.contrib.templating.contexts.juju_state_to_yaml(
-- salt_grains_path)
-- subprocess.check_call([
-- 'salt-call',
-- '--local',
-- 'state.template',
-- state_path,
-- ])
 === removed directory 'hooks/charmhelpers/contrib/ssl'
 === removed file 'hooks/charmhelpers/contrib/ssl/__init__.py'
 --- hooks/charmhelpers/contrib/ssl/__init__.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/ssl/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,78 +0,0 @@
--import subprocess
--from charmhelpers.core import hookenv
--
--
--def generate_selfsigned(keyfile, certfile, keysize="1024", config=None, subject=None, cn=None):
-- """Generate selfsigned SSL keypair
--
-- You must provide one of the 3 optional arguments:
-- config, subject or cn
-- If more than one is provided the leftmost will be used
--
-- Arguments:
-- keyfile -- (required) full path to the keyfile to be created
-- certfile -- (required) full path to the certfile to be created
-- keysize -- (optional) SSL key length
-- config -- (optional) openssl configuration file
-- subject -- (optional) dictionary with SSL subject variables
-- cn -- (optional) cerfificate common name
--
-- Required keys in subject dict:
-- cn -- Common name (eq. FQDN)
--
-- Optional keys in subject dict
-- country -- Country Name (2 letter code)
-- state -- State or Province Name (full name)
-- locality -- Locality Name (eg, city)
-- organization -- Organization Name (eg, company)
-- organizational_unit -- Organizational Unit Name (eg, section)
-- email -- Email Address
-- """
--
-- cmd = []
-- if config:
-- cmd = ["/usr/bin/openssl", "req", "-new", "-newkey",
-- "rsa:{}".format(keysize), "-days", "365", "-nodes", "-x509",
-- "-keyout", keyfile,
-- "-out", certfile, "-config", config]
-- elif subject:
-- ssl_subject = ""
-- if "country" in subject:
-- ssl_subject = ssl_subject + "/C={}".format(subject["country"])
-- if "state" in subject:
-- ssl_subject = ssl_subject + "/ST={}".format(subject["state"])
-- if "locality" in subject:
-- ssl_subject = ssl_subject + "/L={}".format(subject["locality"])
-- if "organization" in subject:
-- ssl_subject = ssl_subject + "/O={}".format(subject["organization"])
-- if "organizational_unit" in subject:
-- ssl_subject = ssl_subject + "/OU={}".format(subject["organizational_unit"])
-- if "cn" in subject:
-- ssl_subject = ssl_subject + "/CN={}".format(subject["cn"])
-- else:
-- hookenv.log("When using \"subject\" argument you must "
-- "provide \"cn\" field at very least")
-- return False
-- if "email" in subject:
-- ssl_subject = ssl_subject + "/emailAddress={}".format(subject["email"])
--
-- cmd = ["/usr/bin/openssl", "req", "-new", "-newkey",
-- "rsa:{}".format(keysize), "-days", "365", "-nodes", "-x509",
-- "-keyout", keyfile,
-- "-out", certfile, "-subj", ssl_subject]
-- elif cn:
-- cmd = ["/usr/bin/openssl", "req", "-new", "-newkey",
-- "rsa:{}".format(keysize), "-days", "365", "-nodes", "-x509",
-- "-keyout", keyfile,
-- "-out", certfile, "-subj", "/CN={}".format(cn)]
--
-- if not cmd:
-- hookenv.log("No config, subject or cn provided,"
-- "unable to generate self signed SSL certificates")
-- return False
-- try:
-- subprocess.check_call(cmd)
-- return True
-- except Exception as e:
-- print "Execution of openssl command failed:\n{}".format(e)
-- return False
 === removed file 'hooks/charmhelpers/contrib/ssl/service.py'
 --- hooks/charmhelpers/contrib/ssl/service.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/ssl/service.py	1970-01-01 00:00:00 +0000
@@ -1,267 +0,0 @@
--import logging
--import os
--from os.path import join as path_join
--from os.path import exists
--import subprocess
--
--
--log = logging.getLogger("service_ca")
--
--logging.basicConfig(level=logging.DEBUG)
--
--STD_CERT = "standard"
--
--# Mysql server is fairly picky about cert creation
--# and types, spec its creation separately for now.
--MYSQL_CERT = "mysql"
--
--
--class ServiceCA(object):
--
-- default_expiry = str(365 * 2)
-- default_ca_expiry = str(365 * 6)
--
-- def __init__(self, name, ca_dir, cert_type=STD_CERT):
-- self.name = name
-- self.ca_dir = ca_dir
-- self.cert_type = cert_type
--
-- ###############
-- # Hook Helper API
-- @staticmethod
-- def get_ca(type=STD_CERT):
-- service_name = os.environ['JUJU_UNIT_NAME'].split('/')[0]
-- ca_path = os.path.join(os.environ['CHARM_DIR'], 'ca')
-- ca = ServiceCA(service_name, ca_path, type)
-- ca.init()
-- return ca
--
-- @classmethod
-- def get_service_cert(cls, type=STD_CERT):
-- service_name = os.environ['JUJU_UNIT_NAME'].split('/')[0]
-- ca = cls.get_ca()
-- crt, key = ca.get_or_create_cert(service_name)
-- return crt, key, ca.get_ca_bundle()
--
-- ###############
--
-- def init(self):
-- log.debug("initializing service ca")
-- if not exists(self.ca_dir):
-- self._init_ca_dir(self.ca_dir)
-- self._init_ca()
--
-- @property
-- def ca_key(self):
-- return path_join(self.ca_dir, 'private', 'cacert.key')
--
-- @property
-- def ca_cert(self):
-- return path_join(self.ca_dir, 'cacert.pem')
--
-- @property
-- def ca_conf(self):
-- return path_join(self.ca_dir, 'ca.cnf')
--
-- @property
-- def signing_conf(self):
-- return path_join(self.ca_dir, 'signing.cnf')
--
-- def _init_ca_dir(self, ca_dir):
-- os.mkdir(ca_dir)
-- for i in ['certs', 'crl', 'newcerts', 'private']:
-- sd = path_join(ca_dir, i)
-- if not exists(sd):
-- os.mkdir(sd)
--
-- if not exists(path_join(ca_dir, 'serial')):
-- with open(path_join(ca_dir, 'serial'), 'wb') as fh:
-- fh.write('02\n')
--
-- if not exists(path_join(ca_dir, 'index.txt')):
-- with open(path_join(ca_dir, 'index.txt'), 'wb') as fh:
-- fh.write('')
--
-- def _init_ca(self):
-- """Generate the root ca's cert and key.
-- """
-- if not exists(path_join(self.ca_dir, 'ca.cnf')):
-- with open(path_join(self.ca_dir, 'ca.cnf'), 'wb') as fh:
-- fh.write(
-- CA_CONF_TEMPLATE % (self.get_conf_variables()))
--
-- if not exists(path_join(self.ca_dir, 'signing.cnf')):
-- with open(path_join(self.ca_dir, 'signing.cnf'), 'wb') as fh:
-- fh.write(
-- SIGNING_CONF_TEMPLATE % (self.get_conf_variables()))
--
-- if exists(self.ca_cert) or exists(self.ca_key):
-- raise RuntimeError("Initialized called when CA already exists")
-- cmd = ['openssl', 'req', '-config', self.ca_conf,
-- '-x509', '-nodes', '-newkey', 'rsa',
-- '-days', self.default_ca_expiry,
-- '-keyout', self.ca_key, '-out', self.ca_cert,
-- '-outform', 'PEM']
-- output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
-- log.debug("CA Init:\n %s", output)
--
-- def get_conf_variables(self):
-- return dict(
-- org_name="juju",
-- org_unit_name="%s service" % self.name,
-- common_name=self.name,
-- ca_dir=self.ca_dir)
--
-- def get_or_create_cert(self, common_name):
-- if common_name in self:
-- return self.get_certificate(common_name)
-- return self.create_certificate(common_name)
--
-- def create_certificate(self, common_name):
-- if common_name in self:
-- return self.get_certificate(common_name)
-- key_p = path_join(self.ca_dir, "certs", "%s.key" % common_name)
-- crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
-- csr_p = path_join(self.ca_dir, "certs", "%s.csr" % common_name)
-- self._create_certificate(common_name, key_p, csr_p, crt_p)
-- return self.get_certificate(common_name)
--
-- def get_certificate(self, common_name):
-- if not common_name in self:
-- raise ValueError("No certificate for %s" % common_name)
-- key_p = path_join(self.ca_dir, "certs", "%s.key" % common_name)
-- crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
-- with open(crt_p) as fh:
-- crt = fh.read()
-- with open(key_p) as fh:
-- key = fh.read()
-- return crt, key
--
-- def __contains__(self, common_name):
-- crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
-- return exists(crt_p)
--
-- def _create_certificate(self, common_name, key_p, csr_p, crt_p):
-- template_vars = self.get_conf_variables()
-- template_vars['common_name'] = common_name
-- subj = '/O=%(org_name)s/OU=%(org_unit_name)s/CN=%(common_name)s' % (
-- template_vars)
--
-- log.debug("CA Create Cert %s", common_name)
-- cmd = ['openssl', 'req', '-sha1', '-newkey', 'rsa:2048',
-- '-nodes', '-days', self.default_expiry,
-- '-keyout', key_p, '-out', csr_p, '-subj', subj]
-- subprocess.check_call(cmd)
-- cmd = ['openssl', 'rsa', '-in', key_p, '-out', key_p]
-- subprocess.check_call(cmd)
--
-- log.debug("CA Sign Cert %s", common_name)
-- if self.cert_type == MYSQL_CERT:
-- cmd = ['openssl', 'x509', '-req',
-- '-in', csr_p, '-days', self.default_expiry,
-- '-CA', self.ca_cert, '-CAkey', self.ca_key,
-- '-set_serial', '01', '-out', crt_p]
-- else:
-- cmd = ['openssl', 'ca', '-config', self.signing_conf,
-- '-extensions', 'req_extensions',
-- '-days', self.default_expiry, '-notext',
-- '-in', csr_p, '-out', crt_p, '-subj', subj, '-batch']
-- log.debug("running %s", " ".join(cmd))
-- subprocess.check_call(cmd)
--
-- def get_ca_bundle(self):
-- with open(self.ca_cert) as fh:
-- return fh.read()
--
--
--CA_CONF_TEMPLATE = """
--[ ca ]
--default_ca = CA_default
--
--[ CA_default ]
--dir = %(ca_dir)s
--policy = policy_match
--database = $dir/index.txt
--serial = $dir/serial
--certs = $dir/certs
--crl_dir = $dir/crl
--new_certs_dir = $dir/newcerts
--certificate = $dir/cacert.pem
--private_key = $dir/private/cacert.key
--RANDFILE = $dir/private/.rand
--default_md = default
--
--[ req ]
--default_bits = 1024
--default_md = sha1
--
--prompt = no
--distinguished_name = ca_distinguished_name
--
--x509_extensions = ca_extensions
--
--[ ca_distinguished_name ]
--organizationName = %(org_name)s
--organizationalUnitName = %(org_unit_name)s Certificate Authority
--
--
--[ policy_match ]
--countryName = optional
--stateOrProvinceName = optional
--organizationName = match
--organizationalUnitName = optional
--commonName = supplied
--
--[ ca_extensions ]
--basicConstraints = critical,CA:true
--subjectKeyIdentifier = hash
--authorityKeyIdentifier = keyid:always, issuer
--keyUsage = cRLSign, keyCertSign
--"""
--
--
--SIGNING_CONF_TEMPLATE = """
--[ ca ]
--default_ca = CA_default
--
--[ CA_default ]
--dir = %(ca_dir)s
--policy = policy_match
--database = $dir/index.txt
--serial = $dir/serial
--certs = $dir/certs
--crl_dir = $dir/crl
--new_certs_dir = $dir/newcerts
--certificate = $dir/cacert.pem
--private_key = $dir/private/cacert.key
--RANDFILE = $dir/private/.rand
--default_md = default
--
--[ req ]
--default_bits = 1024
--default_md = sha1
--
--prompt = no
--distinguished_name = req_distinguished_name
--
--x509_extensions = req_extensions
--
--[ req_distinguished_name ]
--organizationName = %(org_name)s
--organizationalUnitName = %(org_unit_name)s machine resources
--commonName = %(common_name)s
--
--[ policy_match ]
--countryName = optional
--stateOrProvinceName = optional
--organizationName = match
--organizationalUnitName = optional
--commonName = supplied
--
--[ req_extensions ]
--basicConstraints = CA:false
--subjectKeyIdentifier = hash
--authorityKeyIdentifier = keyid:always, issuer
--keyUsage = digitalSignature, keyEncipherment, keyAgreement
--extendedKeyUsage = serverAuth, clientAuth
--"""
 === removed directory 'hooks/charmhelpers/contrib/storage'
 === removed file 'hooks/charmhelpers/contrib/storage/__init__.py'
 === removed directory 'hooks/charmhelpers/contrib/storage/linux'
 === removed file 'hooks/charmhelpers/contrib/storage/linux/__init__.py'
 === removed file 'hooks/charmhelpers/contrib/storage/linux/ceph.py'
 --- hooks/charmhelpers/contrib/storage/linux/ceph.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/storage/linux/ceph.py	1970-01-01 00:00:00 +0000
@@ -1,387 +0,0 @@
--#
--# Copyright 2012 Canonical Ltd.
--#
--# This file is sourced from lp:openstack-charm-helpers
--#
--# Authors:
--# James Page <james.page@ubuntu.com>
--# Adam Gandelman <adamg@ubuntu.com>
--#
--
--import os
--import shutil
--import json
--import time
--
--from subprocess import (
-- check_call,
-- check_output,
-- CalledProcessError
--)
--
--from charmhelpers.core.hookenv import (
-- relation_get,
-- relation_ids,
-- related_units,
-- log,
-- INFO,
-- WARNING,
-- ERROR
--)
--
--from charmhelpers.core.host import (
-- mount,
-- mounts,
-- service_start,
-- service_stop,
-- service_running,
-- umount,
--)
--
--from charmhelpers.fetch import (
-- apt_install,
--)
--
--KEYRING = '/etc/ceph/ceph.client.{}.keyring'
--KEYFILE = '/etc/ceph/ceph.client.{}.key'
--
--CEPH_CONF = """[global]
-- auth supported = {auth}
-- keyring = {keyring}
-- mon host = {mon_hosts}
-- log to syslog = {use_syslog}
-- err to syslog = {use_syslog}
-- clog to syslog = {use_syslog}
--"""
--
--
--def install():
-- ''' Basic Ceph client installation '''
-- ceph_dir = "/etc/ceph"
-- if not os.path.exists(ceph_dir):
-- os.mkdir(ceph_dir)
-- apt_install('ceph-common', fatal=True)
--
--
--def rbd_exists(service, pool, rbd_img):
-- ''' Check to see if a RADOS block device exists '''
-- try:
-- out = check_output(['rbd', 'list', '--id', service,
-- '--pool', pool])
-- except CalledProcessError:
-- return False
-- else:
-- return rbd_img in out
--
--
--def create_rbd_image(service, pool, image, sizemb):
-- ''' Create a new RADOS block device '''
-- cmd = [
-- 'rbd',
-- 'create',
-- image,
-- '--size',
-- str(sizemb),
-- '--id',
-- service,
-- '--pool',
-- pool
-- ]
-- check_call(cmd)
--
--
--def pool_exists(service, name):
-- ''' Check to see if a RADOS pool already exists '''
-- try:
-- out = check_output(['rados', '--id', service, 'lspools'])
-- except CalledProcessError:
-- return False
-- else:
-- return name in out
--
--
--def get_osds(service):
-- '''
-- Return a list of all Ceph Object Storage Daemons
-- currently in the cluster
-- '''
-- version = ceph_version()
-- if version and version >= '0.56':
-- return json.loads(check_output(['ceph', '--id', service,
-- 'osd', 'ls', '--format=json']))
-- else:
-- return None
--
--
--def create_pool(service, name, replicas=2):
-- ''' Create a new RADOS pool '''
-- if pool_exists(service, name):
-- log("Ceph pool {} already exists, skipping creation".format(name),
-- level=WARNING)
-- return
-- # Calculate the number of placement groups based
-- # on upstream recommended best practices.
-- osds = get_osds(service)
-- if osds:
-- pgnum = (len(osds) * 100 / replicas)
-- else:
-- # NOTE(james-page): Default to 200 for older ceph versions
-- # which don't support OSD query from cli
-- pgnum = 200
-- cmd = [
-- 'ceph', '--id', service,
-- 'osd', 'pool', 'create',
-- name, str(pgnum)
-- ]
-- check_call(cmd)
-- cmd = [
-- 'ceph', '--id', service,
-- 'osd', 'pool', 'set', name,
-- 'size', str(replicas)
-- ]
-- check_call(cmd)
--
--
--def delete_pool(service, name):
-- ''' Delete a RADOS pool from ceph '''
-- cmd = [
-- 'ceph', '--id', service,
-- 'osd', 'pool', 'delete',
-- name, '--yes-i-really-really-mean-it'
-- ]
-- check_call(cmd)
--
--
--def _keyfile_path(service):
-- return KEYFILE.format(service)
--
--
--def _keyring_path(service):
-- return KEYRING.format(service)
--
--
--def create_keyring(service, key):
-- ''' Create a new Ceph keyring containing key'''
-- keyring = _keyring_path(service)
-- if os.path.exists(keyring):
-- log('ceph: Keyring exists at %s.' % keyring, level=WARNING)
-- return
-- cmd = [
-- 'ceph-authtool',
-- keyring,
-- '--create-keyring',
-- '--name=client.{}'.format(service),
-- '--add-key={}'.format(key)
-- ]
-- check_call(cmd)
-- log('ceph: Created new ring at %s.' % keyring, level=INFO)
--
--
--def create_key_file(service, key):
-- ''' Create a file containing key '''
-- keyfile = _keyfile_path(service)
-- if os.path.exists(keyfile):
-- log('ceph: Keyfile exists at %s.' % keyfile, level=WARNING)
-- return
-- with open(keyfile, 'w') as fd:
-- fd.write(key)
-- log('ceph: Created new keyfile at %s.' % keyfile, level=INFO)
--
--
--def get_ceph_nodes():
-- ''' Query named relation 'ceph' to detemine current nodes '''
-- hosts = []
-- for r_id in relation_ids('ceph'):
-- for unit in related_units(r_id):
-- hosts.append(relation_get('private-address', unit=unit, rid=r_id))
-- return hosts
--
--
--def configure(service, key, auth, use_syslog):
-- ''' Perform basic configuration of Ceph '''
-- create_keyring(service, key)
-- create_key_file(service, key)
-- hosts = get_ceph_nodes()
-- with open('/etc/ceph/ceph.conf', 'w') as ceph_conf:
-- ceph_conf.write(CEPH_CONF.format(auth=auth,
-- keyring=_keyring_path(service),
-- mon_hosts=",".join(map(str, hosts)),
-- use_syslog=use_syslog))
-- modprobe('rbd')
--
--
--def image_mapped(name):
-- ''' Determine whether a RADOS block device is mapped locally '''
-- try:
-- out = check_output(['rbd', 'showmapped'])
-- except CalledProcessError:
-- return False
-- else:
-- return name in out
--
--
--def map_block_storage(service, pool, image):
-- ''' Map a RADOS block device for local use '''
-- cmd = [
-- 'rbd',
-- 'map',
-- '{}/{}'.format(pool, image),
-- '--user',
-- service,
-- '--secret',
-- _keyfile_path(service),
-- ]
-- check_call(cmd)
--
--
--def filesystem_mounted(fs):
-- ''' Determine whether a filesytems is already mounted '''
-- return fs in [f for f, m in mounts()]
--
--
--def make_filesystem(blk_device, fstype='ext4', timeout=10):
-- ''' Make a new filesystem on the specified block device '''
-- count = 0
-- e_noent = os.errno.ENOENT
-- while not os.path.exists(blk_device):
-- if count >= timeout:
-- log('ceph: gave up waiting on block device %s' % blk_device,
-- level=ERROR)
-- raise IOError(e_noent, os.strerror(e_noent), blk_device)
-- log('ceph: waiting for block device %s to appear' % blk_device,
-- level=INFO)
-- count += 1
-- time.sleep(1)
-- else:
-- log('ceph: Formatting block device %s as filesystem %s.' %
-- (blk_device, fstype), level=INFO)
-- check_call(['mkfs', '-t', fstype, blk_device])
--
--
--def place_data_on_block_device(blk_device, data_src_dst):
-- ''' Migrate data in data_src_dst to blk_device and then remount '''
-- # mount block device into /mnt
-- mount(blk_device, '/mnt')
-- # copy data to /mnt
-- copy_files(data_src_dst, '/mnt')
-- # umount block device
-- umount('/mnt')
-- # Grab user/group ID's from original source
-- _dir = os.stat(data_src_dst)
-- uid = _dir.st_uid
-- gid = _dir.st_gid
-- # re-mount where the data should originally be
-- # TODO: persist is currently a NO-OP in core.host
-- mount(blk_device, data_src_dst, persist=True)
-- # ensure original ownership of new mount.
-- os.chown(data_src_dst, uid, gid)
--
--
--# TODO: re-use
--def modprobe(module):
-- ''' Load a kernel module and configure for auto-load on reboot '''
-- log('ceph: Loading kernel module', level=INFO)
-- cmd = ['modprobe', module]
-- check_call(cmd)
-- with open('/etc/modules', 'r+') as modules:
-- if module not in modules.read():
-- modules.write(module)
--
--
--def copy_files(src, dst, symlinks=False, ignore=None):
-- ''' Copy files from src to dst '''
-- for item in os.listdir(src):
-- s = os.path.join(src, item)
-- d = os.path.join(dst, item)
-- if os.path.isdir(s):
-- shutil.copytree(s, d, symlinks, ignore)
-- else:
-- shutil.copy2(s, d)
--
--
--def ensure_ceph_storage(service, pool, rbd_img, sizemb, mount_point,
-- blk_device, fstype, system_services=[]):
-- """
-- NOTE: This function must only be called from a single service unit for
-- the same rbd_img otherwise data loss will occur.
--
-- Ensures given pool and RBD image exists, is mapped to a block device,
-- and the device is formatted and mounted at the given mount_point.
--
-- If formatting a device for the first time, data existing at mount_point
-- will be migrated to the RBD device before being re-mounted.
--
-- All services listed in system_services will be stopped prior to data
-- migration and restarted when complete.
-- """
-- # Ensure pool, RBD image, RBD mappings are in place.
-- if not pool_exists(service, pool):
-- log('ceph: Creating new pool {}.'.format(pool))
-- create_pool(service, pool)
--
-- if not rbd_exists(service, pool, rbd_img):
-- log('ceph: Creating RBD image ({}).'.format(rbd_img))
-- create_rbd_image(service, pool, rbd_img, sizemb)
--
-- if not image_mapped(rbd_img):
-- log('ceph: Mapping RBD Image {} as a Block Device.'.format(rbd_img))
-- map_block_storage(service, pool, rbd_img)
--
-- # make file system
-- # TODO: What happens if for whatever reason this is run again and
-- # the data is already in the rbd device and/or is mounted??
-- # When it is mounted already, it will fail to make the fs
-- # XXX: This is really sketchy! Need to at least add an fstab entry
-- # otherwise this hook will blow away existing data if its executed
-- # after a reboot.
-- if not filesystem_mounted(mount_point):
-- make_filesystem(blk_device, fstype)
--
-- for svc in system_services:
-- if service_running(svc):
-- log('ceph: Stopping services {} prior to migrating data.'
-- .format(svc))
-- service_stop(svc)
--
-- place_data_on_block_device(blk_device, mount_point)
--
-- for svc in system_services:
-- log('ceph: Starting service {} after migrating data.'
-- .format(svc))
-- service_start(svc)
--
--
--def ensure_ceph_keyring(service, user=None, group=None):
-- '''
-- Ensures a ceph keyring is created for a named service
-- and optionally ensures user and group ownership.
--
-- Returns False if no ceph key is available in relation state.
-- '''
-- key = None
-- for rid in relation_ids('ceph'):
-- for unit in related_units(rid):
-- key = relation_get('key', rid=rid, unit=unit)
-- if key:
-- break
-- if not key:
-- return False
-- create_keyring(service=service, key=key)
-- keyring = _keyring_path(service)
-- if user and group:
-- check_call(['chown', '%s.%s' % (user, group), keyring])
-- return True
--
--
--def ceph_version():
-- ''' Retrieve the local version of ceph '''
-- if os.path.exists('/usr/bin/ceph'):
-- cmd = ['ceph', '-v']
-- output = check_output(cmd)
-- output = output.split()
-- if len(output) > 3:
-- return output[2]
-- else:
-- return None
-- else:
-- return None
 === removed file 'hooks/charmhelpers/contrib/storage/linux/loopback.py'
 --- hooks/charmhelpers/contrib/storage/linux/loopback.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/storage/linux/loopback.py	1970-01-01 00:00:00 +0000
@@ -1,62 +0,0 @@
--
--import os
--import re
--
--from subprocess import (
-- check_call,
-- check_output,
--)
--
--
--##################################################
--# loopback device helpers.
--##################################################
--def loopback_devices():
-- '''
-- Parse through 'losetup -a' output to determine currently mapped
-- loopback devices. Output is expected to look like:
--
-- /dev/loop0: [0807]:961814 (/tmp/my.img)
--
-- :returns: dict: a dict mapping {loopback_dev: backing_file}
-- '''
-- loopbacks = {}
-- cmd = ['losetup', '-a']
-- devs = [d.strip().split(' ') for d in
-- check_output(cmd).splitlines() if d != '']
-- for dev, _, f in devs:
-- loopbacks[dev.replace(':', '')] = re.search('\((\S+)\)', f).groups()[0]
-- return loopbacks
--
--
--def create_loopback(file_path):
-- '''
-- Create a loopback device for a given backing file.
--
-- :returns: str: Full path to new loopback device (eg, /dev/loop0)
-- '''
-- file_path = os.path.abspath(file_path)
-- check_call(['losetup', '--find', file_path])
-- for d, f in loopback_devices().iteritems():
-- if f == file_path:
-- return d
--
--
--def ensure_loopback_device(path, size):
-- '''
-- Ensure a loopback device exists for a given backing file path and size.
-- If it a loopback device is not mapped to file, a new one will be created.
--
-- TODO: Confirm size of found loopback device.
--
-- :returns: str: Full path to the ensured loopback device (eg, /dev/loop0)
-- '''
-- for d, f in loopback_devices().iteritems():
-- if f == path:
-- return d
--
-- if not os.path.exists(path):
-- cmd = ['truncate', '--size', size, path]
-- check_call(cmd)
--
-- return create_loopback(path)
 === removed file 'hooks/charmhelpers/contrib/storage/linux/lvm.py'
 --- hooks/charmhelpers/contrib/storage/linux/lvm.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/storage/linux/lvm.py	1970-01-01 00:00:00 +0000
@@ -1,88 +0,0 @@
--from subprocess import (
-- CalledProcessError,
-- check_call,
-- check_output,
-- Popen,
-- PIPE,
--)
--
--
--##################################################
--# LVM helpers.
--##################################################
--def deactivate_lvm_volume_group(block_device):
-- '''
-- Deactivate any volume gruop associated with an LVM physical volume.
--
-- :param block_device: str: Full path to LVM physical volume
-- '''
-- vg = list_lvm_volume_group(block_device)
-- if vg:
-- cmd = ['vgchange', '-an', vg]
-- check_call(cmd)
--
--
--def is_lvm_physical_volume(block_device):
-- '''
-- Determine whether a block device is initialized as an LVM PV.
--
-- :param block_device: str: Full path of block device to inspect.
--
-- :returns: boolean: True if block device is a PV, False if not.
-- '''
-- try:
-- check_output(['pvdisplay', block_device])
-- return True
-- except CalledProcessError:
-- return False
--
--
--def remove_lvm_physical_volume(block_device):
-- '''
-- Remove LVM PV signatures from a given block device.
--
-- :param block_device: str: Full path of block device to scrub.
-- '''
-- p = Popen(['pvremove', '-ff', block_device],
-- stdin=PIPE)
-- p.communicate(input='y\n')
--
--
--def list_lvm_volume_group(block_device):
-- '''
-- List LVM volume group associated with a given block device.
--
-- Assumes block device is a valid LVM PV.
--
-- :param block_device: str: Full path of block device to inspect.
--
-- :returns: str: Name of volume group associated with block device or None
-- '''
-- vg = None
-- pvd = check_output(['pvdisplay', block_device]).splitlines()
-- for l in pvd:
-- if l.strip().startswith('VG Name'):
-- vg = ' '.join(l.split()).split(' ').pop()
-- return vg
--
--
--def create_lvm_physical_volume(block_device):
-- '''
-- Initialize a block device as an LVM physical volume.
--
-- :param block_device: str: Full path of block device to initialize.
--
-- '''
-- check_call(['pvcreate', block_device])
--
--
--def create_lvm_volume_group(volume_group, block_device):
-- '''
-- Create an LVM volume group backed by a given block device.
--
-- Assumes block device has already been initialized as an LVM PV.
--
-- :param volume_group: str: Name of volume group to create.
-- :block_device: str: Full path of PV-initialized block device.
-- '''
-- check_call(['vgcreate', volume_group, block_device])
 === removed file 'hooks/charmhelpers/contrib/storage/linux/utils.py'
 --- hooks/charmhelpers/contrib/storage/linux/utils.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/storage/linux/utils.py	1970-01-01 00:00:00 +0000
@@ -1,35 +0,0 @@
--from os import stat
--from stat import S_ISBLK
--
--from subprocess import (
-- check_call,
-- check_output,
-- call
--)
--
--
--def is_block_device(path):
-- '''
-- Confirm device at path is a valid block device node.
--
-- :returns: boolean: True if path is a block device, False if not.
-- '''
-- return S_ISBLK(stat(path).st_mode)
--
--
--def zap_disk(block_device):
-- '''
-- Clear a block device of partition table. Relies on sgdisk, which is
-- installed as pat of the 'gdisk' package in Ubuntu.
--
-- :param block_device: str: Full path of block device to clean.
-- '''
-- # sometimes sgdisk exits non-zero; this is OK, dd will clean up
-- call(['sgdisk', '--zap-all', '--mbrtogpt',
-- '--clear', block_device])
-- dev_end = check_output(['blockdev', '--getsz', block_device])
-- gpt_end = int(dev_end.split()[0]) - 100
-- check_call(['dd', 'if=/dev/zero', 'of=%s'%(block_device),
-- 'bs=1M', 'count=1'])
-- check_call(['dd', 'if=/dev/zero', 'of=%s'%(block_device),
-- 'bs=512', 'count=100', 'seek=%s'%(gpt_end)])
 === removed directory 'hooks/charmhelpers/contrib/templating'
 === removed file 'hooks/charmhelpers/contrib/templating/__init__.py'
 === removed file 'hooks/charmhelpers/contrib/templating/contexts.py'
 --- hooks/charmhelpers/contrib/templating/contexts.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/templating/contexts.py	1970-01-01 00:00:00 +0000
@@ -1,104 +0,0 @@
--# Copyright 2013 Canonical Ltd.
--#
--# Authors:
--# Charm Helpers Developers <juju@lists.ubuntu.com>
--"""A helper to create a yaml cache of config with namespaced relation data."""
--import os
--import yaml
--
--import charmhelpers.core.hookenv
--
--
--charm_dir = os.environ.get('CHARM_DIR', '')
--
--
--def dict_keys_without_hyphens(a_dict):
-- """Return the a new dict with underscores instead of hyphens in keys."""
-- return dict(
-- (key.replace('-', '_'), val) for key, val in a_dict.items())
--
--
--def update_relations(context, namespace_separator=':'):
-- """Update the context with the relation data."""
-- # Add any relation data prefixed with the relation type.
-- relation_type = charmhelpers.core.hookenv.relation_type()
-- relations = []
-- context['current_relation'] = {}
-- if relation_type is not None:
-- relation_data = charmhelpers.core.hookenv.relation_get()
-- context['current_relation'] = relation_data
-- # Deprecated: the following use of relation data as keys
-- # directly in the context will be removed.
-- relation_data = dict(
-- ("{relation_type}{namespace_separator}{key}".format(
-- relation_type=relation_type,
-- key=key,
-- namespace_separator=namespace_separator), val)
-- for key, val in relation_data.items())
-- relation_data = dict_keys_without_hyphens(relation_data)
-- context.update(relation_data)
-- relations = charmhelpers.core.hookenv.relations_of_type(relation_type)
-- relations = [dict_keys_without_hyphens(rel) for rel in relations]
--
-- if 'relations_deprecated' not in context:
-- context['relations_deprecated'] = {}
-- if relation_type is not None:
-- relation_type = relation_type.replace('-', '_')
-- context['relations_deprecated'][relation_type] = relations
--
-- context['relations'] = charmhelpers.core.hookenv.relations()
--
--
--def juju_state_to_yaml(yaml_path, namespace_separator=':',
-- allow_hyphens_in_keys=True):
-- """Update the juju config and state in a yaml file.
--
-- This includes any current relation-get data, and the charm
-- directory.
--
-- This function was created for the ansible and saltstack
-- support, as those libraries can use a yaml file to supply
-- context to templates, but it may be useful generally to
-- create and update an on-disk cache of all the config, including
-- previous relation data.
--
-- By default, hyphens are allowed in keys as this is supported
-- by yaml, but for tools like ansible, hyphens are not valid [1].
--
-- [1] http://www.ansibleworks.com/docs/playbooks_variables.html#what-makes-a-valid-variable-name
-- """
-- config = charmhelpers.core.hookenv.config()
--
-- # Add the charm_dir which we will need to refer to charm
-- # file resources etc.
-- config['charm_dir'] = charm_dir
-- config['local_unit'] = charmhelpers.core.hookenv.local_unit()
-- config['unit_private_address'] = charmhelpers.core.hookenv.unit_private_ip()
-- config['unit_public_address'] = charmhelpers.core.hookenv.unit_get(
-- 'public-address'
-- )
--
-- # Don't use non-standard tags for unicode which will not
-- # work when salt uses yaml.load_safe.
-- yaml.add_representer(unicode, lambda dumper,
-- value: dumper.represent_scalar(
-- u'tag:yaml.org,2002:str', value))
--
-- yaml_dir = os.path.dirname(yaml_path)
-- if not os.path.exists(yaml_dir):
-- os.makedirs(yaml_dir)
--
-- if os.path.exists(yaml_path):
-- with open(yaml_path, "r") as existing_vars_file:
-- existing_vars = yaml.load(existing_vars_file.read())
-- else:
-- existing_vars = {}
--
-- if not allow_hyphens_in_keys:
-- config = dict_keys_without_hyphens(config)
-- existing_vars.update(config)
--
-- update_relations(existing_vars, namespace_separator)
--
-- with open(yaml_path, "w+") as fp:
-- fp.write(yaml.dump(existing_vars, default_flow_style=False))
 === removed file 'hooks/charmhelpers/contrib/templating/pyformat.py'
 --- hooks/charmhelpers/contrib/templating/pyformat.py	2013-11-26 17:12:54 +0000
 +++ hooks/charmhelpers/contrib/templating/pyformat.py	1970-01-01 00:00:00 +0000
@@ -1,13 +0,0 @@
--'''
--Templating using standard Python str.format() method.
--'''
--
--from charmhelpers.core import hookenv
--
--
--def render(template, extra={}, **kwargs):
-- """Return the template rendered using Python's str.format()."""
-- context = hookenv.execution_environment()
-- context.update(extra)
-- context.update(kwargs)
-- return template.format(**context)
 === removed directory 'hooks/charmhelpers/contrib/unison'
 === removed file 'hooks/charmhelpers/contrib/unison/__init__.py'
 --- hooks/charmhelpers/contrib/unison/__init__.py	2014-05-09 20:11:59 +0000
 +++ hooks/charmhelpers/contrib/unison/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,257 +0,0 @@
--# Easy file synchronization among peer units using ssh + unison.
--#
--# From *both* peer relation -joined and -changed, add a call to
--# ssh_authorized_peers() describing the peer relation and the desired
--# user + group. After all peer relations have settled, all hosts should
--# be able to connect to on another via key auth'd ssh as the specified user.
--#
--# Other hooks are then free to synchronize files and directories using
--# sync_to_peers().
--#
--# For a peer relation named 'cluster', for example:
--#
--# cluster-relation-joined:
--# ...
--# ssh_authorized_peers(peer_interface='cluster',
--# user='juju_ssh', group='juju_ssh',
--# ensure_user=True)
--# ...
--#
--# cluster-relation-changed:
--# ...
--# ssh_authorized_peers(peer_interface='cluster',
--# user='juju_ssh', group='juju_ssh',
--# ensure_user=True)
--# ...
--#
--# Hooks are now free to sync files as easily as:
--#
--# files = ['/etc/fstab', '/etc/apt.conf.d/']
--# sync_to_peers(peer_interface='cluster',
--# user='juju_ssh, paths=[files])
--#
--# It is assumed the charm itself has setup permissions on each unit
--# such that 'juju_ssh' has read + write permissions. Also assumed
--# that the calling charm takes care of leader delegation.
--#
--# Additionally files can be synchronized only to an specific unit:
--# sync_to_peer(slave_address, user='juju_ssh',
--# paths=[files], verbose=False)
--
--import os
--import pwd
--
--from copy import copy
--from subprocess import check_call, check_output
--
--from charmhelpers.core.host import (
-- adduser,
-- add_user_to_group,
--)
--
--from charmhelpers.core.hookenv import (
-- log,
-- hook_name,
-- relation_ids,
-- related_units,
-- relation_set,
-- relation_get,
-- unit_private_ip,
-- ERROR,
--)
--
--BASE_CMD = ['unison', '-auto', '-batch=true', '-confirmbigdel=false',
-- '-fastcheck=true', '-group=false', '-owner=false',
-- '-prefer=newer', '-times=true']
--
--
--def get_homedir(user):
-- try:
-- user = pwd.getpwnam(user)
-- return user.pw_dir
-- except KeyError:
-- log('Could not get homedir for user %s: user exists?', ERROR)
-- raise Exception
--
--
--def create_private_key(user, priv_key_path):
-- if not os.path.isfile(priv_key_path):
-- log('Generating new SSH key for user %s.' % user)
-- cmd = ['ssh-keygen', '-q', '-N', '', '-t', 'rsa', '-b', '2048',
-- '-f', priv_key_path]
-- check_call(cmd)
-- else:
-- log('SSH key already exists at %s.' % priv_key_path)
-- check_call(['chown', user, priv_key_path])
-- check_call(['chmod', '0600', priv_key_path])
--
--
--def create_public_key(user, priv_key_path, pub_key_path):
-- if not os.path.isfile(pub_key_path):
-- log('Generating missing ssh public key @ %s.' % pub_key_path)
-- cmd = ['ssh-keygen', '-y', '-f', priv_key_path]
-- p = check_output(cmd).strip()
-- with open(pub_key_path, 'wb') as out:
-- out.write(p)
-- check_call(['chown', user, pub_key_path])
--
--
--def get_keypair(user):
-- home_dir = get_homedir(user)
-- ssh_dir = os.path.join(home_dir, '.ssh')
-- priv_key = os.path.join(ssh_dir, 'id_rsa')
-- pub_key = '%s.pub' % priv_key
--
-- if not os.path.isdir(ssh_dir):
-- os.mkdir(ssh_dir)
-- check_call(['chown', '-R', user, ssh_dir])
--
-- create_private_key(user, priv_key)
-- create_public_key(user, priv_key, pub_key)
--
-- with open(priv_key, 'r') as p:
-- _priv = p.read().strip()
--
-- with open(pub_key, 'r') as p:
-- _pub = p.read().strip()
--
-- return (_priv, _pub)
--
--
--def write_authorized_keys(user, keys):
-- home_dir = get_homedir(user)
-- ssh_dir = os.path.join(home_dir, '.ssh')
-- auth_keys = os.path.join(ssh_dir, 'authorized_keys')
-- log('Syncing authorized_keys @ %s.' % auth_keys)
-- with open(auth_keys, 'wb') as out:
-- for k in keys:
-- out.write('%s\n' % k)
--
--
--def write_known_hosts(user, hosts):
-- home_dir = get_homedir(user)
-- ssh_dir = os.path.join(home_dir, '.ssh')
-- known_hosts = os.path.join(ssh_dir, 'known_hosts')
-- khosts = []
-- for host in hosts:
-- cmd = ['ssh-keyscan', '-H', '-t', 'rsa', host]
-- remote_key = check_output(cmd).strip()
-- khosts.append(remote_key)
-- log('Syncing known_hosts @ %s.' % known_hosts)
-- with open(known_hosts, 'wb') as out:
-- for host in khosts:
-- out.write('%s\n' % host)
--
--
--def ensure_user(user, group=None):
-- adduser(user)
-- if group:
-- add_user_to_group(user, group)
--
--
--def ssh_authorized_peers(peer_interface, user, group=None,
-- ensure_local_user=False):
-- """
-- Main setup function, should be called from both peer -changed and -joined
-- hooks with the same parameters.
-- """
-- if ensure_local_user:
-- ensure_user(user, group)
-- priv_key, pub_key = get_keypair(user)
-- hook = hook_name()
-- if hook == '%s-relation-joined' % peer_interface:
-- relation_set(ssh_pub_key=pub_key)
-- elif hook == '%s-relation-changed' % peer_interface:
-- hosts = []
-- keys = []
--
-- for r_id in relation_ids(peer_interface):
-- for unit in related_units(r_id):
-- ssh_pub_key = relation_get('ssh_pub_key',
-- rid=r_id,
-- unit=unit)
-- priv_addr = relation_get('private-address',
-- rid=r_id,
-- unit=unit)
-- if ssh_pub_key:
-- keys.append(ssh_pub_key)
-- hosts.append(priv_addr)
-- else:
-- log('ssh_authorized_peers(): ssh_pub_key '
-- 'missing for unit %s, skipping.' % unit)
-- write_authorized_keys(user, keys)

Juju Charms Collectionpython-django package

Merge lp:~thumper/charms/trusty/python-django/clean-contrib into lp:charms/python-django

Commit message

Description of the change

Preview Diff

Subscribers

Juju Charms Collection
python-django package