Merge lp:~freyes/charms/trusty/memcached/sync-helpers into lp:charms/trusty/memcached
- Trusty Tahr (14.04)
- sync-helpers
- Merge into trunk
Proposed by Felipe Reyes
| Status: | Merged |
|---|---|
| Merged at revision: | 64 |
| Proposed branch: | lp:~freyes/charms/trusty/memcached/sync-helpers |
| Merge into: | lp:charms/trusty/memcached |
| Diff against target: | 837 lines (+476/-33) 21 files modified hooks/charmhelpers/__init__.py (+16/-0) hooks/charmhelpers/contrib/__init__.py (+15/-0) hooks/charmhelpers/contrib/network/__init__.py (+15/-0) hooks/charmhelpers/contrib/network/ip.py (+16/-0) hooks/charmhelpers/contrib/network/ovs/__init__.py (+16/-0) hooks/charmhelpers/contrib/network/ufw.py (+79/-15) hooks/charmhelpers/core/__init__.py (+15/-0) hooks/charmhelpers/core/decorators.py (+57/-0) hooks/charmhelpers/core/fstab.py (+16/-0) hooks/charmhelpers/core/hookenv.py (+16/-0) hooks/charmhelpers/core/host.py (+34/-7) hooks/charmhelpers/core/services/__init__.py (+16/-0) hooks/charmhelpers/core/services/base.py (+16/-0) hooks/charmhelpers/core/services/helpers.py (+16/-0) hooks/charmhelpers/core/sysctl.py (+27/-5) hooks/charmhelpers/core/templating.py (+19/-3) hooks/charmhelpers/fetch/__init__.py (+24/-1) hooks/charmhelpers/fetch/archiveurl.py (+16/-0) hooks/charmhelpers/fetch/bzrurl.py (+25/-1) hooks/charmhelpers/fetch/giturl.py (+20/-0) unit_tests/test_memcached_hooks.py (+2/-1) |
| To merge this branch: | bzr merge lp:~freyes/charms/trusty/memcached/sync-helpers |
| Related bugs: |
| Reviewer | Review Type | Date Requested | Status |
|---|---|---|---|
| Jorge Niedbalski | Pending | ||
| charmers | Pending | ||
| Review via email: | |||
Commit message
Description of the change
Dear Charmers,
This is a MP to syncup the charm helpers and fix some ufw related issues when IPv6 inside a LXC container doesn't work.
Also a minor fix to a test failure after the sync.
Thanks,
To post a comment you must log in.
Revision history for this message 
| Adam Israel (aisrael) wrote : | # |
Revision history for this message
| Adam Israel (aisrael) wrote : | # |
Hey Felipe,
Disregard my last comment. I see now that your change has already been merged with charm-helpers. Thanks again for fixing this!
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
| 1 | === modified file 'hooks/charmhelpers/__init__.py' |
| 2 | --- hooks/charmhelpers/__init__.py 2014-12-18 15:24:05 +0000 |
| 3 | +++ hooks/charmhelpers/__init__.py 2015-02-04 12:57:11 +0000 |
| 4 | @@ -1,3 +1,19 @@ |
| 5 | +# Copyright 2014-2015 Canonical Limited. |
| 6 | +# |
| 7 | +# This file is part of charm-helpers. |
| 8 | +# |
| 9 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 10 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 11 | +# published by the Free Software Foundation. |
| 12 | +# |
| 13 | +# charm-helpers is distributed in the hope that it will be useful, |
| 14 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 15 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 16 | +# GNU Lesser General Public License for more details. |
| 17 | +# |
| 18 | +# You should have received a copy of the GNU Lesser General Public License |
| 19 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 20 | + |
| 21 | # Bootstrap charm-helpers, installing its dependencies if necessary using |
| 22 | # only standard libraries. |
| 23 | import subprocess |
| 24 | |
| 25 | === modified file 'hooks/charmhelpers/contrib/__init__.py' |
| 26 | --- hooks/charmhelpers/contrib/__init__.py 2014-12-04 18:52:50 +0000 |
| 27 | +++ hooks/charmhelpers/contrib/__init__.py 2015-02-04 12:57:11 +0000 |
| 28 | @@ -0,0 +1,15 @@ |
| 29 | +# Copyright 2014-2015 Canonical Limited. |
| 30 | +# |
| 31 | +# This file is part of charm-helpers. |
| 32 | +# |
| 33 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 34 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 35 | +# published by the Free Software Foundation. |
| 36 | +# |
| 37 | +# charm-helpers is distributed in the hope that it will be useful, |
| 38 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 39 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 40 | +# GNU Lesser General Public License for more details. |
| 41 | +# |
| 42 | +# You should have received a copy of the GNU Lesser General Public License |
| 43 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 44 | |
| 45 | === modified file 'hooks/charmhelpers/contrib/network/__init__.py' |
| 46 | --- hooks/charmhelpers/contrib/network/__init__.py 2014-12-04 18:52:50 +0000 |
| 47 | +++ hooks/charmhelpers/contrib/network/__init__.py 2015-02-04 12:57:11 +0000 |
| 48 | @@ -0,0 +1,15 @@ |
| 49 | +# Copyright 2014-2015 Canonical Limited. |
| 50 | +# |
| 51 | +# This file is part of charm-helpers. |
| 52 | +# |
| 53 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 54 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 55 | +# published by the Free Software Foundation. |
| 56 | +# |
| 57 | +# charm-helpers is distributed in the hope that it will be useful, |
| 58 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 59 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 60 | +# GNU Lesser General Public License for more details. |
| 61 | +# |
| 62 | +# You should have received a copy of the GNU Lesser General Public License |
| 63 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 64 | |
| 65 | === modified file 'hooks/charmhelpers/contrib/network/ip.py' |
| 66 | --- hooks/charmhelpers/contrib/network/ip.py 2014-12-04 18:52:50 +0000 |
| 67 | +++ hooks/charmhelpers/contrib/network/ip.py 2015-02-04 12:57:11 +0000 |
| 68 | @@ -1,3 +1,19 @@ |
| 69 | +# Copyright 2014-2015 Canonical Limited. |
| 70 | +# |
| 71 | +# This file is part of charm-helpers. |
| 72 | +# |
| 73 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 74 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 75 | +# published by the Free Software Foundation. |
| 76 | +# |
| 77 | +# charm-helpers is distributed in the hope that it will be useful, |
| 78 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 79 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 80 | +# GNU Lesser General Public License for more details. |
| 81 | +# |
| 82 | +# You should have received a copy of the GNU Lesser General Public License |
| 83 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 84 | + |
| 85 | import glob |
| 86 | import re |
| 87 | import subprocess |
| 88 | |
| 89 | === modified file 'hooks/charmhelpers/contrib/network/ovs/__init__.py' |
| 90 | --- hooks/charmhelpers/contrib/network/ovs/__init__.py 2014-12-04 18:52:50 +0000 |
| 91 | +++ hooks/charmhelpers/contrib/network/ovs/__init__.py 2015-02-04 12:57:11 +0000 |
| 92 | @@ -1,3 +1,19 @@ |
| 93 | +# Copyright 2014-2015 Canonical Limited. |
| 94 | +# |
| 95 | +# This file is part of charm-helpers. |
| 96 | +# |
| 97 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 98 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 99 | +# published by the Free Software Foundation. |
| 100 | +# |
| 101 | +# charm-helpers is distributed in the hope that it will be useful, |
| 102 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 103 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 104 | +# GNU Lesser General Public License for more details. |
| 105 | +# |
| 106 | +# You should have received a copy of the GNU Lesser General Public License |
| 107 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 108 | + |
| 109 | ''' Helpers for interacting with OpenvSwitch ''' |
| 110 | import subprocess |
| 111 | import os |
| 112 | |
| 113 | === modified file 'hooks/charmhelpers/contrib/network/ufw.py' |
| 114 | --- hooks/charmhelpers/contrib/network/ufw.py 2014-12-18 15:24:05 +0000 |
| 115 | +++ hooks/charmhelpers/contrib/network/ufw.py 2015-02-04 12:57:11 +0000 |
| 116 | @@ -1,3 +1,19 @@ |
| 117 | +# Copyright 2014-2015 Canonical Limited. |
| 118 | +# |
| 119 | +# This file is part of charm-helpers. |
| 120 | +# |
| 121 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 122 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 123 | +# published by the Free Software Foundation. |
| 124 | +# |
| 125 | +# charm-helpers is distributed in the hope that it will be useful, |
| 126 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 127 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 128 | +# GNU Lesser General Public License for more details. |
| 129 | +# |
| 130 | +# You should have received a copy of the GNU Lesser General Public License |
| 131 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 132 | + |
| 133 | """ |
| 134 | This module contains helpers to add and remove ufw rules. |
| 135 | |
| 136 | @@ -30,6 +46,10 @@ |
| 137 | from charmhelpers.core import hookenv |
| 138 | |
| 139 | |
| 140 | +class UFWError(Exception): |
| 141 | + pass |
| 142 | + |
| 143 | + |
| 144 | def is_enabled(): |
| 145 | """ |
| 146 | Check if `ufw` is enabled |
| 147 | @@ -37,6 +57,7 @@ |
| 148 | :returns: True if ufw is enabled |
| 149 | """ |
| 150 | output = subprocess.check_output(['ufw', 'status'], |
| 151 | + universal_newlines=True, |
| 152 | env={'LANG': 'en_US', |
| 153 | 'PATH': os.environ['PATH']}) |
| 154 | |
| 155 | @@ -45,6 +66,53 @@ |
| 156 | return len(m) >= 1 |
| 157 | |
| 158 | |
| 159 | +def is_ipv6_ok(): |
| 160 | + """ |
| 161 | + Check if IPv6 support is present and ip6tables functional |
| 162 | + |
| 163 | + :returns: True if IPv6 is working, False otherwise |
| 164 | + """ |
| 165 | + |
| 166 | + # do we have IPv6 in the machine? |
| 167 | + if os.path.isdir('/proc/sys/net/ipv6'): |
| 168 | + # is ip6tables kernel module loaded? |
| 169 | + lsmod = subprocess.check_output(['lsmod'], universal_newlines=True) |
| 170 | + matches = re.findall('^ip6_tables[ ]+', lsmod, re.M) |
| 171 | + if len(matches) == 0: |
| 172 | + # ip6tables support isn't complete, let's try to load it |
| 173 | + try: |
| 174 | + subprocess.check_output(['modprobe', 'ip6_tables'], |
| 175 | + universal_newlines=True) |
| 176 | + # great, we could load the module |
| 177 | + return True |
| 178 | + except subprocess.CalledProcessError as ex: |
| 179 | + hookenv.log("Couldn't load ip6_tables module: %s" % ex.output, |
| 180 | + level="WARN") |
| 181 | + # we are in a world where ip6tables isn't working |
| 182 | + # so we inform that the machine doesn't have IPv6 |
| 183 | + return False |
| 184 | + else: |
| 185 | + # the module is present :) |
| 186 | + return True |
| 187 | + |
| 188 | + else: |
| 189 | + # the system doesn't have IPv6 |
| 190 | + return False |
| 191 | + |
| 192 | + |
| 193 | +def disable_ipv6(): |
| 194 | + """ |
| 195 | + Disable ufw IPv6 support in /etc/default/ufw |
| 196 | + """ |
| 197 | + exit_code = subprocess.call(['sed', '-i', 's/IPV6=.*/IPV6=no/g', |
| 198 | + '/etc/default/ufw']) |
| 199 | + if exit_code == 0: |
| 200 | + hookenv.log('IPv6 support in ufw disabled', level='INFO') |
| 201 | + else: |
| 202 | + hookenv.log("Couldn't disable IPv6 support in ufw", level="ERROR") |
| 203 | + raise UFWError("Couldn't disable IPv6 support in ufw") |
| 204 | + |
| 205 | + |
| 206 | def enable(): |
| 207 | """ |
| 208 | Enable ufw |
| 209 | @@ -54,18 +122,11 @@ |
| 210 | if is_enabled(): |
| 211 | return True |
| 212 | |
| 213 | - if not os.path.isdir('/proc/sys/net/ipv6'): |
| 214 | - # disable IPv6 support in ufw |
| 215 | - hookenv.log("This machine doesn't have IPv6 enabled", level="INFO") |
| 216 | - exit_code = subprocess.call(['sed', '-i', 's/IPV6=yes/IPV6=no/g', |
| 217 | - '/etc/default/ufw']) |
| 218 | - if exit_code == 0: |
| 219 | - hookenv.log('IPv6 support in ufw disabled', level='INFO') |
| 220 | - else: |
| 221 | - hookenv.log("Couldn't disable IPv6 support in ufw", level="ERROR") |
| 222 | - raise Exception("Couldn't disable IPv6 support in ufw") |
| 223 | + if not is_ipv6_ok(): |
| 224 | + disable_ipv6() |
| 225 | |
| 226 | output = subprocess.check_output(['ufw', 'enable'], |
| 227 | + universal_newlines=True, |
| 228 | env={'LANG': 'en_US', |
| 229 | 'PATH': os.environ['PATH']}) |
| 230 | |
| 231 | @@ -91,6 +152,7 @@ |
| 232 | return True |
| 233 | |
| 234 | output = subprocess.check_output(['ufw', 'disable'], |
| 235 | + universal_newlines=True, |
| 236 | env={'LANG': 'en_US', |
| 237 | 'PATH': os.environ['PATH']}) |
| 238 | |
| 239 | @@ -135,7 +197,7 @@ |
| 240 | cmd += ['to', dst] |
| 241 | |
| 242 | if port is not None: |
| 243 | - cmd += ['port', port] |
| 244 | + cmd += ['port', str(port)] |
| 245 | |
| 246 | if proto is not None: |
| 247 | cmd += ['proto', proto] |
| 248 | @@ -192,9 +254,11 @@ |
| 249 | :param action: `open` or `close` |
| 250 | """ |
| 251 | if action == 'open': |
| 252 | - subprocess.check_output(['ufw', 'allow', name]) |
| 253 | + subprocess.check_output(['ufw', 'allow', str(name)], |
| 254 | + universal_newlines=True) |
| 255 | elif action == 'close': |
| 256 | - subprocess.check_output(['ufw', 'delete', 'allow', name]) |
| 257 | + subprocess.check_output(['ufw', 'delete', 'allow', str(name)], |
| 258 | + universal_newlines=True) |
| 259 | else: |
| 260 | - raise Exception(("'{}' not supported, use 'allow' " |
| 261 | - "or 'delete'").format(action)) |
| 262 | + raise UFWError(("'{}' not supported, use 'allow' " |
| 263 | + "or 'delete'").format(action)) |
| 264 | |
| 265 | === modified file 'hooks/charmhelpers/core/__init__.py' |
| 266 | --- hooks/charmhelpers/core/__init__.py 2014-12-01 16:24:03 +0000 |
| 267 | +++ hooks/charmhelpers/core/__init__.py 2015-02-04 12:57:11 +0000 |
| 268 | @@ -0,0 +1,15 @@ |
| 269 | +# Copyright 2014-2015 Canonical Limited. |
| 270 | +# |
| 271 | +# This file is part of charm-helpers. |
| 272 | +# |
| 273 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 274 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 275 | +# published by the Free Software Foundation. |
| 276 | +# |
| 277 | +# charm-helpers is distributed in the hope that it will be useful, |
| 278 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 279 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 280 | +# GNU Lesser General Public License for more details. |
| 281 | +# |
| 282 | +# You should have received a copy of the GNU Lesser General Public License |
| 283 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 284 | |
| 285 | === added file 'hooks/charmhelpers/core/decorators.py' |
| 286 | --- hooks/charmhelpers/core/decorators.py 1970-01-01 00:00:00 +0000 |
| 287 | +++ hooks/charmhelpers/core/decorators.py 2015-02-04 12:57:11 +0000 |
| 288 | @@ -0,0 +1,57 @@ |
| 289 | +# Copyright 2014-2015 Canonical Limited. |
| 290 | +# |
| 291 | +# This file is part of charm-helpers. |
| 292 | +# |
| 293 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 294 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 295 | +# published by the Free Software Foundation. |
| 296 | +# |
| 297 | +# charm-helpers is distributed in the hope that it will be useful, |
| 298 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 299 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 300 | +# GNU Lesser General Public License for more details. |
| 301 | +# |
| 302 | +# You should have received a copy of the GNU Lesser General Public License |
| 303 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 304 | + |
| 305 | +# |
| 306 | +# Copyright 2014 Canonical Ltd. |
| 307 | +# |
| 308 | +# Authors: |
| 309 | +# Edward Hope-Morley <opentastic@gmail.com> |
| 310 | +# |
| 311 | + |
| 312 | +import time |
| 313 | + |
| 314 | +from charmhelpers.core.hookenv import ( |
| 315 | + log, |
| 316 | + INFO, |
| 317 | +) |
| 318 | + |
| 319 | + |
| 320 | +def retry_on_exception(num_retries, base_delay=0, exc_type=Exception): |
| 321 | + """If the decorated function raises exception exc_type, allow num_retries |
| 322 | + retry attempts before raise the exception. |
| 323 | + """ |
| 324 | + def _retry_on_exception_inner_1(f): |
| 325 | + def _retry_on_exception_inner_2(*args, **kwargs): |
| 326 | + retries = num_retries |
| 327 | + multiplier = 1 |
| 328 | + while True: |
| 329 | + try: |
| 330 | + return f(*args, **kwargs) |
| 331 | + except exc_type: |
| 332 | + if not retries: |
| 333 | + raise |
| 334 | + |
| 335 | + delay = base_delay * multiplier |
| 336 | + multiplier += 1 |
| 337 | + log("Retrying '%s' %d more times (delay=%s)" % |
| 338 | + (f.__name__, retries, delay), level=INFO) |
| 339 | + retries -= 1 |
| 340 | + if delay: |
| 341 | + time.sleep(delay) |
| 342 | + |
| 343 | + return _retry_on_exception_inner_2 |
| 344 | + |
| 345 | + return _retry_on_exception_inner_1 |
| 346 | |
| 347 | === modified file 'hooks/charmhelpers/core/fstab.py' |
| 348 | --- hooks/charmhelpers/core/fstab.py 2014-12-01 16:24:03 +0000 |
| 349 | +++ hooks/charmhelpers/core/fstab.py 2015-02-04 12:57:11 +0000 |
| 350 | @@ -1,6 +1,22 @@ |
| 351 | #!/usr/bin/env python |
| 352 | # -*- coding: utf-8 -*- |
| 353 | |
| 354 | +# Copyright 2014-2015 Canonical Limited. |
| 355 | +# |
| 356 | +# This file is part of charm-helpers. |
| 357 | +# |
| 358 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 359 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 360 | +# published by the Free Software Foundation. |
| 361 | +# |
| 362 | +# charm-helpers is distributed in the hope that it will be useful, |
| 363 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 364 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 365 | +# GNU Lesser General Public License for more details. |
| 366 | +# |
| 367 | +# You should have received a copy of the GNU Lesser General Public License |
| 368 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 369 | + |
| 370 | __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>' |
| 371 | |
| 372 | import io |
| 373 | |
| 374 | === modified file 'hooks/charmhelpers/core/hookenv.py' |
| 375 | --- hooks/charmhelpers/core/hookenv.py 2014-12-18 15:24:05 +0000 |
| 376 | +++ hooks/charmhelpers/core/hookenv.py 2015-02-04 12:57:11 +0000 |
| 377 | @@ -1,3 +1,19 @@ |
| 378 | +# Copyright 2014-2015 Canonical Limited. |
| 379 | +# |
| 380 | +# This file is part of charm-helpers. |
| 381 | +# |
| 382 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 383 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 384 | +# published by the Free Software Foundation. |
| 385 | +# |
| 386 | +# charm-helpers is distributed in the hope that it will be useful, |
| 387 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 388 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 389 | +# GNU Lesser General Public License for more details. |
| 390 | +# |
| 391 | +# You should have received a copy of the GNU Lesser General Public License |
| 392 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 393 | + |
| 394 | "Interactions with the Juju environment" |
| 395 | # Copyright 2013 Canonical Ltd. |
| 396 | # |
| 397 | |
| 398 | === modified file 'hooks/charmhelpers/core/host.py' |
| 399 | --- hooks/charmhelpers/core/host.py 2014-12-18 15:24:05 +0000 |
| 400 | +++ hooks/charmhelpers/core/host.py 2015-02-04 12:57:11 +0000 |
| 401 | @@ -1,3 +1,19 @@ |
| 402 | +# Copyright 2014-2015 Canonical Limited. |
| 403 | +# |
| 404 | +# This file is part of charm-helpers. |
| 405 | +# |
| 406 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 407 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 408 | +# published by the Free Software Foundation. |
| 409 | +# |
| 410 | +# charm-helpers is distributed in the hope that it will be useful, |
| 411 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 412 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 413 | +# GNU Lesser General Public License for more details. |
| 414 | +# |
| 415 | +# You should have received a copy of the GNU Lesser General Public License |
| 416 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 417 | + |
| 418 | """Tools for working with the host system""" |
| 419 | # Copyright 2012 Canonical Ltd. |
| 420 | # |
| 421 | @@ -168,18 +184,18 @@ |
| 422 | log("Removing non-directory file {} prior to mkdir()".format(path)) |
| 423 | os.unlink(realpath) |
| 424 | os.makedirs(realpath, perms) |
| 425 | - os.chown(realpath, uid, gid) |
| 426 | elif not path_exists: |
| 427 | os.makedirs(realpath, perms) |
| 428 | - os.chown(realpath, uid, gid) |
| 429 | + os.chown(realpath, uid, gid) |
| 430 | + os.chmod(realpath, perms) |
| 431 | |
| 432 | |
| 433 | def write_file(path, content, owner='root', group='root', perms=0o444): |
| 434 | - """Create or overwrite a file with the contents of a string""" |
| 435 | + """Create or overwrite a file with the contents of a byte string.""" |
| 436 | log("Writing file {} {}:{} {:o}".format(path, owner, group, perms)) |
| 437 | uid = pwd.getpwnam(owner).pw_uid |
| 438 | gid = grp.getgrnam(group).gr_gid |
| 439 | - with open(path, 'w') as target: |
| 440 | + with open(path, 'wb') as target: |
| 441 | os.fchown(target.fileno(), uid, gid) |
| 442 | os.fchmod(target.fileno(), perms) |
| 443 | target.write(content) |
| 444 | @@ -345,7 +361,7 @@ |
| 445 | ip_output = (line for line in ip_output if line) |
| 446 | for line in ip_output: |
| 447 | if line.split()[1].startswith(int_type): |
| 448 | - matched = re.search('.*: (bond[0-9]+\.[0-9]+)@.*', line) |
| 449 | + matched = re.search('.*: (' + int_type + r'[0-9]+\.[0-9]+)@.*', line) |
| 450 | if matched: |
| 451 | interface = matched.groups()[0] |
| 452 | else: |
| 453 | @@ -389,6 +405,9 @@ |
| 454 | * 0 => Installed revno is the same as supplied arg |
| 455 | * -1 => Installed revno is less than supplied arg |
| 456 | |
| 457 | + This function imports apt_cache function from charmhelpers.fetch if |
| 458 | + the pkgcache argument is None. Be sure to add charmhelpers.fetch if |
| 459 | + you call this function, or pass an apt_pkg.Cache() instance. |
| 460 | ''' |
| 461 | import apt_pkg |
| 462 | if not pkgcache: |
| 463 | @@ -407,13 +426,21 @@ |
| 464 | os.chdir(cur) |
| 465 | |
| 466 | |
| 467 | -def chownr(path, owner, group): |
| 468 | +def chownr(path, owner, group, follow_links=True): |
| 469 | uid = pwd.getpwnam(owner).pw_uid |
| 470 | gid = grp.getgrnam(group).gr_gid |
| 471 | + if follow_links: |
| 472 | + chown = os.chown |
| 473 | + else: |
| 474 | + chown = os.lchown |
| 475 | |
| 476 | for root, dirs, files in os.walk(path): |
| 477 | for name in dirs + files: |
| 478 | full = os.path.join(root, name) |
| 479 | broken_symlink = os.path.lexists(full) and not os.path.exists(full) |
| 480 | if not broken_symlink: |
| 481 | - os.chown(full, uid, gid) |
| 482 | + chown(full, uid, gid) |
| 483 | + |
| 484 | + |
| 485 | +def lchownr(path, owner, group): |
| 486 | + chownr(path, owner, group, follow_links=False) |
| 487 | |
| 488 | === modified file 'hooks/charmhelpers/core/services/__init__.py' |
| 489 | --- hooks/charmhelpers/core/services/__init__.py 2014-12-01 16:24:03 +0000 |
| 490 | +++ hooks/charmhelpers/core/services/__init__.py 2015-02-04 12:57:11 +0000 |
| 491 | @@ -1,2 +1,18 @@ |
| 492 | +# Copyright 2014-2015 Canonical Limited. |
| 493 | +# |
| 494 | +# This file is part of charm-helpers. |
| 495 | +# |
| 496 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 497 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 498 | +# published by the Free Software Foundation. |
| 499 | +# |
| 500 | +# charm-helpers is distributed in the hope that it will be useful, |
| 501 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 502 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 503 | +# GNU Lesser General Public License for more details. |
| 504 | +# |
| 505 | +# You should have received a copy of the GNU Lesser General Public License |
| 506 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 507 | + |
| 508 | from .base import * # NOQA |
| 509 | from .helpers import * # NOQA |
| 510 | |
| 511 | === modified file 'hooks/charmhelpers/core/services/base.py' |
| 512 | --- hooks/charmhelpers/core/services/base.py 2014-12-01 16:24:03 +0000 |
| 513 | +++ hooks/charmhelpers/core/services/base.py 2015-02-04 12:57:11 +0000 |
| 514 | @@ -1,3 +1,19 @@ |
| 515 | +# Copyright 2014-2015 Canonical Limited. |
| 516 | +# |
| 517 | +# This file is part of charm-helpers. |
| 518 | +# |
| 519 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 520 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 521 | +# published by the Free Software Foundation. |
| 522 | +# |
| 523 | +# charm-helpers is distributed in the hope that it will be useful, |
| 524 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 525 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 526 | +# GNU Lesser General Public License for more details. |
| 527 | +# |
| 528 | +# You should have received a copy of the GNU Lesser General Public License |
| 529 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 530 | + |
| 531 | import os |
| 532 | import re |
| 533 | import json |
| 534 | |
| 535 | === modified file 'hooks/charmhelpers/core/services/helpers.py' |
| 536 | --- hooks/charmhelpers/core/services/helpers.py 2014-12-01 16:24:03 +0000 |
| 537 | +++ hooks/charmhelpers/core/services/helpers.py 2015-02-04 12:57:11 +0000 |
| 538 | @@ -1,3 +1,19 @@ |
| 539 | +# Copyright 2014-2015 Canonical Limited. |
| 540 | +# |
| 541 | +# This file is part of charm-helpers. |
| 542 | +# |
| 543 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 544 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 545 | +# published by the Free Software Foundation. |
| 546 | +# |
| 547 | +# charm-helpers is distributed in the hope that it will be useful, |
| 548 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 549 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 550 | +# GNU Lesser General Public License for more details. |
| 551 | +# |
| 552 | +# You should have received a copy of the GNU Lesser General Public License |
| 553 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 554 | + |
| 555 | import os |
| 556 | import yaml |
| 557 | from charmhelpers.core import hookenv |
| 558 | |
| 559 | === modified file 'hooks/charmhelpers/core/sysctl.py' |
| 560 | --- hooks/charmhelpers/core/sysctl.py 2014-12-01 16:24:03 +0000 |
| 561 | +++ hooks/charmhelpers/core/sysctl.py 2015-02-04 12:57:11 +0000 |
| 562 | @@ -1,6 +1,22 @@ |
| 563 | #!/usr/bin/env python |
| 564 | # -*- coding: utf-8 -*- |
| 565 | |
| 566 | +# Copyright 2014-2015 Canonical Limited. |
| 567 | +# |
| 568 | +# This file is part of charm-helpers. |
| 569 | +# |
| 570 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 571 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 572 | +# published by the Free Software Foundation. |
| 573 | +# |
| 574 | +# charm-helpers is distributed in the hope that it will be useful, |
| 575 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 576 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 577 | +# GNU Lesser General Public License for more details. |
| 578 | +# |
| 579 | +# You should have received a copy of the GNU Lesser General Public License |
| 580 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 581 | + |
| 582 | __author__ = 'Jorge Niedbalski R. <jorge.niedbalski@canonical.com>' |
| 583 | |
| 584 | import yaml |
| 585 | @@ -10,25 +26,31 @@ |
| 586 | from charmhelpers.core.hookenv import ( |
| 587 | log, |
| 588 | DEBUG, |
| 589 | + ERROR, |
| 590 | ) |
| 591 | |
| 592 | |
| 593 | def create(sysctl_dict, sysctl_file): |
| 594 | """Creates a sysctl.conf file from a YAML associative array |
| 595 | |
| 596 | - :param sysctl_dict: a dict of sysctl options eg { 'kernel.max_pid': 1337 } |
| 597 | - :type sysctl_dict: dict |
| 598 | + :param sysctl_dict: a YAML-formatted string of sysctl options eg "{ 'kernel.max_pid': 1337 }" |
| 599 | + :type sysctl_dict: str |
| 600 | :param sysctl_file: path to the sysctl file to be saved |
| 601 | :type sysctl_file: str or unicode |
| 602 | :returns: None |
| 603 | """ |
| 604 | - sysctl_dict = yaml.load(sysctl_dict) |
| 605 | + try: |
| 606 | + sysctl_dict_parsed = yaml.safe_load(sysctl_dict) |
| 607 | + except yaml.YAMLError: |
| 608 | + log("Error parsing YAML sysctl_dict: {}".format(sysctl_dict), |
| 609 | + level=ERROR) |
| 610 | + return |
| 611 | |
| 612 | with open(sysctl_file, "w") as fd: |
| 613 | - for key, value in sysctl_dict.items(): |
| 614 | + for key, value in sysctl_dict_parsed.items(): |
| 615 | fd.write("{}={}\n".format(key, value)) |
| 616 | |
| 617 | - log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict), |
| 618 | + log("Updating sysctl_file: %s values: %s" % (sysctl_file, sysctl_dict_parsed), |
| 619 | level=DEBUG) |
| 620 | |
| 621 | check_call(["sysctl", "-p", sysctl_file]) |
| 622 | |
| 623 | === modified file 'hooks/charmhelpers/core/templating.py' |
| 624 | --- hooks/charmhelpers/core/templating.py 2014-12-18 15:24:05 +0000 |
| 625 | +++ hooks/charmhelpers/core/templating.py 2015-02-04 12:57:11 +0000 |
| 626 | @@ -1,3 +1,19 @@ |
| 627 | +# Copyright 2014-2015 Canonical Limited. |
| 628 | +# |
| 629 | +# This file is part of charm-helpers. |
| 630 | +# |
| 631 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 632 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 633 | +# published by the Free Software Foundation. |
| 634 | +# |
| 635 | +# charm-helpers is distributed in the hope that it will be useful, |
| 636 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 637 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 638 | +# GNU Lesser General Public License for more details. |
| 639 | +# |
| 640 | +# You should have received a copy of the GNU Lesser General Public License |
| 641 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 642 | + |
| 643 | import os |
| 644 | |
| 645 | from charmhelpers.core import host |
| 646 | @@ -5,7 +21,7 @@ |
| 647 | |
| 648 | |
| 649 | def render(source, target, context, owner='root', group='root', |
| 650 | - perms=0o444, templates_dir=None): |
| 651 | + perms=0o444, templates_dir=None, encoding='UTF-8'): |
| 652 | """ |
| 653 | Render a template. |
| 654 | |
| 655 | @@ -48,5 +64,5 @@ |
| 656 | level=hookenv.ERROR) |
| 657 | raise e |
| 658 | content = template.render(context) |
| 659 | - host.mkdir(os.path.dirname(target), owner, group) |
| 660 | - host.write_file(target, content, owner, group, perms) |
| 661 | + host.mkdir(os.path.dirname(target), owner, group, perms=0o755) |
| 662 | + host.write_file(target, content.encode(encoding), owner, group, perms) |
| 663 | |
| 664 | === modified file 'hooks/charmhelpers/fetch/__init__.py' |
| 665 | --- hooks/charmhelpers/fetch/__init__.py 2014-12-01 16:54:37 +0000 |
| 666 | +++ hooks/charmhelpers/fetch/__init__.py 2015-02-04 12:57:11 +0000 |
| 667 | @@ -1,3 +1,19 @@ |
| 668 | +# Copyright 2014-2015 Canonical Limited. |
| 669 | +# |
| 670 | +# This file is part of charm-helpers. |
| 671 | +# |
| 672 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 673 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 674 | +# published by the Free Software Foundation. |
| 675 | +# |
| 676 | +# charm-helpers is distributed in the hope that it will be useful, |
| 677 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 678 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 679 | +# GNU Lesser General Public License for more details. |
| 680 | +# |
| 681 | +# You should have received a copy of the GNU Lesser General Public License |
| 682 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 683 | + |
| 684 | import importlib |
| 685 | from tempfile import NamedTemporaryFile |
| 686 | import time |
| 687 | @@ -64,9 +80,16 @@ |
| 688 | 'trusty-juno/updates': 'trusty-updates/juno', |
| 689 | 'trusty-updates/juno': 'trusty-updates/juno', |
| 690 | 'juno/proposed': 'trusty-proposed/juno', |
| 691 | - 'juno/proposed': 'trusty-proposed/juno', |
| 692 | 'trusty-juno/proposed': 'trusty-proposed/juno', |
| 693 | 'trusty-proposed/juno': 'trusty-proposed/juno', |
| 694 | + # Kilo |
| 695 | + 'kilo': 'trusty-updates/kilo', |
| 696 | + 'trusty-kilo': 'trusty-updates/kilo', |
| 697 | + 'trusty-kilo/updates': 'trusty-updates/kilo', |
| 698 | + 'trusty-updates/kilo': 'trusty-updates/kilo', |
| 699 | + 'kilo/proposed': 'trusty-proposed/kilo', |
| 700 | + 'trusty-kilo/proposed': 'trusty-proposed/kilo', |
| 701 | + 'trusty-proposed/kilo': 'trusty-proposed/kilo', |
| 702 | } |
| 703 | |
| 704 | # The order of this list is very important. Handlers should be listed in from |
| 705 | |
| 706 | === modified file 'hooks/charmhelpers/fetch/archiveurl.py' |
| 707 | --- hooks/charmhelpers/fetch/archiveurl.py 2014-12-01 16:54:37 +0000 |
| 708 | +++ hooks/charmhelpers/fetch/archiveurl.py 2015-02-04 12:57:11 +0000 |
| 709 | @@ -1,3 +1,19 @@ |
| 710 | +# Copyright 2014-2015 Canonical Limited. |
| 711 | +# |
| 712 | +# This file is part of charm-helpers. |
| 713 | +# |
| 714 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 715 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 716 | +# published by the Free Software Foundation. |
| 717 | +# |
| 718 | +# charm-helpers is distributed in the hope that it will be useful, |
| 719 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 720 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 721 | +# GNU Lesser General Public License for more details. |
| 722 | +# |
| 723 | +# You should have received a copy of the GNU Lesser General Public License |
| 724 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 725 | + |
| 726 | import os |
| 727 | import hashlib |
| 728 | import re |
| 729 | |
| 730 | === modified file 'hooks/charmhelpers/fetch/bzrurl.py' |
| 731 | --- hooks/charmhelpers/fetch/bzrurl.py 2014-12-01 16:54:37 +0000 |
| 732 | +++ hooks/charmhelpers/fetch/bzrurl.py 2015-02-04 12:57:11 +0000 |
| 733 | @@ -1,3 +1,19 @@ |
| 734 | +# Copyright 2014-2015 Canonical Limited. |
| 735 | +# |
| 736 | +# This file is part of charm-helpers. |
| 737 | +# |
| 738 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 739 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 740 | +# published by the Free Software Foundation. |
| 741 | +# |
| 742 | +# charm-helpers is distributed in the hope that it will be useful, |
| 743 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 744 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 745 | +# GNU Lesser General Public License for more details. |
| 746 | +# |
| 747 | +# You should have received a copy of the GNU Lesser General Public License |
| 748 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 749 | + |
| 750 | import os |
| 751 | from charmhelpers.fetch import ( |
| 752 | BaseFetchHandler, |
| 753 | @@ -11,10 +27,12 @@ |
| 754 | |
| 755 | try: |
| 756 | from bzrlib.branch import Branch |
| 757 | + from bzrlib import bzrdir, workingtree, errors |
| 758 | except ImportError: |
| 759 | from charmhelpers.fetch import apt_install |
| 760 | apt_install("python-bzrlib") |
| 761 | from bzrlib.branch import Branch |
| 762 | + from bzrlib import bzrdir, workingtree, errors |
| 763 | |
| 764 | |
| 765 | class BzrUrlFetchHandler(BaseFetchHandler): |
| 766 | @@ -35,8 +53,14 @@ |
| 767 | from bzrlib.plugin import load_plugins |
| 768 | load_plugins() |
| 769 | try: |
| 770 | + local_branch = bzrdir.BzrDir.create_branch_convenience(dest) |
| 771 | + except errors.AlreadyControlDirError: |
| 772 | + local_branch = Branch.open(dest) |
| 773 | + try: |
| 774 | remote_branch = Branch.open(source) |
| 775 | - remote_branch.bzrdir.sprout(dest).open_branch() |
| 776 | + remote_branch.push(local_branch) |
| 777 | + tree = workingtree.WorkingTree.open(dest) |
| 778 | + tree.update() |
| 779 | except Exception as e: |
| 780 | raise e |
| 781 | |
| 782 | |
| 783 | === modified file 'hooks/charmhelpers/fetch/giturl.py' |
| 784 | --- hooks/charmhelpers/fetch/giturl.py 2014-12-04 18:52:50 +0000 |
| 785 | +++ hooks/charmhelpers/fetch/giturl.py 2015-02-04 12:57:11 +0000 |
| 786 | @@ -1,3 +1,19 @@ |
| 787 | +# Copyright 2014-2015 Canonical Limited. |
| 788 | +# |
| 789 | +# This file is part of charm-helpers. |
| 790 | +# |
| 791 | +# charm-helpers is free software: you can redistribute it and/or modify |
| 792 | +# it under the terms of the GNU Lesser General Public License version 3 as |
| 793 | +# published by the Free Software Foundation. |
| 794 | +# |
| 795 | +# charm-helpers is distributed in the hope that it will be useful, |
| 796 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
| 797 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
| 798 | +# GNU Lesser General Public License for more details. |
| 799 | +# |
| 800 | +# You should have received a copy of the GNU Lesser General Public License |
| 801 | +# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>. |
| 802 | + |
| 803 | import os |
| 804 | from charmhelpers.fetch import ( |
| 805 | BaseFetchHandler, |
| 806 | @@ -16,6 +32,8 @@ |
| 807 | apt_install("python-git") |
| 808 | from git import Repo |
| 809 | |
| 810 | +from git.exc import GitCommandError |
| 811 | + |
| 812 | |
| 813 | class GitUrlFetchHandler(BaseFetchHandler): |
| 814 | """Handler for git branches via generic and github URLs""" |
| 815 | @@ -46,6 +64,8 @@ |
| 816 | mkdir(dest_dir, perms=0o755) |
| 817 | try: |
| 818 | self.clone(source, dest_dir, branch) |
| 819 | + except GitCommandError as e: |
| 820 | + raise UnhandledSource(e.message) |
| 821 | except OSError as e: |
| 822 | raise UnhandledSource(e.strerror) |
| 823 | return dest_dir |
| 824 | |
| 825 | === modified file 'unit_tests/test_memcached_hooks.py' |
| 826 | --- unit_tests/test_memcached_hooks.py 2014-12-19 18:24:29 +0000 |
| 827 | +++ unit_tests/test_memcached_hooks.py 2015-02-04 12:57:11 +0000 |
| 828 | @@ -59,7 +59,8 @@ |
| 829 | self.apt_update.assert_called_with(fatal=True) |
| 830 | self.apt_install.assert_called_with(["memcached", "python-cheetah", |
| 831 | "python-memcache"], fatal=True) |
| 832 | - check_output.assert_any_call(['ufw', 'allow', 'ssh']) |
| 833 | + check_output.assert_any_call(['ufw', 'allow', 'ssh'], |
| 834 | + universal_newlines=True) |
| 835 | |
| 836 | def test_start(self): |
| 837 | memcached_hooks.start() |
Hi Felipe,
I stumbled across this MP while trying to run unit tests, under LXC, for Wordpress, which uses the memcached charm.
This branch does fix the issue with ufw and memcached, but I wonder if this would be better applied directly to lp:~charm-helpers, so more people can benefit from your work?