Implement idempotency behaviour
This change implements idempotency so that apply can be run multiple
times with the same config ID. The aim of the idempotency behaviour is
to leave containers running when their config has not changed, but
replace containers which have modified config.
The logic sequence for idempotency is as follows:
- For each existing container with a matching config_id and
managed_by:
- delete containers which no longer exist in config
- delete containers with missing config_data label
- delete containers where config_data label differs from current
config
- Do a full rename to desired names since deletes have occured
- Only create containers from config if there is no container running
with that name
- exec actions will be run regardless, so commands they run may require
their own idempotency behaviour
This change won't modify the behaviour of docker-cmd hook idempotency
since config IDs are never reused.
Change-Id: I29d07f7910258495804477d08de6040116527e8e
This commit is contained in:
Steve Baker
parent
903bc389b6
commit
82d6ff40fd
38
README.rst
38
README.rst
|
|
@ -69,8 +69,8 @@ Now lets try running the exact same ``paunch apply`` command:
|
|||
|
||||
$ paunch --verbose apply --file examples/hello-world.yml --config-id hi
|
||||
|
||||
This will fail with an error because there already exists a container labeled
|
||||
with ``"config_id": "hi"``. **WARNING TODO NOT IMPLEMENTED YET**
|
||||
This will not make any changes at all due to the idempotency behaviour of
|
||||
paunch.
|
||||
|
||||
Lets try again with a unique --config-id:
|
||||
|
||||
|
|
@ -115,6 +115,38 @@ This will result in a ``hello`` container being run, which will be deleted the
|
|||
next time the ``docker-cmd`` hook does its own ``cleanup`` run since it won't
|
||||
be aware of a ``config_id`` called ``hi``.
|
||||
|
||||
Idempotency Behaviour
|
||||
---------------------
|
||||
|
||||
In many cases the user will want to use the same --config-id with changed
|
||||
config data. The aim of the idempotency behaviour is to leave containers
|
||||
running when their config has not changed, but replace containers which have
|
||||
modified config.
|
||||
|
||||
When ``paunch apply`` is run with the same ``--config-id`` but modified config
|
||||
data, the following logic is applied:
|
||||
|
||||
* For each existing container with a matching config_id and managed_by:
|
||||
* delete containers which no longer exist in config
|
||||
* delete containers with missing config_data label
|
||||
* delete containers where config_data label differs from current config
|
||||
* Do a full rename to desired names since deletes have occured
|
||||
* Only create containers from config if there is no container running with that name
|
||||
* ``exec`` actions will be run regardless, so commands they run may require
|
||||
their own idempotency behaviour
|
||||
|
||||
Only configuration data is used to determine whether something has changed to
|
||||
trigger replacing the container during ``apply``. This means that changing the
|
||||
contents of a file referred to in ``env_file`` will *not* trigger replacement
|
||||
unless something else changes in the configuration data (such as the path
|
||||
specified in ``env_file``).
|
||||
|
||||
The most common reason to restart containers is to have them running with an
|
||||
updated image. As such it is recommended that stable image tags such as
|
||||
``latest`` are not used when specifying the ``image``, and that changing the
|
||||
release version tag in the configuration data is the recommended way of
|
||||
propagating image changes to the running containers.
|
||||
|
||||
Configuration Format
|
||||
--------------------
|
||||
|
||||
|
|
@ -155,7 +187,7 @@ env_file:
|
|||
|
||||
image:
|
||||
String, mandatory. Specify the image to start the container from. Can either
|
||||
be a repository/tag or a partial image ID.
|
||||
be a repositorys/tag or a partial image ID.
|
||||
|
||||
net:
|
||||
String. Set the network mode for the container.
|
||||
|
|
|
|||
|
|
@ -27,16 +27,26 @@ class ComposeV1Builder(object):
|
|||
|
||||
def apply(self):
|
||||
|
||||
self.delete_missing_and_updated()
|
||||
|
||||
stdout = []
|
||||
stderr = []
|
||||
deploy_status_code = 0
|
||||
key_fltr = lambda k: self.config[k].get('start_order', 0)
|
||||
|
||||
container_names = self.runner.container_names(self.config_id)
|
||||
desired_names = set([cn[-1] for cn in container_names])
|
||||
|
||||
for container in sorted(self.config, key=key_fltr):
|
||||
LOG.debug("Running container: %s" % container)
|
||||
action = self.config[container].get('action', 'run')
|
||||
exit_codes = self.config[container].get('exit_codes', [0])
|
||||
|
||||
if action == 'run':
|
||||
if container in desired_names:
|
||||
LOG.debug('Skipping existing container: %s' % container)
|
||||
continue
|
||||
|
||||
cmd = [
|
||||
self.runner.docker_cmd,
|
||||
'run',
|
||||
|
|
@ -62,6 +72,40 @@ class ComposeV1Builder(object):
|
|||
LOG.debug('Completed $ %s' % ' '.join(cmd))
|
||||
return stdout, stderr, deploy_status_code
|
||||
|
||||
def delete_missing_and_updated(self):
|
||||
container_names = self.runner.container_names(self.config_id)
|
||||
for cn in container_names:
|
||||
container = cn[0]
|
||||
|
||||
# if the desired name is not in the config, delete it
|
||||
if cn[-1] not in self.config:
|
||||
LOG.debug("Deleting container (removed): %s" % container)
|
||||
self.runner.remove_container(container)
|
||||
continue
|
||||
|
||||
ex_data_str = self.runner.inspect(
|
||||
container, '{{index .Config.Labels "config_data"}}')
|
||||
if not ex_data_str:
|
||||
LOG.debug("Deleting container (no config_data): %s"
|
||||
% container)
|
||||
self.runner.remove_container(container)
|
||||
continue
|
||||
|
||||
try:
|
||||
ex_data = json.loads(ex_data_str)
|
||||
except Exception:
|
||||
ex_data = None
|
||||
|
||||
new_data = self.config.get(cn[-1])
|
||||
if new_data != ex_data:
|
||||
LOG.debug("Deleting container (changed config_data): %s"
|
||||
% container)
|
||||
self.runner.remove_container(container)
|
||||
|
||||
# deleting containers is an opportunity for renames to their
|
||||
# preferred name
|
||||
self.runner.rename_containers()
|
||||
|
||||
def label_arguments(self, cmd, container):
|
||||
if self.labels:
|
||||
for i, v in self.labels.items():
|
||||
|
|
|
|||
|
|
@ -17,13 +17,13 @@ import json
|
|||
import mock
|
||||
|
||||
from paunch.builder import compose1
|
||||
from paunch import runner
|
||||
from paunch.tests import base
|
||||
|
||||
|
||||
class TestComposeV1Builder(base.TestCase):
|
||||
|
||||
@mock.patch('paunch.runner.DockerRunner', autospec=True)
|
||||
def test_apply(self, runner):
|
||||
def test_apply(self):
|
||||
config = {
|
||||
'one': {
|
||||
'start_order': 0,
|
||||
|
|
@ -48,20 +48,56 @@ class TestComposeV1Builder(base.TestCase):
|
|||
}
|
||||
}
|
||||
|
||||
r = runner.return_value
|
||||
r.managed_by = 'tester'
|
||||
r = runner.DockerRunner(managed_by='tester', docker_cmd='docker')
|
||||
exe = mock.Mock()
|
||||
exe.side_effect = [
|
||||
('', '', 0), # ps for delete_missing_and_updated container_names
|
||||
('', '', 0), # ps for after delete_missing_and_updated renames
|
||||
('', '', 0), # ps to only create containers which don't exist
|
||||
('Created one-12345678', '', 0),
|
||||
('Created two-12345678', '', 0),
|
||||
('Created three-12345678', '', 0),
|
||||
('Created four-12345678', '', 0),
|
||||
('a\nb\nc', '', 0)
|
||||
]
|
||||
r.discover_container_name = lambda n, c: '%s-12345678' % n
|
||||
r.unique_container_name = lambda n: '%s-12345678' % n
|
||||
r.docker_cmd = 'docker'
|
||||
r.execute.return_value = ('Done!', '', 0)
|
||||
r.execute = exe
|
||||
|
||||
builder = compose1.ComposeV1Builder('foo', config, r)
|
||||
stdout, stderr, deploy_status_code = builder.apply()
|
||||
self.assertEqual(0, deploy_status_code)
|
||||
self.assertEqual(['Done!', 'Done!', 'Done!', 'Done!', 'Done!'], stdout)
|
||||
self.assertEqual([
|
||||
'Created one-12345678',
|
||||
'Created two-12345678',
|
||||
'Created three-12345678',
|
||||
'Created four-12345678',
|
||||
'a\nb\nc'
|
||||
], stdout)
|
||||
self.assertEqual([], stderr)
|
||||
|
||||
r.execute.assert_has_calls([
|
||||
exe.assert_has_calls([
|
||||
# ps for delete_missing_and_updated container_names
|
||||
mock.call(
|
||||
['docker', 'ps', '-a',
|
||||
'--filter', 'label=managed_by=tester',
|
||||
'--filter', 'label=config_id=foo',
|
||||
'--format', '{{.Names}} {{.Label "container_name"}}']
|
||||
),
|
||||
# ps for after delete_missing_and_updated renames
|
||||
mock.call(
|
||||
['docker', 'ps', '-a',
|
||||
'--filter', 'label=managed_by=tester',
|
||||
'--format', '{{.Names}} {{.Label "container_name"}}']
|
||||
),
|
||||
# ps to only create containers which don't exist
|
||||
mock.call(
|
||||
['docker', 'ps', '-a',
|
||||
'--filter', 'label=managed_by=tester',
|
||||
'--filter', 'label=config_id=foo',
|
||||
'--format', '{{.Names}} {{.Label "container_name"}}']
|
||||
),
|
||||
# run one
|
||||
mock.call(
|
||||
['docker', 'run', '--name', 'one-12345678',
|
||||
'--label', 'config_id=foo',
|
||||
|
|
@ -70,6 +106,7 @@ class TestComposeV1Builder(base.TestCase):
|
|||
'--label', 'config_data=%s' % json.dumps(config['one']),
|
||||
'--detach=true', 'centos:7']
|
||||
),
|
||||
# run two
|
||||
mock.call(
|
||||
['docker', 'run', '--name', 'two-12345678',
|
||||
'--label', 'config_id=foo',
|
||||
|
|
@ -78,6 +115,7 @@ class TestComposeV1Builder(base.TestCase):
|
|||
'--label', 'config_data=%s' % json.dumps(config['two']),
|
||||
'--detach=true', 'centos:7']
|
||||
),
|
||||
# run three
|
||||
mock.call(
|
||||
['docker', 'run', '--name', 'three-12345678',
|
||||
'--label', 'config_id=foo',
|
||||
|
|
@ -86,6 +124,7 @@ class TestComposeV1Builder(base.TestCase):
|
|||
'--label', 'config_data=%s' % json.dumps(config['three']),
|
||||
'--detach=true', 'centos:7']
|
||||
),
|
||||
# run four
|
||||
mock.call(
|
||||
['docker', 'run', '--name', 'four-12345678',
|
||||
'--label', 'config_id=foo',
|
||||
|
|
@ -94,6 +133,145 @@ class TestComposeV1Builder(base.TestCase):
|
|||
'--label', 'config_data=%s' % json.dumps(config['four']),
|
||||
'--detach=true', 'centos:7']
|
||||
),
|
||||
# execute within four
|
||||
mock.call(
|
||||
['docker', 'exec', 'four-12345678', 'ls', '-l', '/']
|
||||
),
|
||||
])
|
||||
|
||||
def test_apply_idempotency(self):
|
||||
config = {
|
||||
# not running yet
|
||||
'one': {
|
||||
'start_order': 0,
|
||||
'image': 'centos:7',
|
||||
},
|
||||
# running, but with a different config
|
||||
'two': {
|
||||
'start_order': 1,
|
||||
'image': 'centos:7',
|
||||
},
|
||||
# running with the same config
|
||||
'three': {
|
||||
'start_order': 2,
|
||||
'image': 'centos:7',
|
||||
},
|
||||
# not running yet
|
||||
'four': {
|
||||
'start_order': 10,
|
||||
'image': 'centos:7',
|
||||
},
|
||||
'four_ls': {
|
||||
'action': 'exec',
|
||||
'start_order': 20,
|
||||
'command': ['four', 'ls', '-l', '/']
|
||||
}
|
||||
}
|
||||
|
||||
r = runner.DockerRunner(managed_by='tester', docker_cmd='docker')
|
||||
exe = mock.Mock()
|
||||
exe.side_effect = [
|
||||
# ps for delete_missing_and_updated container_names
|
||||
('''five five
|
||||
six six
|
||||
two-12345678 two
|
||||
three-12345678 three''', '', 0),
|
||||
# rm five
|
||||
('', '', 0),
|
||||
# rm six
|
||||
('', '', 0),
|
||||
# inspect two
|
||||
('{"start_order": 1, "image": "centos:6"}', '', 0),
|
||||
# rm two, changed config data
|
||||
('', '', 0),
|
||||
# inspect three
|
||||
('{"start_order": 2, "image": "centos:7"}', '', 0),
|
||||
# ps for after delete_missing_and_updated renames
|
||||
('', '', 0),
|
||||
# ps to only create containers which don't exist
|
||||
('three-12345678 three', '', 0),
|
||||
('Created one-12345678', '', 0),
|
||||
('Created two-12345678', '', 0),
|
||||
('Created four-12345678', '', 0),
|
||||
('a\nb\nc', '', 0)
|
||||
]
|
||||
r.discover_container_name = lambda n, c: '%s-12345678' % n
|
||||
r.unique_container_name = lambda n: '%s-12345678' % n
|
||||
r.execute = exe
|
||||
|
||||
builder = compose1.ComposeV1Builder('foo', config, r)
|
||||
stdout, stderr, deploy_status_code = builder.apply()
|
||||
self.assertEqual(0, deploy_status_code)
|
||||
self.assertEqual([
|
||||
'Created one-12345678',
|
||||
'Created two-12345678',
|
||||
'Created four-12345678',
|
||||
'a\nb\nc'
|
||||
], stdout)
|
||||
self.assertEqual([], stderr)
|
||||
|
||||
exe.assert_has_calls([
|
||||
# ps for delete_missing_and_updated container_names
|
||||
mock.call(
|
||||
['docker', 'ps', '-a',
|
||||
'--filter', 'label=managed_by=tester',
|
||||
'--filter', 'label=config_id=foo',
|
||||
'--format', '{{.Names}} {{.Label "container_name"}}']
|
||||
),
|
||||
# rm containers not in config
|
||||
mock.call(['docker', 'rm', '-f', 'five']),
|
||||
mock.call(['docker', 'rm', '-f', 'six']),
|
||||
# rm two, changed config
|
||||
mock.call(['docker', 'inspect', '--format',
|
||||
'{{index .Config.Labels "config_data"}}',
|
||||
'two-12345678']),
|
||||
mock.call(['docker', 'rm', '-f', 'two-12345678']),
|
||||
# check three, config hasn't changed
|
||||
mock.call(['docker', 'inspect', '--format',
|
||||
'{{index .Config.Labels "config_data"}}',
|
||||
'three-12345678']),
|
||||
# ps for after delete_missing_and_updated renames
|
||||
mock.call(
|
||||
['docker', 'ps', '-a',
|
||||
'--filter', 'label=managed_by=tester',
|
||||
'--format', '{{.Names}} {{.Label "container_name"}}']
|
||||
),
|
||||
# ps to only create containers which don't exist
|
||||
mock.call(
|
||||
['docker', 'ps', '-a',
|
||||
'--filter', 'label=managed_by=tester',
|
||||
'--filter', 'label=config_id=foo',
|
||||
'--format', '{{.Names}} {{.Label "container_name"}}']
|
||||
),
|
||||
# run one
|
||||
mock.call(
|
||||
['docker', 'run', '--name', 'one-12345678',
|
||||
'--label', 'config_id=foo',
|
||||
'--label', 'container_name=one',
|
||||
'--label', 'managed_by=tester',
|
||||
'--label', 'config_data=%s' % json.dumps(config['one']),
|
||||
'--detach=true', 'centos:7']
|
||||
),
|
||||
# run two
|
||||
mock.call(
|
||||
['docker', 'run', '--name', 'two-12345678',
|
||||
'--label', 'config_id=foo',
|
||||
'--label', 'container_name=two',
|
||||
'--label', 'managed_by=tester',
|
||||
'--label', 'config_data=%s' % json.dumps(config['two']),
|
||||
'--detach=true', 'centos:7']
|
||||
),
|
||||
# don't run three, its already running
|
||||
# run four
|
||||
mock.call(
|
||||
['docker', 'run', '--name', 'four-12345678',
|
||||
'--label', 'config_id=foo',
|
||||
'--label', 'container_name=four',
|
||||
'--label', 'managed_by=tester',
|
||||
'--label', 'config_data=%s' % json.dumps(config['four']),
|
||||
'--detach=true', 'centos:7']
|
||||
),
|
||||
# execute within four
|
||||
mock.call(
|
||||
['docker', 'exec', 'four-12345678', 'ls', '-l', '/']
|
||||
),
|
||||
|
|
|
|||
Loading…
Reference in New Issue