With commit f7a50a24b9 (Change ID:
I99541d36aed6d4b9f83746c1cd1a5521b310f1f6,
https://review.openstack.org/#/c/369485/) the inventory configuration
was split into multiple files to enable scenario AIO testing. This had
the side effect of no longer populating inventory with a full config
file - the tests/inventory/openstack_user_config.yml file was a symlink
to etc/openstack_deploy/openstack_user_config.yml.aio, which only had a
handful of groups.
The result was that only 7 host entries appeared in inventory: aio1,
galera, rabbitmq, memcache, log, repo, and haproxy containers. These
coincide with the groups defined only in the
openstack_user_config.yml.aio file.
To make the inventory tests more robust against changes like these, this
patch adds some module-level fixtures that generate the test
configuration file from all the files ending with '.aio'. This file is
dynamically constructed at the beginning of each test run, and deleted
afterwards. The symlink to the openstack_user_config.yml.aio file is
removed in favor of this dynamically created file, though a placeholder
file remains to keep the directory in git.
A test was added to ensure any configured groups have hosts added; this
should serve as a safeguard against similar reconstructions. No check is
made for groups defined by the environment but unpopulated by config,
such as cinder_volumes_containers.
The original file could have been kept as a placeholder, but it would be
marked as changed by git after every test run.
The management tests were also updated to make use of the dynamic
configuration creation.
Change-Id: Ie4ba9c50315736a0b86e0caa2cccb0908c452a49
In past versions, upgrading OpenStack-Ansible lead to contention between
the code base and deployer customizations when env.d files were changed.
Deployers were encouraged to make their customizations, while the
project needed to sometimes adjust architecture. Detecting these
conflicts in an automated way was difficult, since the files were simple
dictionaries and lists, leaving no metadata to describe intent for the
changes.
This change modifies the dynamic inventory system to first use the
in-tree env.d directory as the base environment, then reads in files
from the /etc/openstack_deploy/env.d directory and updates existing keys
with the new values. In this way, the OSA project can modify the
environment and deployers can customize the environment without directly
manipulating the same files.
As part of this change, the env.d directory was moved in to the
playbooks/inventory directory, in order to reduce the path manipulation
done inside of the dynamic_inventory.py script. The example files were
left in the etc/openstack_deploy directory for reference.
Note that this change supports deleting elements by specifying a
empty value, such as an empty list or an empty dictionary.
When overriding, only the path to the values that changed is necessary.
For example, changing the 'is_metal' property for cinder only needs the
following in /etc/openstack_deploy/env.d/cinder.yml:
container_skel:
cinder_volumes_container:
properties:
is_metal: false
This is instead of the entirity of the container_skel dict or even the
other top-level dicts.
For AIO/gate scenarioes, the env.d copy logic has been removed, as it is
now redundant.
Change-Id: Ic637fa385fd3fec7365fb9bc5e0ff54a7f4c8bee
Now that Ie8bedca2ca047ebddac32189e85cb201601d068c
has merged, this symlink points at a non-existent
file and provides no value to the inventory tests
Change-Id: I139a32162d6c5a252bad697e49b8708acaaf0989
This change introduces a testing framework for the dynamic_inventory.py
script, which will be helpful in debugging any changes made to the
script. Current testing is indirect, relying on the other, very high
level gate tests, which may take longer to identify inventory generation
issues.
Currently, it does not do actual unit testing, instead opting to start
by testing from 'outside', calling the script through a subprocess. In
time, this should be replaced, but as a start it's easier to implement
without making many changes.
The 'tests/inventory' directory has the necessary pieces symlinked in;
namely, env.d, openstack_environment.yml, and a sample
openstack_user-config.yml (based on the AIO one already used in the gates).
A new tox environment, 'inventory' was added rather than creating a new
'gate-check-inventory.sh' file. This should fit the general
openstack-infra testing framework better.
For the first test, I decided to add a loop that runs 100 times because
of the nature of the script. Since the dynamic_inventory script does not
assign IP addresses sequentially, the loop helps exercise a larger
sample than just one run per test. On my local machine, this took
approximately 2 seconds. Running it 1,000 times took about 30 seconds.
Thus, the time spent right now is minimal in comparison to the rest of
the gate jobs.
As it stands, the test suite is by no means complete, but serves as a
starting point.
Change-Id: I7410360ecec919639a299f72f2f3cc818bce8e33
Nolan Brubaker