When we're logging inside of ansible, turning off ansi colors can
be nice. Especially inside of Zuul.
Change-Id: I719ebb106fbb31795cc02c9504507bdae607bcdd
In debugging why I631e3e3592a9be5c7da642cf34eec6d9758bf161 was not
rolling out to the zuul executors, I eventually found
/etc/facter/facts.d/ansible.json was present on a number of hosts and
effectively pinning the project-config checkout to an old reference.
Given the dates around file creation and the pinned change number, it
seems very likely something happened during the openstack->opendev
transition where some hosts had ansible run with a specific
project_config_ref set as a fact. Although the fact is now unset,
nothing will ever clear the old ansible.json which keeps applying.
We could remove all stale files by hand; but a) it's not clear where
it ran (although likely only hosts starting with z*) and b) we leave a
chance for it to happen again.
We could clear this after every puppet run; but it seems like this
role should leave the system largely in the state it ran puppet in.
Thus this clears out the file before the puppet run if no facts are
given. This should fix our immediate issue on all hosts.
Change-Id: I96adf4fb77f06b18fe73b6f10d1b07b7fe44385f
We are already organizing our hiera content basically the same way as
ansible needs for variables to be provided. If we reorganize the
directories slightly (to be coordinated with dirs on puppetmaster)
then we can have a single directory do double-duty.
Change-Id: I6ac90a7439ed8a5d9433d9526f37e44668b360ff
This patch ensures that hiera data and puppet modules, and puppet config
are copied to the right directory depending on the current puppet
version. Since it's possible for the ansible management server and the
managed nodes to have different puppet versions, we need to account for
the possibility that the source and destination paths might be
different. We also don't need to hardcode the various config paths in
config or manage environments since we're using the defaults and
hardcoding them would make them incorrect for one or the other puppet
versions.
Change-Id: I164f91f9a7942e8c5f059652634ec1078ae41aae
With newer versions of ansible, module arguments are assumed to
be strings unless otherwise specified. Our 'facts' argument is
expected to be a dictionary, so tell ansible that.
Without this, the argument will arrive as a string and be written
to the facter file inside string quotes. Facter will produce the
following error:
undefined method `each' for #<String:0x000000016ee640>
Change-Id: Iea626466d0920349f434f678fe2f71cbee259383
We keep a local copy of this file for operational purposes, but the
upstream one has grown a few things. Sync to make it easier to submit
our local changes upstream.
Change-Id: I84927750fce1a7c6101e8d5c131c0b13020bb6aa
We were copy pastaing code in the puppet cmd generation for noop and
environment handling. Collapse that to make code easier to read and
hopefully avoid future bugs where only one side is edited.
Local testing shows that puppet parses commands like:
puppet apply test.pp --noop
just fine.
Change-Id: Ie7665f72b9327b6b834d358699addf2c60a95ec0
check_mode causes other things that we may want to actually happen
(such as setting up hiera) not to run. This way we can do all of
those but still run puppet in noop.
Change-Id: I32f07a032d28f5c5d04e9eeb50b94f7c0a57ce53
The valid type names can be found in
AnsibleModule._CHECK_ARGUMENT_TYPES_DISPATCHER, and any deviation
from those causes us to no longer be able to puppet any hosts. The
type name for strings is, courtesy of Ansible's implementation being
Python, "str".
Change-Id: Ia4d4cde9926f7506a864739b1b616438c5fe6f43
puppet apply ignores the show-diff option.
Update the code so that both puppet apply and puppet agent
support the show-diff option.
Change-Id: I5b59b8ba8e50a44458d01ec97113a2ce313c1d9a
When moving to ansible 2.1.0.0, we'll need to be more explicity about
our AnsibleModule parameters. Ansible now defaults them to strings.
This will ensure groups is a list.
Change-Id: Ic126c200a432277c0ef406d4406f527d6a3c9fa7
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The --show-diff option to puppet agent is show-diff, not show_diff.
Also, there is a doc string about which version logdest was added in.
These are not important for Infra. However, keeping in sync makes it
easier to track changes that might actually matter.
Change-Id: Id16b9d7510efa4b8f3645997617cd00ee870e83a
For reasons, ansible adds to hostvars some large blobs. The size of
these blobs causes issues with puppetdb. Removing the two largest and
useless ones until the general solution is solved.
Change-Id: I5800cf1990fc4f97828fa158094e242f8f197d99
So that we can upstream this, make the log output from the puppet
command configurable with a default value being what it was for us
before. As part of that, restore returning stdout and stderr even though
they'll be empty.
Depends-On: I22b1d0e1fb635f2c626d75a11764725c8753bf24
Change-Id: I245ac8c3533cce4a598909c03e1f2ba0f7b06850
Stop sending the complete puppet output back via json. Do send
it to local syslog instead.
This is mostly because right now all of our puppet output is going
into a single very long json line in our log file.
Change-Id: Ie6d2d82def95ef2bb79ae862707218a44e0138ca
The show_diff option is superseeded by a new built-in option in ansible
so does not need to be kept in the docs here.
Also, upstream needs the simplejson fallback. We do not, but keeping in
sync is easier.
Change-Id: I45a34614d08f4518f031c652915dae97cc3ccbed
Turns out we want module.exit_json, not module_exit_json. Oops. Also,
two other small ansible task cleanups found looking at the runs.
Change-Id: I8737d5b1e675bfb89ee1db2f2c434c601d419f5e
Checking for all of the paths in ansible leads to a TON of skipped lines
in the ansible log. That's not good for anybody, and we're already
processing the paths in python - let's do a quick filter to check for
which ones exist before returning them.
Change-Id: Iddf3c56c802598329a18c374cf667a6f165f78ca
Also, post non-facter facts - we have a hybrid system, why not report
all of our facts to puppetdb.
Change-Id: I246c5fd31739921d072edf8614c748dfaa611927
The timeout command is there to improve life and workaround puppet
deficiencies. However, it's not working around deficiencies on systems
that do not have the timeout command if we blindly use it.
The puppet specific timeout options are more complex and out of scope of
this.
Issue: https://github.com/ansible/ansible-modules-extras/issues/1273
Change-Id: Id2afbe7a8d0a9b52295a36eb8bde4ffd40fa8c21
After we run puppet, we should copy the puppet logs back to the
host so that we can inject them into puppetdb.
Change-Id: I51332b02950d6fb4d4ff0c1edeca774d84df4270
Find any files that match common, $fqdn and $group in the hiera
structure and copy them to the target host's hiera structure so that
the puppet apply command can operate.
Change-Id: I833858e86b9e8fe0750642476f118ca7d0358380
The puppet module is now upstream. We'll keep a copy here until we've
installed 2.0 everywhere. But for now, just sync with what is
currently upstream.
Change-Id: I4ffe174f733f9392fb420e304c1b2a02860f9c39
When we run puppet on the node, we may want to pass in parameters. There
are two different mechanisms that can be used for this, facter and
hiera. Allow users to pass in dictionaries of data containing key/value
pairs to either set ephemerally via facter environment variables, or
permanently by creating hiera files on disk so that subsequent runs will
keep those values.
Change-Id: Id99c3c2c20764ed4ba4259bd53f8067289374403