Commit Graph

53 Commits

Author SHA1 Message Date
Monty Taylor 0505f79daa Add color parameter
When we're logging inside of ansible, turning off ansi colors can
be nice. Especially inside of Zuul.

Change-Id: I719ebb106fbb31795cc02c9504507bdae607bcdd
2020-04-21 16:04:48 -05:00
Ian Wienand e9fc2df1bc Clear out old facts
In debugging why I631e3e3592a9be5c7da642cf34eec6d9758bf161 was not
rolling out to the zuul executors, I eventually found
/etc/facter/facts.d/ansible.json was present on a number of hosts and
effectively pinning the project-config checkout to an old reference.

Given the dates around file creation and the pinned change number, it
seems very likely something happened during the openstack->opendev
transition where some hosts had ansible run with a specific
project_config_ref set as a fact.  Although the fact is now unset,
nothing will ever clear the old ansible.json which keeps applying.

We could remove all stale files by hand; but a) it's not clear where
it ran (although likely only hosts starting with z*) and b) we leave a
chance for it to happen again.

We could clear this after every puppet run; but it seems like this
role should leave the system largely in the state it ran puppet in.

Thus this clears out the file before the puppet run if no facts are
given.  This should fix our immediate issue on all hosts.

Change-Id: I96adf4fb77f06b18fe73b6f10d1b07b7fe44385f
2019-05-30 07:35:21 +10:00
Monty Taylor 24eb60b34d
Reorganize hieradata to match ansible variables
We are already organizing our hiera content basically the same way as
ansible needs for variables to be provided. If we reorganize the
directories slightly (to be coordinated with dirs on puppetmaster)
then we can have a single directory do double-duty.

Change-Id: I6ac90a7439ed8a5d9433d9526f37e44668b360ff
2018-08-02 13:11:29 -05:00
Colleen Murphy 4d64868882 Don't hardcode puppet-3-specific config paths
This patch ensures that hiera data and puppet modules, and puppet config
are copied to the right directory depending on the current puppet
version. Since it's possible for the ansible management server and the
managed nodes to have different puppet versions, we need to account for
the possibility that the source and destination paths might be
different. We also don't need to hardcode the various config paths in
config or manage environments since we're using the defaults and
hardcoding them would make them incorrect for one or the other puppet
versions.

Change-Id: I164f91f9a7942e8c5f059652634ec1078ae41aae
2018-04-22 00:13:21 +02:00
James E. Blair 8d0f0bfd0a Set facts type to dict
With newer versions of ansible, module arguments are assumed to
be strings unless otherwise specified.  Our 'facts' argument is
expected to be a dictionary, so tell ansible that.

Without this, the argument will arrive as a string and be written
to the facter file inside string quotes.  Facter will produce the
following error:

  undefined method `each' for #<String:0x000000016ee640>

Change-Id: Iea626466d0920349f434f678fe2f71cbee259383
2017-01-12 09:55:37 -08:00
Monty Taylor 1c69c305e7
Sync puppet module changes from upstream ansible
We keep a local copy of this file for operational purposes, but the
upstream one has grown a few things. Sync to make it easier to submit
our local changes upstream.

Change-Id: I84927750fce1a7c6101e8d5c131c0b13020bb6aa
2016-10-01 08:11:13 -07:00
James E. Blair e8632f41a4
Don't set detailed-exitcodes when noop
Change-Id: I3819ad5920a0a88b0b52ecb9b77a58c6d588fbdc
2016-10-01 08:11:13 -07:00
James E. Blair fccf47a713
Add debug argument
To run puppet in debug mode.

Change-Id: I00baef254db2f9eaee23fb9eacded8efc5a2afc4
2016-10-01 08:11:13 -07:00
Clark Boylan 8cc1cd9126
Simplify puppet cmd generation
We were copy pastaing code in the puppet cmd generation for noop and
environment handling. Collapse that to make code easier to read and
hopefully avoid future bugs where only one side is edited.

Local testing shows that puppet parses commands like:

  puppet apply test.pp --noop

just fine.

Change-Id: Ie7665f72b9327b6b834d358699addf2c60a95ec0
2016-10-01 08:11:12 -07:00
James E. Blair b4f591b72d
Add noop argument
check_mode causes other things that we may want to actually happen
(such as setting up hiera) not to run.  This way we can do all of
those but still run puppet in noop.

Change-Id: I32f07a032d28f5c5d04e9eeb50b94f7c0a57ce53
2016-10-01 08:11:12 -07:00
James E. Blair 1a20bc89eb
Allow any logdest
Puppet's logdest option can also point to a file, so allow any
string through here.

Change-Id: I36fc99fa7e6169463acd24a5a6f4b54cf23849b7
2016-10-01 08:11:12 -07:00
Jenkins ede55a8691 Merge "Honor show-diff options with puppet apply" 2016-08-05 20:03:36 +00:00
Jeremy Stanley 10e657d56d Correct AnsibleModule string type names to str
The valid type names can be found in
AnsibleModule._CHECK_ARGUMENT_TYPES_DISPATCHER, and any deviation
from those causes us to no longer be able to puppet any hosts. The
type name for strings is, courtesy of Ansible's implementation being
Python, "str".

Change-Id: Ia4d4cde9926f7506a864739b1b616438c5fe6f43
2016-07-23 22:53:06 +00:00
Ramy Asselin 92737dfe27 Honor show-diff options with puppet apply
puppet apply ignores the show-diff option.
Update the code so that both puppet apply and puppet agent
support the show-diff option.

Change-Id: I5b59b8ba8e50a44458d01ec97113a2ce313c1d9a
2016-07-19 09:18:39 -07:00
Paul Belanger b8e3bdd1fd
Be explicit with AnsibleModule types
When moving to ansible 2.1.0.0, we'll need to be more explicity about
our AnsibleModule parameters. Ansible now defaults them to strings.

This will ensure groups is a list.

Change-Id: Ic126c200a432277c0ef406d4406f527d6a3c9fa7
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-07-18 14:35:12 -04:00
Jenkins b5fc21ca37 Merge "Call main() from a conditional" 2016-06-06 13:02:06 +00:00
Jenkins 7f32a9eeaf Merge "Remove dead code" 2016-06-03 15:08:59 +00:00
Paul Belanger b6ebba9545
Call main() from a conditional
See ansible docs[1] for more information.

[1] http://docs.ansible.com/ansible/developing_modules.html

Change-Id: I21062e1eb47632ac39e35d7267dab96b62500743
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-05-26 13:31:11 -04:00
Jenkins 5af0d99453 Merge "The default for logdest needs to not be a list" 2016-02-24 19:37:51 +00:00
Monty Taylor fdc43c9f8a The default for logdest needs to not be a list
There are three possible values - none of them are a list.

Change-Id: Iade056998eeccb1845aa6232ea803073567add2f
2016-02-24 12:57:39 -06:00
Monty Taylor 42c28a7456 Stop posting non-facter facts
No idea if these are breaking things, but maybe they are.

Change-Id: Id9492d901439407a9ffe9976be1fc26afc035b45
2016-02-24 10:42:37 -06:00
Monty Taylor 36916f8622 Return useful error messages on requests exception
We need to be able to debug this.

Change-Id: Ie20bc1293d80bbbd45479c12dd65314b27c61e40
2016-02-24 10:42:33 -06:00
Monty Taylor c64119b530 Sync with upstream ansible-modules-extras
The --show-diff option to puppet agent is show-diff, not show_diff.
Also, there is a doc string about which version logdest was added in.

These are not important for Infra. However, keeping in sync makes it
easier to track changes that might actually matter.

Change-Id: Id16b9d7510efa4b8f3645997617cd00ee870e83a
2016-02-24 08:57:45 -06:00
Jenkins 54064729df Merge "Make puppet logout destination configurable" 2016-02-24 14:53:11 +00:00
Spencer Krum 952ffbe9be Remove some problematic facts
For reasons, ansible adds to hostvars some large blobs. The size of
these blobs causes issues with puppetdb. Removing the two largest and
useless ones until the general solution is solved.

Change-Id: I5800cf1990fc4f97828fa158094e242f8f197d99
2016-02-23 16:05:43 -08:00
Spencer Krum ffeeba25a4 Variables enter ansible modules as p['thing']
Change-Id: Ifc61e8d37d296a2c9f334ce2bc8ef18fb1d67874
2016-02-23 13:49:44 -08:00
Spencer Krum 1f32232b7f Allow post_puppetdb to fail to post facts
This prevents failure of the module when puppetdb has a belly ache.

Change-Id: Ief2dbb740ed358d977c3656feda1f3932d5d5e08
2016-02-23 13:22:56 -08:00
Spencer Krum 0909fa99f7 Store the facts payload
This enable us to look at the object being sent over and debug why
puppetdb is barfing on it.

Change-Id: Ie2edf5bedc34bb6541f070d2f6d83af746fdf6bd
2016-02-23 13:09:35 -08:00
Spencer Krum 652c192d7f Remove dead code
This variable assignment is wiped out by the next statement, remove it.

Change-Id: Ibc042d6a98170646f03fc00a8ffd16d9f9cce9f4
2016-02-10 19:27:05 -08:00
Monty Taylor d028a88572 Make puppet logout destination configurable
So that we can upstream this, make the log output from the puppet
command configurable with a default value being what it was for us
before. As part of that, restore returning stdout and stderr even though
they'll be empty.

Depends-On: I22b1d0e1fb635f2c626d75a11764725c8753bf24
Change-Id: I245ac8c3533cce4a598909c03e1f2ba0f7b06850
2016-02-09 10:36:16 -06:00
James E. Blair 403d157c45 Fix typo in puppet apply command
This was missing a terminal space which caused the command to be:

  --logdest syslog--no-noop

Change-Id: I13827a5ed30f782bdacc38acebf284d7226ebce9
2016-01-22 16:56:09 -08:00
James E. Blair 7fff1b962c Log puppet to syslog instead of json
Stop sending the complete puppet output back via json.  Do send
it to local syslog instead.

This is mostly because right now all of our puppet output is going
into a single very long json line in our log file.

Change-Id: Ie6d2d82def95ef2bb79ae862707218a44e0138ca
2016-01-21 18:07:37 -05:00
Monty Taylor e1c1430bd6 Sync with upstream ansible
The show_diff option is superseeded by a new built-in option in ansible
so does not need to be kept in the docs here.

Also, upstream needs the simplejson fallback. We do not, but keeping in
sync is easier.

Change-Id: I45a34614d08f4518f031c652915dae97cc3ccbed
2016-01-21 18:06:29 -05:00
Monty Taylor 69cf3cd23c module is a variable, not a name prefix
Turns out we want module.exit_json, not module_exit_json. Oops. Also,
two other small ansible task cleanups found looking at the runs.

Change-Id: I8737d5b1e675bfb89ee1db2f2c434c601d419f5e
2016-01-18 19:14:36 -05:00
Monty Taylor c25e5aa543 Perform filepath checking in python
Checking for all of the paths in ansible leads to a TON of skipped lines
in the ansible log. That's not good for anybody, and we're already
processing the paths in python - let's do a quick filter to check for
which ones exist before returning them.

Change-Id: Iddf3c56c802598329a18c374cf667a6f165f78ca
2016-01-14 15:39:25 -05:00
Monty Taylor f8c70cb1f2 Don't post facts if there are none
Also, post non-facter facts - we have a hybrid system, why not report
all of our facts to puppetdb.

Change-Id: I246c5fd31739921d072edf8614c748dfaa611927
2015-11-27 17:57:24 +00:00
Monty Taylor d049384a82 Don't use the timeout command if it's not there
The timeout command is there to improve life and workaround puppet
deficiencies. However, it's not working around deficiencies on systems
that do not have the timeout command if we blindly use it.

The puppet specific timeout options are more complex and out of scope of
this.

Issue: https://github.com/ansible/ansible-modules-extras/issues/1273
Change-Id: Id2afbe7a8d0a9b52295a36eb8bde4ffd40fa8c21
2015-11-27 17:57:17 +00:00
Monty Taylor ad8aab1470 The parameter is --server not -- server
We never noticed this because we have puppetmaster set in our
puppet.conf files.

Change-Id: Ic05896fecc08cc4143f878dd87189f0397252ebb
2015-11-25 08:40:18 -05:00
Monty Taylor b134ce47a8 Return stderr/stdout on changes too
If changes were made, you sometimes want info on what they were.

Change-Id: Ibdb662c0d66e73028211374ea58f106876d74b59
2015-11-25 08:40:18 -05:00
Monty Taylor 92c56e9e99 Fetch puppet logs and facts after puppet run
After we run puppet, we should copy the puppet logs back to the
host so that we can inject them into puppetdb.

Change-Id: I51332b02950d6fb4d4ff0c1edeca774d84df4270
2015-11-25 08:40:18 -05:00
Jenkins d48398a75a Merge "Copy in matching hiera files" 2015-11-23 09:37:24 +00:00
Monty Taylor 898ddbc0e3 Add space back
Change-Id: Ibf0d6f715e06a0f6d519a7be8a3166584bbc7217
2015-11-16 16:12:27 -05:00
Spencer Krum 854b97e9ea Fix pep8 warnings
Change-Id: I9f199dfa45885e51a925232fac54dabe7c844ed1
2015-11-12 22:43:52 -08:00
Monty Taylor c424d8b555 Copy in matching hiera files
Find any files that match common, $fqdn and $group in the hiera
structure and copy them to the target host's hiera structure so that
the puppet apply command can operate.

Change-Id: I833858e86b9e8fe0750642476f118ca7d0358380
2015-11-11 14:59:35 -08:00
Joshua Hesketh 258d42aec6 Fix typos in documentation
Change-Id: If6a84b0f225a69183d0806fa55cea267475f7dc9
2015-10-28 11:47:04 +09:00
Monty Taylor 5259c2d672 Sync with ansible-modules-extras
The puppet module is now upstream. We'll keep a copy here until we've
installed 2.0 everywhere. But for now, just sync with what is
currently upstream.

Change-Id: I4ffe174f733f9392fb420e304c1b2a02860f9c39
2015-06-20 09:50:42 -04:00
Monty Taylor 5faac02239 Ensure filepath exists
Before we can write facter facts of hiera hieras, we need for the
directory to exist.

Change-Id: Id855da44d9e2cb06824ed3507158aeaa466768f7
2015-03-12 10:08:44 -04:00
Clark Boylan 226adc7f23 Its 0o not 0c to make an octal number
Change-Id: Ifa2fe70bbb170d2a9cffad5a5280dfa74b59230a
2015-03-11 14:22:34 -07:00
Monty Taylor eb5c691571 Add ability to run puppet with passed-in data
When we run puppet on the node, we may want to pass in parameters. There
are two different mechanisms that can be used for this, facter and
hiera. Allow users to pass in dictionaries of data containing key/value
pairs to either set ephemerally via facter environment variables, or
permanently by creating hiera files on disk so that subsequent runs will
keep those values.

Change-Id: Id99c3c2c20764ed4ba4259bd53f8067289374403
2015-03-11 11:13:01 -04:00
Monty Taylor 50173449a6 Add all of the files needed to make this a role
Similar to puppet modules, ansible roles can be uploaded to ansible
galaxy and can be consumed in a role-per-repo manner.
2015-02-22 22:59:39 -05:00