Commit Graph

312 Commits

Author SHA1 Message Date
Harry Rybacki 265146ed2e Become 1.3.0
Change-Id: Ia5d9f82219ceffd50e5d1b3f3be3c216a63e2e74
2019-12-17 09:43:36 -05:00
Harry Rybacki a71617627a Add locks to cache and cleanup kinit logic
After reviewing reports of multiple CCache cropping up in logs, we
found an issue in the way novajoin is initiating and updating
cache files containing keytabs. The result was numerous extra cache
files being created and overwritten.

With this change we ensure that the credentials cache is properly
shared across workers and that when new credentials are being
created, the cache files are locked to avoid potential conflicts.

Updates DEBUG level logging to include useful cache troubleshooting
breadcrumbs.

Change-Id: I07e0004f77e0d52ab2a2707c5fe50f48f718b717
Co-Authored-By: Ade Lee <alee@redhat.com>
2019-12-09 13:21:15 -05:00
Ade Lee e418762753 Conditionally import nss
python-nss does not exist (and is not needed) in RHEL8.
We need to conditionally import nss to avoid errors in RHEL8.

Change-Id: I699fbfab4c2106f24260c99905b1bd40a8e683a8
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1758771
2019-12-04 16:35:04 +00:00
Grzegorz Grasza 4aa08a25b8 Remove functional tests gate for centos7 and workarounds for python2
CentOS 7 job is running with python2, which is no longer supported by
nova. Devstack also dropped it's platform tests for CentOS.

Change-Id: I74b04cf6ecd978bfc5fbf3b2c7484a79d28ae826
2019-12-04 15:46:47 +01:00
Harry Rybacki c9299d5c37 Add configurable delay to connection rety attempts with IPA
Presently, when novajoin fails to make a connection with the IPA
server, for any reason, it will immediately re-attempt to make
the connection when the backoff is unset (it is off by default).
As a result, any timing related issues could be the source of
the connection issues will likely result in no connection at all.

This change adds a new configuration option, retry_delay, which
will halt subsequent connection attempts for N seconds where N
is the retry_delay. By default this is set to 5 seconds, mirroring
internal ipalib behavior[1].

[1] - https://github.com/freeipa/freeipa/blob/master/ipalib/install/kinit.py#L29-L30

Change-Id: Iec96e4bd6643c0a657c8db424cc72deb10f170bd
2019-10-21 10:21:51 -04:00
Harry Rybacki b971c7836f Become 1.2.0
Change-Id: I5b26e62b0de1204284b30a0dc972e1dd6534f608
2019-09-12 13:16:39 -04:00
Harry Rybacki 4f1353bb1f Handle add_host and delete_host cases more robustly
Presently novajoin has no way of differentiating between hosts and
hostnames. As a result, it is possible for a host to be inadvertantly
deleted in certain conditions.

This fix aims to resolve this and other join/delete edge cases  by
passing the instance-id (server uuid) from nova along in the
description field that is passed to IdM. We can use this
description and id to ensure we delete only the hosts we meant to.

Overview of changes:
- Persist nova instance-id in IdM's Description field
- Update join logic to handle hosts with old Description field
- Update join logic to cause nova deploy failure when attempting to
  add a host with a hostname that is already enrolled
- Add new DuplicateInstanceError exception type
- Add new DeleteInstanceIdMismatch exception type
- Add inline comments documenting code flow
- IPAClient add_host doc strings for clarity

Change-Id: I676bac162a6ec35366c506bdb660cf3913131afd
2019-09-12 09:26:39 -04:00
Zuul 6be279e35b Merge "Update Fault class to pass error key up" 2019-09-12 06:42:31 +00:00
Zuul 0bcb4ec41b Merge "Fix error message when OTP is missing, add logging" 2019-09-11 20:32:33 +00:00
Zuul 366480da1a Merge "Additional logging in functional tests" 2019-09-11 20:32:32 +00:00
Zuul 694faba25a Merge "Sync cloud-init script with tripleo ipaclient*.yaml" 2019-09-11 20:28:47 +00:00
Harry Rybacki 657e75c5e1 Update Fault class to pass error key up
While debugging nova-compute logs it was noted that error messages
were not being populated rather the fault_name was. Updating the
response we hand back to Nova to contain message within the
'error' key of the returned object.

Change-Id: I2e0f415a512e53261b1e366cd75b310dd06eec27
2019-09-11 14:12:02 -04:00
Zuul f32dd28f2e Merge "Fix IPAConnectionError string formatting" 2019-09-09 17:28:45 +00:00
Harry Rybacki 051f8ef9f6 Fix IPAConnectionError string formatting
Exception message had incorrect string format that would result
in a TypeError being raise if/when this exception was caught.

Change-Id: I676631d79394e512371a8367f84b91761e983faa
2019-09-08 11:45:43 -04:00
Harry Rybacki 5b6b8607ec Log statement missing string replacement arg
Include missing arg

Change-Id: Ic494e58fc3b1f74f574e0dd8255ddc36ad2249c9
2019-09-05 15:46:40 -04:00
Grzegorz Grasza 956cc87cc1 Fix error message when OTP is missing, add logging
* Fix cloud-init error message when OTP is missing
* Add a log message in novajoin-server

Change-Id: Ib299269c564744af6a5fcded9195d27be1c14ce7
Related-Bug: 1836529
2019-08-30 14:57:13 +00:00
Grzegorz Grasza 610e03d33b Additional logging in functional tests
Print setup-ipa-client.log in case the enrollment test fails.

Change-Id: I8980da9da9e9edfc95c58345bff7bde722677f81
2019-08-30 14:57:03 +00:00
Grzegorz Grasza 938a999a02 Sync cloud-init script with tripleo ipaclient*.yaml
* Always use the FQDN supplied in the metadata.
    * Read the metadata from network if hostname could not be determined.

Related: rhbz#1734156
Change-Id: I0aaa32d71569240e5b8ced270f9296782e2e08b8
2019-08-30 14:56:54 +00:00
Zuul 323efaad22 Merge "Helper scripts for modifying cloud-config-novajoin.json" 2019-08-30 14:31:48 +00:00
Grzegorz Grasza 6506855623 Helper scripts for modifying cloud-config-novajoin.json
Adding two helper scripts to unpack cloud-config-novajoin.json to a more
readable form. After editing cloud-config-novajoin.yaml, it still has to
be packed into cloud-config-novajoin.json with tools/cloud-config-pack.py.

Related-Bug: 1836529
Change-Id: I3d6a3e5f97608f7f8a0070f40ddeb8a7113c33b3
2019-08-28 12:23:59 +02:00
Ade Lee ade787b90c Add debug messages
We are having a hard time keeping track of which operations
correspond to which request.  This patch adds the ability to track
operations in the notifier with the message_id of the notification
being processed.  This message_id (which is generated by oslo is
a uuid

For the server, we could also set the message_id to the request_id
of the python-requests object received, but this is already
logged as part of the server logs.

Change-Id: Ie8b885a2b5cba6684e92c49eed4a99d24621402e
2019-08-22 07:54:17 +00:00
Ade Lee 9750c363f6 Refactor method names
Debugging is confusing when the same names are used for methods
in two different controllers.  Fixing this to more accurately
reflect whats going on.

Change-Id: I3740cd3ae81776cb1ecf066e617e615d880dc2e8
2019-08-20 11:02:18 -04:00
Ade Lee 6ed30c9476 Fix backoff mechanism
Right now, the backoff mechanism is broken when the backoff is
set to something non-zero.  Basically, you go into this state where
you retry ad infinitum, leading to inconsistent behavior.

This change fixes the mechanism so that you only get a fixed number
of retries.  You can choose (through a new config parameter) to allow
backoff (or not).

To restore some of the old behavior, the default for the connect_retries
parameter has been increased from 2 to 4, and the max backoff time has
been decreased from 1024 to 512 seconds.  Its unlikely that we'd ever
reach that backoff time without a large number of retries, but 1024
seems too long.

And there is a new exception that is thrown when the connection
fails.  This will result in nice 500 errors in the novajoin-server,
and some log messages for the notifier.

Change-Id: I10547fbde8966c8694346ed8c054e627bee2ee51
2019-08-20 11:02:07 -04:00
Grzegorz Grasza 2c8c868a2c Add mkhomedir option to ipa-client-install
To allow PAM to create a home directory for the user who does not have one,
ipa-client-install needs this option.

Change-Id: I550f48e5a6cb808f0bc68ee1bdbddba9660531c2
Related-bug: #1823727
2019-05-29 15:17:12 +02:00
Zuul 9bfcc96619 Merge "Always use fqdn" 2019-05-23 16:01:11 +00:00
OpenDev Sysadmins ea0c404c74 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:50:10 +00:00
Ade Lee 5b897ce8de Always use fqdn
There is no reason not to use the fqdn, as we always check for
it.  It is needed in some cases eg. when deploying when the domain
does not match the realm.

Change-Id: Iefefc8d134af64c12d206c75b69ead8d9e94ac39
2019-04-11 11:44:42 -04:00
Michele Baldessari e8b18c4bd4 Fix python3 errors in novajoin-ipa-setup
With python3 we are getting the following error:
Traceback (most recent call last):
  File "/usr/bin/novajoin-ipa-setup", line 103, in <module>
    args['tls_ca_cert'] = cafile.decode('UTF-8')
  AttributeError: 'str' object has no attribute 'decode'

Let's just use the normal assignment in case the .decode() call
fails.

Also apply the same fixes to files/cloud-config-novajoin.json

Tested and we correctly get past this error.

Change-Id: Ia77ebde46ff885c9f929d834f981acb97f26a4ec
Co-Authored-By: Luca Miccini <lmiccini@redhat.com>
Closes-Bug: #1820961
2019-03-21 08:58:42 +01:00
Grzegorz Grasza 74fd4c4357 Become 1.1.1
Change-Id: Ia5483264808812e8b7550aa44240d05c9b7337de
2019-02-07 19:33:24 +01:00
Zuul b0d81f8c86 Merge "Remove unneeded call to nova API, defer other API calls" 2019-01-31 19:17:44 +00:00
Grzegorz Grasza fe512714e2 Fix for ipalib 4.7.2 (Fedora 29)
This fixes u"unknown command 'b'xxx''" errors.

Change-Id: I155bd37e7007fce4e083f8e5f7c4a3511a44ae4a
2019-01-29 15:12:56 +01:00
Grzegorz Grasza db868ea7c1 Remove unneeded call to nova API, defer other API calls
We get an instance ID directly from nova, which calls our API,
consequently we don't need to call back to nova to double check
if the instance ID realy exists.

Additionally, defer calling keystone and glance APIs to the moment
that the retrieved objects are realy needed.

Change-Id: I64a20c88229490690798aaf75ca0d96d98032467
2019-01-29 15:09:16 +01:00
Juan Antonio Osorio Robles 5633d348e3 Change default policy to check service project and not role
In TripleO and devstack alike, service users are part of the "service"
project; while TripleO doesn't have a "service" role. So lets depend on
the project to enforce policy. This way this will still work out of the
box with TripleO.

Change-Id: I01cf7b38904bb0311658348dcdc0b0efd4f36c0e
Closes-Bug: #1812844
2019-01-22 16:31:59 +02:00
Grzegorz Grasza 462305315c Check policy when handling a HTTP request
* Add default policy for handling the create request.
* Allow it to be accessed only by nova service.
* Remove unused code copied from cinder.

Change-Id: Ieaa407f27c6774d1fd17850a9571de5554360bae
2019-01-18 11:27:09 +01:00
Grzegorz Grasza b3f961e331 Become 1.1.0
Change-Id: I38e1cb161dabec02de72a7d22cb1ace7cda6f142
2019-01-08 13:54:33 +01:00
Grzegorz Grasza ff9ed8461e Fix StringIO import and patch paramiko in Python 3
This fixes ModuleNotFoundError: No module named 'StringIO',
raised in Python 3 functional tests. We also patch paramiko
on Python 3, since we use it in functional tests.

Change-Id: I357dd9c3ec7c0a76d31b7f94ec0e844d9bdcb5c5
2018-12-20 15:55:51 +01:00
Grzegorz Grasza e34047457c Install novajoin-ipa-setup script in /usr/libexec
Since novajoin is branchless, we need to support
older releases, which may call this script from the
old location. Additional novajoin-ipa-setup script is
installed in the old location for backward compatibility
and calls the new "compiled" script in the default bin
directory.

Change-Id: I0a25ffb1f5fd3f8723bff8a1bcfc6dfc486b2b4a
2018-12-19 21:05:51 +00:00
Grzegorz Grasza 22a90c2508 Workaround https://bugzilla.redhat.com/show_bug.cgi?id=1624606
This is a workaround for a nss-3.40/certmonger/389-ds/p11-proxy
issue in Fedora.

Change-Id: I3ff7324d4878b64c2e2af75b3a465710a33655b5
2018-12-19 21:05:41 +00:00
Zuul 59b91ceebf Merge "Test compact_services metadata" 2018-12-19 20:50:58 +00:00
Zuul b2a94e550b Merge "Bump development status and add Python 3 support" 2018-12-19 19:46:01 +00:00
Zuul 617541b67a Merge "Run unit tests on Python 3.6" 2018-12-19 19:46:00 +00:00
Zuul 43cf3a0b43 Merge "Reconnect on httplib.ResponseNotReady" 2018-12-19 19:45:59 +00:00
Zuul e8467a2a94 Merge "Run functional tests on Python 3" 2018-12-19 19:45:59 +00:00
Zuul 6a1f2e1a87 Merge "Fix errors preventing novajoin to start on Python 3" 2018-12-19 19:45:58 +00:00
Grzegorz Grasza f5aab5544d Test compact_services metadata
This tests new and old formats, as well as instance metadata updates.

Change-Id: Ie7b3bcdbb98bb2786000207b72e7b289d5051b8f
2018-12-19 20:39:53 +01:00
Grzegorz Grasza 62475be1c2 Enhance the basic enrollment test
This additionaly logs into the server instance and
checks the IPA client configuration.

Change-Id: Ic249032e9241f13c843119f798efbe598a62c88e
2018-12-19 16:27:31 +01:00
Grzegorz Grasza 8aaf9007bb Fix ImportError in Centos CI
This fixes an import error with the newest devstack, which conflicts
with the system FreeIPA installation and results in:
ImportError: cannot import name decorate in dogpile.

Change-Id: I8bade87962f3adacbc26a666ea02fedb74963bfa
2018-12-13 17:20:49 +01:00
Ade Lee c75c4c05a1 Become 1.0.22
Change-Id: I2a44e1e52317c1bf05aa76e07c7bd5283521e02e
2018-12-13 00:05:49 -05:00
Douglas Mendizábal 2c0091d23d Handle compact services on multiple lines
This patch adds logic to handle compact service metadata that
has been split into multiple lines to avoid hitting the metadata
size limit.

Co-Authored-By: Grzegorz Grasza <xek@redhat.com>
Change-Id: Ida39f5768c67f982b2fe316f6fae4988a74c8534
2018-12-12 20:56:16 +00:00
Grzegorz Grasza 46529a445d Bump development status and add Python 3 support
We have the functional tests going in the CI and novajoin is part
of TripleO and RDO, already being considered to run on many large
production systems, so I think the Beta designation is appropriate.

Change-Id: I7bb279160bb1bc25512898251876293699c5063d
2018-12-03 16:38:53 +01:00