After reviewing reports of multiple CCache cropping up in logs, we
found an issue in the way novajoin is initiating and updating
cache files containing keytabs. The result was numerous extra cache
files being created and overwritten.
With this change we ensure that the credentials cache is properly
shared across workers and that when new credentials are being
created, the cache files are locked to avoid potential conflicts.
Updates DEBUG level logging to include useful cache troubleshooting
breadcrumbs.
Change-Id: I07e0004f77e0d52ab2a2707c5fe50f48f718b717
Co-Authored-By: Ade Lee <alee@redhat.com>
python-nss does not exist (and is not needed) in RHEL8.
We need to conditionally import nss to avoid errors in RHEL8.
Change-Id: I699fbfab4c2106f24260c99905b1bd40a8e683a8
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1758771
CentOS 7 job is running with python2, which is no longer supported by
nova. Devstack also dropped it's platform tests for CentOS.
Change-Id: I74b04cf6ecd978bfc5fbf3b2c7484a79d28ae826
Presently, when novajoin fails to make a connection with the IPA
server, for any reason, it will immediately re-attempt to make
the connection when the backoff is unset (it is off by default).
As a result, any timing related issues could be the source of
the connection issues will likely result in no connection at all.
This change adds a new configuration option, retry_delay, which
will halt subsequent connection attempts for N seconds where N
is the retry_delay. By default this is set to 5 seconds, mirroring
internal ipalib behavior[1].
[1] - https://github.com/freeipa/freeipa/blob/master/ipalib/install/kinit.py#L29-L30
Change-Id: Iec96e4bd6643c0a657c8db424cc72deb10f170bd
Presently novajoin has no way of differentiating between hosts and
hostnames. As a result, it is possible for a host to be inadvertantly
deleted in certain conditions.
This fix aims to resolve this and other join/delete edge cases by
passing the instance-id (server uuid) from nova along in the
description field that is passed to IdM. We can use this
description and id to ensure we delete only the hosts we meant to.
Overview of changes:
- Persist nova instance-id in IdM's Description field
- Update join logic to handle hosts with old Description field
- Update join logic to cause nova deploy failure when attempting to
add a host with a hostname that is already enrolled
- Add new DuplicateInstanceError exception type
- Add new DeleteInstanceIdMismatch exception type
- Add inline comments documenting code flow
- IPAClient add_host doc strings for clarity
Change-Id: I676bac162a6ec35366c506bdb660cf3913131afd
While debugging nova-compute logs it was noted that error messages
were not being populated rather the fault_name was. Updating the
response we hand back to Nova to contain message within the
'error' key of the returned object.
Change-Id: I2e0f415a512e53261b1e366cd75b310dd06eec27
Exception message had incorrect string format that would result
in a TypeError being raise if/when this exception was caught.
Change-Id: I676631d79394e512371a8367f84b91761e983faa
* Fix cloud-init error message when OTP is missing
* Add a log message in novajoin-server
Change-Id: Ib299269c564744af6a5fcded9195d27be1c14ce7
Related-Bug: 1836529
* Always use the FQDN supplied in the metadata.
* Read the metadata from network if hostname could not be determined.
Related: rhbz#1734156
Change-Id: I0aaa32d71569240e5b8ced270f9296782e2e08b8
Adding two helper scripts to unpack cloud-config-novajoin.json to a more
readable form. After editing cloud-config-novajoin.yaml, it still has to
be packed into cloud-config-novajoin.json with tools/cloud-config-pack.py.
Related-Bug: 1836529
Change-Id: I3d6a3e5f97608f7f8a0070f40ddeb8a7113c33b3
We are having a hard time keeping track of which operations
correspond to which request. This patch adds the ability to track
operations in the notifier with the message_id of the notification
being processed. This message_id (which is generated by oslo is
a uuid
For the server, we could also set the message_id to the request_id
of the python-requests object received, but this is already
logged as part of the server logs.
Change-Id: Ie8b885a2b5cba6684e92c49eed4a99d24621402e
Debugging is confusing when the same names are used for methods
in two different controllers. Fixing this to more accurately
reflect whats going on.
Change-Id: I3740cd3ae81776cb1ecf066e617e615d880dc2e8
Right now, the backoff mechanism is broken when the backoff is
set to something non-zero. Basically, you go into this state where
you retry ad infinitum, leading to inconsistent behavior.
This change fixes the mechanism so that you only get a fixed number
of retries. You can choose (through a new config parameter) to allow
backoff (or not).
To restore some of the old behavior, the default for the connect_retries
parameter has been increased from 2 to 4, and the max backoff time has
been decreased from 1024 to 512 seconds. Its unlikely that we'd ever
reach that backoff time without a large number of retries, but 1024
seems too long.
And there is a new exception that is thrown when the connection
fails. This will result in nice 500 errors in the novajoin-server,
and some log messages for the notifier.
Change-Id: I10547fbde8966c8694346ed8c054e627bee2ee51
To allow PAM to create a home directory for the user who does not have one,
ipa-client-install needs this option.
Change-Id: I550f48e5a6cb808f0bc68ee1bdbddba9660531c2
Related-bug: #1823727
There is no reason not to use the fqdn, as we always check for
it. It is needed in some cases eg. when deploying when the domain
does not match the realm.
Change-Id: Iefefc8d134af64c12d206c75b69ead8d9e94ac39
With python3 we are getting the following error:
Traceback (most recent call last):
File "/usr/bin/novajoin-ipa-setup", line 103, in <module>
args['tls_ca_cert'] = cafile.decode('UTF-8')
AttributeError: 'str' object has no attribute 'decode'
Let's just use the normal assignment in case the .decode() call
fails.
Also apply the same fixes to files/cloud-config-novajoin.json
Tested and we correctly get past this error.
Change-Id: Ia77ebde46ff885c9f929d834f981acb97f26a4ec
Co-Authored-By: Luca Miccini <lmiccini@redhat.com>
Closes-Bug: #1820961
We get an instance ID directly from nova, which calls our API,
consequently we don't need to call back to nova to double check
if the instance ID realy exists.
Additionally, defer calling keystone and glance APIs to the moment
that the retrieved objects are realy needed.
Change-Id: I64a20c88229490690798aaf75ca0d96d98032467
In TripleO and devstack alike, service users are part of the "service"
project; while TripleO doesn't have a "service" role. So lets depend on
the project to enforce policy. This way this will still work out of the
box with TripleO.
Change-Id: I01cf7b38904bb0311658348dcdc0b0efd4f36c0e
Closes-Bug: #1812844
* Add default policy for handling the create request.
* Allow it to be accessed only by nova service.
* Remove unused code copied from cinder.
Change-Id: Ieaa407f27c6774d1fd17850a9571de5554360bae
This fixes ModuleNotFoundError: No module named 'StringIO',
raised in Python 3 functional tests. We also patch paramiko
on Python 3, since we use it in functional tests.
Change-Id: I357dd9c3ec7c0a76d31b7f94ec0e844d9bdcb5c5
Since novajoin is branchless, we need to support
older releases, which may call this script from the
old location. Additional novajoin-ipa-setup script is
installed in the old location for backward compatibility
and calls the new "compiled" script in the default bin
directory.
Change-Id: I0a25ffb1f5fd3f8723bff8a1bcfc6dfc486b2b4a
This fixes an import error with the newest devstack, which conflicts
with the system FreeIPA installation and results in:
ImportError: cannot import name decorate in dogpile.
Change-Id: I8bade87962f3adacbc26a666ea02fedb74963bfa
This patch adds logic to handle compact service metadata that
has been split into multiple lines to avoid hitting the metadata
size limit.
Co-Authored-By: Grzegorz Grasza <xek@redhat.com>
Change-Id: Ida39f5768c67f982b2fe316f6fae4988a74c8534
We have the functional tests going in the CI and novajoin is part
of TripleO and RDO, already being considered to run on many large
production systems, so I think the Beta designation is appropriate.
Change-Id: I7bb279160bb1bc25512898251876293699c5063d