This adds configuration of the default ephemeral encryption format and
sets default encryption attributes in the driver block device mapping
when needed. This includes generation of a secret passphrase when one
has not been provided.
Co-Authored-By: melanie witt <melwittt@gmail.com>
Related to blueprint ephemeral-encryption-libvirt
Change-Id: I052441076c677c0fe76a8d9421af70b0ffa1d400
md5 is not an approved algorithm in FIPS mode, and trying to
instantiate a hashlib.md5() will fail when the system is running in
FIPS mode.
md5 is allowed when in a non-security context. There is a plan to
add a keyword parameter (usedforsecurity) to hashlib.md5() to annotate
whether or not the instance is being used in a security context.
In the case where it is not, the instantiation of md5 will be allowed.
See https://bugs.python.org/issue9216 for more details.
Some downstream python versions already support this parameter. To
support these versions, a new encapsulation of md5() has been added to
oslo_utils. See https://review.opendev.org/#/c/750031/
In this case, md5 is used to generate fingerprints when ssh keys are
being generated and imported. Without this patch, these operations
fail on FIPS enabled systems.
Change-Id: I1fe8e8bb181fa2a704eec38be09619e5a648f2f1
Provide a method to communicate with the configured key manager service
using Castellan, a generic key manager interface that supports multiple
backends including - but not limited to - Barbican. Once again, there's
nothing using this yet but tests, though this will change shortly.
Part of blueprint add-emulated-virtual-tpm
Change-Id: Iff6195d252b018f008bb9d137e4d80c54b70b2d1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Nothing to complicated here, other than working around mypy's dislike of
changing variable types and inability to process 'if six.PY3'
conditional blocks because it doesn't run the code.
Part of blueprint add-emulated-virtual-tpm
Change-Id: I805eaa8b6fb55ce9cbc8f6b8b777af48302ba2ba
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
These particular imports are no longer needed in a Python 3-only world.
Change-Id: Ia1b60ce238713b86f126e2d404199d102fdbc5bc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Its a bit concerning that this doesn't have unit test coverage,
but I'm just here to make things no use utils.execute().
Change-Id: I76ccea47245d46def4b500e29681200ae951b7d0
blueprint: execs-ive-had-a-few
These top-level functions were not called from anywhere but tests.
- fetch_ca
- ensure_ca_filesystem
- fetch_crl
- decrypt_text
- revoke_certs_by_user
- revoke_certs_by_project
- revoke_certs_by_user_and_project
- generate_x509_cert
- generate_vpn_files
These other functions are used by the above and are no longer used
anywhere.
- ca_folder
- ca_path
- key_path
- crl_path
- revoke_cert
- _project_cert_subject
- _user_cert_subject
- _ensure_project_folder
- sign_csr
- _sign_csr
Tests for these are removed as are a number of scripts found in
'nova/CA', which were only used by the aforementioned functions.
Change-Id: Ie1dadc6bf935f777e0cd0c54a0a21b79545714c5
This drops the paramiko < 2 compatibility code so we only need to
support one major version.
Depends-On: I2369638282b4fefccd8484a5039fcfa9795069a7
(global requirements change)
Change-Id: Ife4df9e64299e1182d77d568d1deed5ec3b608b3
Closes-Bug: #1483132
Only use PyCrypto/PyCryptodome work-around with Paramiko 1.x and use
straight-forward Paramiko interface with 2.x.
TODO: Revert this and PyCrypto/PyCryptodome work-around when Paramiko
is upgraded to 2.x (ie replace `generate_keys(bits)` call with
`paramiko.RSAKey.generate(bits)`).
Change If88beeb3983705621fe736995939ac20b2daf1f3 added a work-around
for the partially-PyCrypto-compatible PyCryptodome causing Paramiko,
which has a dependency on PyCrypto, to break. This work-around
entails implementing Paramiko internals (ie how to generate a key) in
Nova in a way compatible with both PyCrypto and PyCryptodom.
This work-around is itself a source of failure with Paramiko 2 which
has replaced the PyCrypto requirement with the cryptography Python
package. As Paramiko no longer depends on PyCrypto, Nova doesn't have
an explicit PyCrypto requirement, and there's no implicit dependency
on PyCrypto, when Nova tries to import PyCrypto it fails. Even if
PyCrypto was installed, the work-around would still fail because the
Paramiko interface that Nova is using as part of the work-around
changed with the major version change (ie 1.x => 2.x).
Change-Id: I5d6543e690a3b4495476027fd8a4894ff8c42bf6
Related-Bug: #1483132
Move all crypto configuration into [crypto] group. Update references to
these configuration options to reflect new group.
Change-Id: I660b5c20e2b33f276a9d4c49b0e6279d7d8e41c7
The config options of the "nova.conf" section "crypto"
got moved to the new central location "nova/conf/crypto.py"
Change-Id: Ia5e970694d384ef39a6050efb5db2f61e6f4205b
Implements: blueprint centralize-config-options-newton
Newer versions of pysaml2 uses pycryptodome, so if by
accident if this library gets installed, Nova breaks.
paramiko folks are working on this:
https://github.com/paramiko/paramiko/issues/637
In the meanwhile, we should tolerate if either pycrypto
or pycryptodome is installed.
Closes-Bug: #1545370
Change-Id: If88beeb3983705621fe736995939ac20b2daf1f3
Since oslo 1.6.0 timeutils.isotime() and timeutils.strtime() methods
are deprecated.
DeprecationWarning: Using function/method
'oslo_utils.timeutils.strtime()' is deprecated in version '1.6' and
will be removed in a future version: use either
datetime.datetime.isoformat() or datetime.datetime.strftime() instead
Change-Id: If69bd8a6bee052556ba8853afef3941bcd1e7b13
Closes-Bug: 1479056
Co-Authored-By: Diana Clarke <diana.joan.clarke@gmail.com>
For example in:
Ie4e95999795d349a5897f7a180e34187485bd8f1
we added a try/finally to switch current directory to execute a command
and switch back to original directory. With the new cwd parameter
added in processutils.execute we can just use the cwd parameter and
not have to do the extra explicit steps to switch directories. This
cwd just delegates to subprocess.Popen's cwd param.
Related-Bug: 1414530
Change-Id: I808f51f8ac750b5c07a2988ff0ed2f40a68e1ee1
This patch is a code reduction cleanup fix. Exec calls to openssl
can mostly be replaced with a proper python library called
cryptography.
Changed the following:
* Removed convert_from_sshrsa_to_pkcs8 and _to_sequence since
cryptography already has a method to do this conversion.
* Replaced openssl rsautl call in ssh_encrypt_text and decrypt_text
with cryptography's public key encrypt and decrypt methods.
* Replaced call to openssl x509 with cryptography's
load_pem_x509_certificate which also includes a way to retrieve
the fingerprint.
* generate_fingerprint's calls to paramiko were replaced with
cryptography's serialization module. Cryptography supports all
key sizes of elliptic curve keys whereas paramiko did not.
* Removed the requirement on pyasn1
Change-Id: I750b49e6a73cd18af8199c1c859c9f6047252387
fileutils is graduated in the oslo.utils library.
Remove old implementation of the read_cached_file function
in the nova.utils and move from the fileutils the read_cached_file
and delete_cached_file functions to the nova.utils module and
the unit tests for these functions.
Implements: blueprint graduate-fileutils[1]
[1] https://blueprints.launchpad.net/oslo-incubator/+spec/graduate-fileutils
Depends-On: I51ba9076e1fbc16145ee2311f47b7768c16dcb20 (requirements)
Change-Id: I849f1c74ec811dbe82ba6270569a008a49eab465
Fix bytes vs Unicode issues:
* ssh_encrypt_text() now encodes text to UTF-8 if text type is Unicode.
* Encode Unicode to ASCII before encoding it to base64
* On Python 3, decode binascii.hexlify() and base64.encodestring()
output from ASCII to get Unicode (str type in Python 3) is required.
* convert_from_sshrsa_to_pkcs8(): reuse binascii.hexlify() instead of
a loop using struck.unpack() and "%02X" format.
* Replace str.encode('hex') with binascii.hexlify(str)
* Replace string.strip(text) with text.strip()
* Replace StringIO.StringIO with six.StringIO
* Add test on the output type of ssh_encrypt_text() and decrypt_text()
* Call utils.execute() with binary=True to not decode stdout/stderr
(to get them as raw bytes).
* Replace reduce() with six.moves.reduce()
* convert_version_to_str(): replace a/b with a//b to get integer
division
* tox.ini: add the following tests to Python 3.4
- nova.tests.unit.compute.test_keypairs
- nova.tests.unit.test_crypto
Blueprint nova-python3
Change-Id: I83d927166c0864020b205ac7495473795da7830d
With this change, "tox -e py34" now pass.
Changes:
* Replace unichr() with six.unichr()
* Replace StringIO.StringIO() with six.Bytes() in
nova.crypto.generate_key_pair()
* On Python 3, replace UserDict.IterableUserDict with
collections.UserDict
* Replace __builtin__.open with six.moves.builtins.open
* Replace ConfigParser import with six.moves.configparser
* On Python 3, get the original "queue" module instead of the "Queue"
module in nova/virt/libvirt/host.py
* Replace urllib2 with six.moves.urllib
* tox.ini: "tox -e py34" now only runs nova.tests.unit.test_versions
Blueprint nova-python3
Change-Id: I056769c7c5b32276894f7aade8c0a27af81c42ae
Currently, the Subject created for the X509 certificate
is too long, resulting in exceptions and failing to
create the keypair.
This change shortens the Subject.
Unit test added to prove the issue's existence.
Issue found by: Andrey Kurilin
Co-authored-by: Andrey Kurilin <akurilin@mirantis.com>
Change-Id: I7885c120ce81a22d416f806779bf9a12092d5040
Closes-Bug: #1447653
Nova already has a dependency on paramiko and therefore should
take advantage of it for generating key pairs. This will reduce
the code complexity and remove calls to exec.
Change-Id: Ibb01f5227ded9b79816c064a06a1f6724f765e78
Removes unused arguments in test_keypairs.
Sets the keypair 'type' field to enum in the v2.2 validation
schema.
Removes check if the keypair 'type' is valid, as it will be done
by the validation schema.
Adds functional tests for ssh and x509 keypairs.
Closes-Bug: 1434033
Change-Id: I39f0a363458a17a4011a0661a45dc449f122ba1a
X509 certificates are used by Windows for passwordless
authentication (WinRM) in a way which can be considered
consistent with the usage of SSH keys on Linux, as both
are based on public / private keypairs.
Adds x509 certificate creation implementation.
Adds 'keys' in InstanceMetadata.
Implements: blueprint keypair-x509-certificates
Depends-On: Id5b210d7afe5c0a590abcbd42b9ff85b071a5c55
Change-Id: I1ab518553cf757ae35548fa89b6c8fee7ec32b8d
Convert the use of the incubated version of the log module
to the new oslo.log library.
Sync oslo-incubator modules to update their imports as well.
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
Change-Id: Ic4932e3f58191869c30bd07a010a6e9fdcb2a12c
The call to utils.execute ends up in /opt/stack/nova/nova/utils.py which
ultimately calls processutils.execute() in the oslo_concurrency module.
If there's an error when executing the command which calls an bash script
then an exception ProcessExecutionError will be raised.
This patch adds a finally statement to ensure the dir will be switched back
to its original one.
Closes-Bug: #1414530
Change-Id: Ie4e95999795d349a5897f7a180e34187485bd8f1
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
This commit includes:
- using oslo_utils instead of oslo.utils
- using oslo_serialization instead of oslo.serialization
- using oslo_db instead of oslo.db
- using oslo_i18n instead of oslo.i18n
- using oslo_middleware instead of oslo.middleware
- using oslo_config instead of oslo.config
- using oslo_messaging instead of "from oslo import messaging"
- using oslo_vmware instead of oslo.vmware
Change-Id: I3e2eb147b321ce3e928817b62abcb7d023c5f13f
oslo.concurrency-0.3.0 has moved its path to oslo_concurrency,
the old path oslo.concurrency can still work but is deprecated now.
Change-Id: I1538dad5cbacf1e6041c571d5bc55994689c08b0
Closes-Bug: #1398656
oslo.i18n uses different marker functions to separate the
translatable messages into different catalogs, which the translation
teams can prioritize translating. For details, please refer to:
http://docs.openstack.org/developer/oslo.i18n/guidelines.html#guidelines-for-use-in-openstack
There were not marker fuctions some places in directory network.
This commit makes changes:
* Add missing marker functions
* Use ',' instead of '%' while adding variables to log messages
Added a hacking rule for the log exception about checking
translation for it.
Change-Id: If80ea6f177bb65afcdffce71550bb38fedcc54eb
Let's switch to the newly released oslo library for the
processutils and lockutils. We use the config fixture(s) to
specify disable_process_locking and lock_path in the CONF
variable of oslo.concurrency library for correctly setting the
flags.
The previous attempt at updating nova to use oslo.concurrency
was reverted as it stopped running any tests. lockutils-wrapper
should have been used in ini/shell scripts
instead of lockutils.py module as the the module is
no longer runnable (__main__ has been removed). In this
review we use lockutils-wrapper correctly. The test harness
has since been updated to bail out if 0 tests are run as
well for additional insurance.
Change-Id: I14c75ef2196fd08c9465d648e5cd53777bd9c7d1
Co-Authored-By: Nikola Dipanov <ndipanov@redhat.com>
Co-Authored-By: Corey Wright <corey.wright@rackspace.com>
This patch made unit tests stop running (always runs 0 unit tests) we
should revert and put a check in place to prevent that from happening
so that we don't regress the rest of Nova trying to figure out the
oslo.concurrency issue.
This reverts commit 382294762b.
Change-Id: I98b04ac2a0ba67ba8af44867799317fdf95b8aff
Cert revocation was broken by
32b0adb591. os.chdir() never returns
anything, so this method would always raise an exception. The proper
way to handle an error from os.chdir() is to catch OSError.
There were existing tests for this code, but they conveniently mocked
os.chdir() to return values that are never actually returned. The
tests were fixed to match the real behavior.
Change-Id: I7549bb60a7d43d53d6f81eecea31cbb9720cc8b6
Closes-bug: #1376368
oslo.utils library now provides the functionality previously in
oslo-incubator's excutils, importutils, network_utils, strutils
timeutils, units etc. Some modules already moved to oslo.utils
will still be around since other code in nova/openstack/common/
are using it and will be removed in a subsequent commit.
Change-Id: Idc716342535fdfa680963e0e073ddb46f5f1eb34
On Python's without reference counting (e.g. PyPy) the current code will keep
the file descriptors open for an arbitrary length of time, until the GC runs.
Change-Id: Idf81df95894fed1d13a68705ef490d5a29367ff1
According to NIST 800-131A, RSA key lenght for digital signature
must >= 2048 bit. Now we use 1024 bit key to generate x509 cert
file. Need to increase the key length to 2048 bit.
Change-Id: I59f614b5d8a79f9e0a96503867cfca176be5c757
Closes-Bug: 1369487
oslo.i18n provides the i18n functions that were provided by
oslo-incubator's gettextutils module. Some tests that were
using internal details of the library were removed.
Change-Id: I44cfd5552e0dd86af21073419d31622f5fdb28e0
Catch processExecutionError if revoking the
certificate fails.
This change has been abandoned by Chuck Short, so
I am continuing where he left off (mattoliverau).
Continues abandoned change: 17741
Change-Id: I9714ea8cece87256ff5f9a936286c1da3d628af9
Closes-Bug: #883320
Co-Authored-By: chuck.short@canonical.com
Our translation policy
(https://wiki.openstack.org/wiki/LoggingStandards#Log_Translation) calls
for not translating debug level logs. This is to help prioritize log
translation. Furthermore translation has a performance overhead, even if
the log isn't used (since nova doesn't support lazy translation yet).
This is the final patch ensuring that all fines in the nova directory
no longer have translations for debug messages.
Change-Id: Ib10f5826e2238cbb2d4315c48be570eb31b2ea55
An SSH key generated by Nova contains the comment 'Generated by Nova'. Older
versions (prior to 0.7.2) of cloud-init trip over the spaces in the comment
and as a result of that the key injected into the root account is not disabled
by cloud-init. Yes, it's a cloud-init bug but it's also a regression in the
sense that older OpenStack installations (Essex and older) don't contain
spaces in Nova generated key comments and thus older cloud-init's are not
affected in these environments.
This patch replaces the spaces with dashes, i.e., 'Generated-by-Nova'.
Spaces were introduced by commit: 114109dbf4
Ubuntu cloud images with cloud-init < 0.7.2: 10.04 LTS and 12.04 LTS
cloud-init bug report: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1220273
Closes-Bug: #1297685
Change-Id: I1761f61dfbba58be98351ae4a51884b03268cf09
Remove nova.crypto.compute_md5() as it is:
* unused
* last user removed 3 years ago with commit bab3060
* untested
* incorrectly documented
* return type is not a tuple, but a string
* redundant
* copied from boto which is a nova requirement
* burdened with license text more verbose than associated code
Change-Id: Ia033d3359098fc0af8de00248dc14b4fe280e844
We don't need to have the vi modelines in each source file,
it can be set in a user's vimrc if required.
Also a check is added to hacking to detect if they are re-added.
Change-Id: I347307a5145b2760c69085b6ca850d6a9137ffc6
Closes-Bug: #1229324
When doing a keypair-add, I was hitting a permission denied issue with
ssh-keygen -l. This turned out to be a configuration issue, but the
error message on the client side was pretty unhelpful with debugging
the issue:
BadRequest: Keypair data is invalid (HTTP 400)
Fix that so we get:
BadRequest: Keypair data is invalid: failed to generate fingerprint (HTTP 400)
Also fix the other uses of InvalidKeypair so we include the general
"keypair data is invalid" prefix.
Change-Id: I209e76adaf7c18a9f2f2dbb6da62db6c1cd9461e
Previous _ was monkey patched into builtins whenever
certain modules were imported. This removes that and
simply imports it when it is needed.
Change-Id: I0af2c6d8a230e94440d655d13cab9107ac20d13c
It was being used as a hex string parser, which is not necessary given
that the int builtin can do this.
Change-Id: I710093fc83c175ac08d827bb93ba878703ee817a
There are some small warts (mostly around other execute() like
methods which reuse the exception). I will fix these warts in later
reviews.
Change-Id: Ice9cdbdc5f3e5a9f8365f5d99acf1863a9fe3e7a
Lee Yarwood