Commit Graph

36 Commits

Author SHA1 Message Date
wu.shiming d783b5889f Drop lower-constraints job
We facing errors related to the new pip resolver, this
topic was discussed on the ML and QA team proposed to
to test lower-constraints [1].

I propose to drop this job because the complexity and recurring pain needed
to maintain that now exceeds the benefits provided by this mechanismes, and
We need to drop the lower-constraints.txt file as well as the corresponding
tox target

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019390.html

Change-Id: Ib0ef1fadf4fe44f1d6af2f7d542b8af919652501
2021-04-16 15:15:18 +08:00
Zuul 1195f95016 Merge "Fix l-c testing for ubuntu focal" 2020-09-10 17:15:40 +00:00
Ghanshyam Mann 49029b8792 Fix l-c testing for ubuntu focal
As per victoria cycle testing runtime and community goal[1]
we need to migrate upstream CI/CD to Ubuntu Focal(20.04).

- Bump the lower constraints for required deps which added python3.8 support
in their later version.

Story: #2007865
Task: #40206

[1] https://governance.openstack.org/tc/goals/selected/victoria/migrate-ci-cd-jobs-to-ubuntu-focal.html

Change-Id: If3ec085184fbba81f455164b803afb6f66e46fd0
2020-09-10 13:02:57 +02:00
Hervé Beraud 6f9cd3f7a7 Use unittest.mock instead of mock
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.

Change-Id: I90d966a150d128e177f157e292035cfb71d89ad1
2020-07-17 09:55:29 +02:00
Doug Hellmann 77e2f67d38 switch to importlib.metadata for entrypoint loading
Importing pkg_resources scans every installed distribution to find
all of the entry points. importlib.metadata can import entry points
without scanning all of the installed packages.

Change-Id: I9917b10292d191710b37838b3cf099d75139f123
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2020-07-05 19:46:27 -04:00
Hervé Beraud 632db25246 Refresh python versions
Introduce support of python 3.8 and move tox and jobs to py38.

jsonschema 3.2.0 [1] support python 3.8

[1] https://github.com/Julian/jsonschema/pull/627

Change-Id: Ibcfa044dd0f3b29fd290559795ea1d98e194e886
2020-05-26 10:36:06 +02:00
Monty Taylor 2ddbf57ee6 Remove use of six
We're python3 now, we don't need to use six.

Change-Id: I7c1f0bc89838432aa0ce916beb462916c8763cd1
2020-05-10 08:29:40 -05:00
Monty Taylor 9b253307a7 Remove some unneeded things from test-requirements
We don't need to depend directly on python-subunit.
In python3 we don't need safe_hasattr.
We don't use testrepository.

In lower constraints, os-client-config and mox3 are
transitive depends that we do not care about so don't
need to track lower pins for them.

futures and ipaddress were removed from requirements
but not from lc.

Finally, remove future from lc and from __future__
since we're python3 only.

Change-Id: Ic22079f78f77221e83986e5a835457dbefc3ded7
2020-05-10 08:23:38 -05:00
Monty Taylor 8e5893f0ad Revert "Switch to futurist for concurrency"
This partially reverts commit b6a22e3749.

The addition of futurist as a hard depend broke python2, and at the
same time didn't mark the release as having done so. In looking at the
issue we realized that we don't actually need the hard depend.

Change-Id: I2d874f618f5b3f66d49cd2e964c6e05655f22c0f
2020-03-26 10:50:26 -05:00
Monty Taylor b6a22e3749 Switch to futurist for concurrency
As we're being used inside of services more and those services
frequently use things like eventlet, allow people to pass in
an Executor object from futurist to override the default
ThreadPoolExecutor object.

Change-Id: I6c04defc28998d49199383a6cc6d5f5611a99e25
2020-03-05 09:06:49 -06:00
Akihiro Motoki 999e38cf08 Bump min version of decorator
decorator.decorate decorator is only implemented in decorator 4.0.0.
The current lower bound of decorator is 3.4.0 but it does not work
with openstacksdk. It is found during a OSC change [1].
This commit bump the minimum version of decorator.
It looks like that decorator 4.0.0 works but we now test with
decorator 4.4.1 (the latest version),
so I chose the latest version as the minimum version.

[1] https://review.opendev.org/#/c/674325/

Change-Id: I7479a84ef19c7f1e881b9d56c12d2798340eb53b
2020-01-09 19:54:17 +09:00
Riccardo Pittau 13c6bc2bd9 Add reset_interfaces argument to patch_node
Ironic uses reset_interfaces option when patching a node to
reset all the hardware interfaces of a node.
This patch adds that argument to the patch_node method in
the baremetal module and introduces a patch method under
the node module to be able to evaluate the reset_interfaces
parameter.
Also, it modifies the _prepare_request method in resource
to accept query parameters in urls and build the request
uri keeping those into account.

Increasing minimum version of mock to 3.0.0 to be able to use
testing features not supported before that version, e.g.
assert_called_once

Change-Id: I8bca403df7d38a7ac1d066c5f1d7e2bff1deb054
2020-01-06 11:21:21 +01:00
Riccardo Pittau 835ab9d133 Increase dogpile version for Py3.7 compatibility
The library dogpile.cache is not compatible with Python 3.7 until
version 0.6.5 because of new reserved word 'async'.
For more info please check:
3c4351c11f

Change-Id: I851ed4ec14cb0a97c9acc38f77cdab6f269ed40c
2019-10-28 15:16:46 +01:00
Zuul 6fa13c234e Merge "Re-add functional tests on flavor content" 2019-10-24 22:17:28 +00:00
Monty Taylor 25674ccd45 Re-add functional tests on flavor content
A new keystoneauth release is out that fixes authenticated
discovery, meaning we'll pick up new server microversion,
which means we can start testing for such in our server
functional test.

Depends-On: https://review.opendev.org/690876
Change-Id: I51b7fd28f3469a5718057ece2f29f3655ef717e4
2019-10-24 17:19:40 +09:00
Eric Fried b54d03a4c0 Make proxy honor raise_exc in REST primitives
Previously, invoking a REST primitive (.get(), .put(), etc.) on a proxy
would ignore the raise_exc kwarg and always use raise_exc=False, causing
any error coming from the actual service [1] to return a Response object
even for responses with status >=400.

With this change, the raise_exc kwarg is honored: when True, REST
primitives with status >=400 will cause an appropriate ksa exception to
be raised.

[1] as opposed to lower level e.g. communication errors, which would
still raise

Change-Id: I463e9a63760a6e61827ba957dd9e5d23bd79f4e8
2019-09-24 17:25:05 -05:00
Eric Fried 6e7cbb9678 Bump keystoneauth1 minimum to 3.16.0
Because we should always be tracking to the latest ksa anyway, but
also...

...to allow from_conf to to take advantage of retry config options
introduced in [1] and [2].

[1] https://review.opendev.org/666287
[2] https://review.opendev.org/672930

Change-Id: I85669b12a6948ab5accd3d7df71dcbacce413e33
2019-08-13 12:09:30 -05:00
Monty Taylor 95bf14908e Add support for global_request_id
Added support for setting global_request_id on a Connection.
If done, this will cause all requests sent to send the request id
header to the OpenStack services. Since Connection can otherwise
be used multi-threaded, add a method global_request that returns
a new Connection based on the old Connection but on which
the new global_request_id has been set. Since a Connection
can be used as a context manager, this also means the global_request
method can be used in with statements.

Change-Id: I70964cdd79741703c0b9b911b3b2f27c248130f0
2019-08-13 08:12:40 -04:00
Eric Fried f7860861a1 Pin to latest os-service-types
OpenstackSDK should always be using the latest os-service-types.

This commit bumps the lower bounds to the current latest version
(1.7.0), and adds a canary test so that the CI will fail as soon as a
new os-service-types version hits upper-constraints.

Change-Id: Ie98ec9cfd58badc2b05a2a144d1d559d7ed6553b
2019-06-06 15:57:25 -05:00
Eric Fried 820790225c Handle oslo.config exceptions in from_conf
In the following cases, we used to happily allow the Connection to the
service, using default Adapter settings:

- If the conf section for a given service is missing, or present but
  without ksa adapter opts registered.
- If the conf section for a given service has bogus values.

Now for these scenarios, we disable that service with a helpful reason.

The service disabling is supported at a broader level: If someone sets
has_{service_type} to false in their config, we remove the adapter for
that service and replace it with something that throws errors when
people try to use it. They can optionally set
{service_type}_disabled_reason to make the message more informative.

Co-Authored-By: Monty Taylor <mordred@inaugust.com>

Change-Id: I3aa1f1633790e6e958bbc510ac5e5a11c0c27a9f
2019-06-06 12:25:00 -05:00
Monty Taylor 86ad9debd1 Make factory for a CloudRegion from CONF objects
This commit enables SDK consumers using oslo.config for keystoneauth1
Adapter settings by introducing a method:

openstack.config.cloud_region.from_conf

This accepts:
- An oslo.config ConfigOpts containing Adapter options in sections named
  according to project (e.g. [nova], not [compute]). Current behavior is
  to use defaults if no such section exists, which may not be what we
  want long term.
- A Session. This is currently required - if unspecified, a
  ConfigException is raised - but in the future we probably want to
  support creating one (and an auth) from the conf.
- Other kwargs to be passed to the CloudRegion constructor.

The method returns a CloudRegion that can be used to create a
Connection.

Needed-By: blueprint openstacksdk-in-nova
Co-Authored-By: Eric Fried <openstack@fried.cc>
Change-Id: I05fb4da39d2eefc91828ace02db2741b62a2cb0a
2019-06-04 08:36:23 -05:00
Monty Taylor c8b96cddd3 Collect request stats
This is a thing that nodepool has been doing for ages. With the upcoming
changes to remove the task manager, the mechanism it has been using to
put activity in the right place isn't going to be available anymore. But
also, people using openstacksdk from within a service might also want to
be able to do the same logging.

This improves upon the old method as well, as it uses the history in the
response object to get and report on all of the calls made as part of a
request. This will catch things that do auto retries.

While we're in there, add support for reporting to prometheus instead.
The prometheus support does not read from config, and does not run an
http service, since openstacksdk is a library. It is expected that
an application that uses openstacksdk and wants request stats collected
will pass a prometheus_client.CollectorRegistry to collector_registry.

Change-Id: I7218179dd5f0c068a52a4704b2ce1a0942fdc0d1
2019-03-11 19:45:04 +00:00
Zuul 47afa68392 Merge "Move pep8 requirements in to test-requirements" 2019-03-11 17:20:03 +00:00
Monty Taylor e7a9b52998 Revert "Revert "Replace TaskManager with a keystoneauth concurrency""
This reverts commit feae8da9bf.

Change-Id: I6f85cc58a404063a262cf7338c827cde61493b43
2019-03-10 22:17:24 +00:00
Monty Taylor feae8da9bf Revert "Replace TaskManager with a keystoneauth concurrency"
We need to cut a quick release for masakari and this is a bit
much to have in a quick release.

This reverts commit c9b60f2b86.

Change-Id: I4771ee4136e54f22979e159932f36040397d4d4c
2019-03-10 22:17:11 +00:00
Monty Taylor 8748f4a4f8 Move pep8 requirements in to test-requirements
We had these in tox.ini before to streamline pep8 (it didn't need
to actually install openstacksdk) But now that we're doing local
hacking checks, we have to install the code, which means this
optimization is bong. Put things back in test-requirements like
sane people. While doing that, remove readme, which isn't in
openstack/requirements and just let doc8 check it.

Change-Id: Id1860a1df33610ad7072c13ef9ee61395b785b94
2019-03-10 16:32:03 +00:00
Artem Goncharov c9b60f2b86 Replace TaskManager with a keystoneauth concurrency
We've added concurrency and rate-limiting controls to keystoneauth. That
means we don't need to do them in openstacksdk.

Depends-On: https://review.openstack.org/#/c/640389/
Change-Id: I5d7bd12606785365d2f5b5b52ec7a2316459b68f
2019-03-02 15:24:59 +00:00
Monty Taylor 603ba0d7d9
Remove all the deprecated stuff
We're really close to a 1.0 release. Let's go ahead and remove the stuff
that was marked deprecated and see what breaks.

Depends-On: https://review.openstack.org/610695
Change-Id: I0b1b7a0060a3cc0ac1772e5ef2d2a9673159cd77
2018-10-15 13:21:57 -05:00
Monty Taylor 071e567b32
Use discovery instead of config to create proxies
Since the dawn of time we've labored under the crippling burden of
needing to explicitly request a version via configuration in order to
get a usable handle to the cloud. This is despite the hilarity of the
existence of a system for discovering available versions since basically
the beginning of the entire OpenStack project.

Today we shall be liberated from the tyranny of terrible past life
decisions on the part of our forefathers and shall usher forth the
shining freedom of actually using the discovery system.

Change-Id: I11c16d37d3ab3d77bed3a0bcbd98f1fa33b9555f
2018-10-06 07:44:29 -05:00
Monty Taylor de981a324a
Add method for returning a raw response for an object
Some use cases are complex and need access to the raw response object.
Add a method, get_object_raw, which just returns the requests response
object.

python-requests needs at least 2.18 for Response to be a contextmanager.

Change-Id: I03f31f33cbef93c30acf56de5e06acf0d721bc03
2018-07-27 19:49:29 -04:00
Josephine Seifert f2aeaead02 Implement signature generation functionality
Openstacksdk is extended with a signature generation functionality.
This adds cryptography to the requirements and lower constraints.

Change-Id: Idc15b9a12d408bd4b2e096da8402c374be56f9fa
Story: 2002128
Co-Authored-By: Markus Hentsch <markus.hentsch@secustack.com>
2018-07-18 13:18:40 +00:00
Dmitry Tantsur 46bb367ad6 Allow configuring status_code_retries and connect_retries via cloud config
Change-Id: I45cf09f9aca8e4ed515a93e4903deb3df83ac65b
2018-06-15 12:19:01 +02:00
Monty Taylor 264c802ad8
Pass default_microversion to adapter constructor
Based on discussion with the API-SIG, we determined that if someone sets
*_api_version or OS_*_API_VERSION to a value that looks like a
microversion, we should pass it to default_microversion in addition to
using it for major version discovery.

Update the docs to reflect the microversion story (and to remove shade
references)

Also, throw an error on missing support for default microversion

We've given the user the ability to set a default microversion. However,
that microversion might not be available on the cloud in question. Throw
an error in such cases.

Depends-On: https://review.openstack.org/568640
Change-Id: Iad7194dc9e933660c0d0df9130c51d505dda50cb
2018-06-09 09:15:20 -05:00
Monty Taylor 9c774d2959
Defer all endpoint discovery to keystoneauth
keystoneauth has support for service type aliases and version discovery.
Stop doing it locally and just pass data to keystoneauth.

Depends-On: https://review.openstack.org/567602
Change-Id: If60d02a8216ca0719fa628431515a0c3b37bf607
2018-05-10 12:06:42 -05:00
Monty Taylor c9daa593ac
Add ipaddress and futures to lower-constraints
These were missing but that didn't get noticed before. Changing the
requirements in the next patch triggers the error.

Change-Id: I65605e14fc73e4eb223b7c31a98d4758ca280c9d
2018-05-10 11:59:23 -05:00
Doug Hellmann 1f70f692e6 add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: I7ec32aefc9caac30f4e93704fc33b9a98cb1c622
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-03-25 09:07:32 -04:00