Commit Graph

2072 Commits

Author SHA1 Message Date
Andreas Jaeger 08c07018dc Fix pep8 and py27, functional jobs
pyflake8 was updated and not properly constrainted by hacking, update
to hacking 3.1.0 and fix the problems found.

py27 failed since global upper-constraints includes py3 only packages.
Use lower-constraints for this test.

The functional jobs run on master with py27, remove the py27 master job,
change the jobs so that py3 is used everywhere.

Change-Id: I695d60e53857e03f206dcd7cf10c06c76c8d6cc0
2020-06-01 07:54:56 -05:00
Sean McGinnis 3db89db8c6 Break out py27 job templates
To allow non-py27 supporting repos to use shade job templates, this
breaks out the py27 and py3 jobs into separate templates. Relevant repos
can then continue to use the existing template names without picking up
the unsupported job runtimes they no longer need.

Remove the py27-tips jobs - they just don't make any sense any more.

Change-Id: Ia211e31723ede592f19a8df6c18e45699ca25788
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-11 13:20:26 -05:00
Yves-Gwenael Bourhis 8e183ca4d2 URL encode swift objects endpoints
Swift containers and objects can contain any kind of characters.
If a container or name contains special characters, shade was failing to
access them.
We need to url encode containers and objects when creating the url to access
them.

Story: 2005828
Task: 33585
Change-Id: I170277293eb4ed1a12cf62bcf863707cb3d918eb
2019-06-07 14:34:48 +02:00
Zuul 0d1858d4bc Merge "fix the word 'bandwith'" 2019-03-10 17:26:02 +00:00
Morgan Fainberg b23000c660 Fix dogpile.cache 0.7.0 interaction
Due to the change in behavior in dogpile.cache > 0.7.0 where bound
methods can no longer be passed to the cache_on_arguments decorator,
openstackSDK now explicitly provides a non-method wrapper for all
elements passed to cache_on_arguments. This is handled via an explicit
no-op decorator. functools.wraps is used to preserve data from the
original method.

Needed-By: https://review.openstack.org/#/c/624993
Change-Id: Ied27fa1e834d145246815afcb67c59d48669ffb2
Story: 2004605
Task: 28502
2019-02-22 16:31:40 +00:00
melissaml 169a6d3b5a fix the word 'bandwith'
Change-Id: If919871f982a76fb5c30ea36cfae9fea6a39480f
2019-02-18 03:52:44 +00:00
Monty Taylor 0ec8b8e2df
Restrict inventory test to devstack-admin
There are multiple different cloud entries in clouds.yaml now. That
means that doing an inventory across all configured clouds gets weird,
especially when we're tossing system scoped accounts in there.

Cherry-picked from https://review.openstack.org/622373/

Change-Id: I0530ca35da0cc3baebab2c3bc53eea70028ceed5
2019-01-13 14:48:20 +00:00
Zuul cad15cca9a Merge "Fix grant_role() when user not in default domain" 2019-01-09 15:01:04 +00:00
Clint Byrum 186dac6535 Split parser creation and parser for inventory
Access to the object separate from parsing allows us to use
argparse-manpage to generate a man page in the Debian package
automatically. We also need to set 'prog' explicitly rather
than let it be picked up via argv[0] so that when we load
the parser via argparse-manpage it produces the right value.

Change-Id: I654b4408444f804f900951333a6ebc3372d5037e
2019-01-07 03:39:43 -08:00
wacuuu 9cef36534a Fix grant_role() when user not in default domain
Cherry-picked from 3fce61329 in openstacksdk:

3fce613297

Change-Id: I67f91ac2096241507ea2df01fa2b85c0014d86ef
Story: 2004624
Task: 28550
2018-12-17 13:46:33 +01:00
Monty Taylor 6f40858c05
Support v4-fixed-ip and v6-fixed-ip in create_server
novaclient supports v4-fixed-ip and v6-fixed-ip as aliases for fixed_ip.
As shade used to use novaclient and passed this content through the
switch to REST resulted in a behavior regression for shade users. That's
bad, mmkay?

Add in support for the same aliases to create_server, so that users
upgrading to newer shade are hosed.

Closes-Bug: #1798489
Change-Id: I55bc88e9b2cfa5ad1b6a27640db11bc848584c21
2018-10-18 10:42:24 -05:00
Monty Taylor 3b2cad5d31
Decouple OpenStackCloud from Connection
Revert the openstacksdk subclassing from shade. The idea was to
reduce the workload, but trying to make sure that the Cloud abstraction
in openstacksdk doesn't break shade's contract while we update things is
a ton of work to meet the contract that's not really valuable to people.

Instead, we'll put shade on lifesupport and only accept bugfix patches.

Revert "Make OpenStackCloud a subclass of Connection"

This reverts commit ab3f400064.

Revert "Use openstack.config directly for config"

This reverts commit 2b48637b67.

Revert "Remove the task manager"

This reverts commit 28e95889a0.

Change-Id: I3f5b5fb26af2f6c0bbaade24a04c3d1f274c8cce
2018-10-16 08:58:01 -05:00
Monty Taylor 28e95889a0
Remove the task manager
The underlying openstacksdk library is shifting how task manager works,
so stop trying to send it a task manager. This is a feature basically in
place just for nodepool which is being expanded to be more usable by
everyone. The likelihood that anyone other than nodepool is using it is
... very low.

Change-Id: I04be3937589a805a5f9686c91a78933eebcfa022
2018-09-25 07:17:39 -05:00
Chuck Short 8e56081226 Replace assertRaisesRegexp with assertRaisesRegex
This replaces the deprecated (in python 3.2) unittest.TestCase method
assertRaisesRegexp() with assertRaisesRegex().

Change-Id: I1f20db33e3acfc87c0e7a5c8cf45dc1b3121619e
Signed-off-by: Chuck Short <chucks@redhat.com>
2018-08-27 09:35:38 -04:00
Zuul b274873695 Merge "python-shade expose MTU setting." 2018-07-30 18:17:45 +00:00
Zuul dc434983b6 Merge "Add support for static routes" 2018-07-30 18:06:35 +00:00
Ryan Brady 1ddc618d2d Add support for static routes
The networking API v2 specification, which is implemented by OpenStack
Neutron, features an optional routes parameter - when updating a router
PUT requests). Static routes are crucial for routers to handle traffic
from subnets not directly connected to a router.  This patch adds the
routes parameter to the OpenStackCloud.update_router method as a list of
dictionaries with destination and nexthop parameters.

Example:
[
   {
      "destination": "179.24.1.0/24",
      "nexthop": "172.24.3.99"
    }
]

Change-Id: I14205b6bb071d0b46967f29b6287f74d8796add8
2018-07-26 07:54:15 -04:00
Toure Dunnon 545b5aae28 python-shade expose MTU setting.
The networking API v2 specification, which is implemented by
openstack neutron, features an optional MTU parameter - when
creating a network, this allows operators to specify the value
for the maximum transmission unit value.

Change-Id: I288f02551555fff3e8b350fc6d7c6ae8f60c405c
2018-07-23 13:06:24 -04:00
Hongbin Lu 51e2989fd0 Use valid filters to list floating IPs in neutron
Shade used 'attached' as a filter for listing floating IPs,
but this is not a supported parameter in neutron. In before, the test
passed because neutron server ignored this parameter and returned
all the floating IPs. However, neutron is planing to employ a strict
validation on the list endpoint [1]. As a result, neutron will reject
the request with 400 response if the request contains unknown filter.

This patch fixes the usage of neutron API. It passes filters to neutron
API only if they are known supported filters. The list of supported
filters for listing floating IPs can be found in the neutron API
reference [2].

[1] https://review.openstack.org/#/c/574907/
[2] https://developer.openstack.org/api-ref/network/v2/#id130

Change-Id: I124adfc5e9cdd5bc20aacf23f9dcc10c55b0870b
2018-07-19 19:30:15 +00:00
Valentin Boucher ebf7f58763 Fix doc mistake
The wait arg in create_stack have
a mistake in doc description

Change-Id: I402a4586e5edee50fb3bfbc9b14429d9c1fd14bc
Signed-off-by: Valentin Boucher <valentin.boucher@kontron.com>
2018-07-19 12:04:26 -04:00
Zuul d32f20c740 Merge "Make OpenStackCloud a subclass of Connection" 2018-07-17 17:14:19 +00:00
Zuul 7f88a79624 Merge "Improve Magnum cluster templates functions" 2018-07-15 22:08:24 +00:00
Monty Taylor ab3f400064
Make OpenStackCloud a subclass of Connection
The shade code lives in openstacksdk now and is a part of Connection.
Start making shade a subclass of that for backwards compat.

Mostly only deal with the constructor for now, but go ahead and do
list_flavors and get_flavor_by_id while we're looking at the extra
config override. Also remove list_projects because of its use of the
_normalize module.

keystoneauth lower-constraint needs to be bumped because of
status_code_retries.

Change-Id: Ibbe6e167d6285b30a9adbd0c5a89bc679c5645f3
2018-07-13 06:17:25 -05:00
David Shrewsbury 7460ad36d8 Fix for passing dict for get_* methods
We are not accepting dicts for the name_or_id parameter, only objects.

Change-Id: I31f0f127f71f10a2f11f89e10ac8911816786963
2018-07-03 16:31:58 -04:00
Monty Taylor e95d8e9abf
Finish migrating image tests to requests-mock
We let some of these slip through originally. Fix them so we can do
other things.

Guess what? There were bugs in the v1 codepath.

Change-Id: Ia51598548cde0ddb4f9e96b166274e9a104cf649
2018-06-24 16:36:04 -05:00
Monty Taylor 7c9d461796
Convert image_client mocks in test_shade_operator
These two methods were mocking the image client.

Change-Id: I2a5c249f55662caaf3a669283721f283f6b1b565
2018-06-24 10:36:34 -05:00
Monty Taylor 43977d15c4
Convert test_caching to requests-mock
Remove the last bits in test_caching that were mocking clients directly.

Change-Id: I4477993ca2d87bd9bfa4542f7884e41182ee2040
2018-06-24 10:36:34 -05:00
Monty Taylor 43e216ba57
Convert domain params tests to requests_mock
We still have places using client method mocking. Remove them.

Change-Id: I34fbc19aca4b6514215ed497117c635fa810473d
2018-06-24 10:36:34 -05:00
Monty Taylor abd61fd839
Use RequestsMockTestCase everywhere
We still have classes using the old base class. Fix them.

Change-Id: I8aa24d3dc641225f1e24e473d4a5966f11cc8c00
2018-06-24 10:36:34 -05:00
Feilong Wang dcaf63b608 Improve Magnum cluster templates functions
There are two improvements in this patch:

1. Add alias to rename xxx_cluster_template to xxx_coe_cluster_template.
This a pre clean so that we can support xxx_coe_cluster to avoid conflits
with Senlin's functions.

2. Support new Magnum API endpoints, /clustertemplates and /clusters.
Those two endpoints added in Newton.

Change-Id: I78c37f0df8f63a13c534f3dcaca2e27073f0d730
2018-06-22 15:37:44 +12:00
Mohammed Naser 07a4b846ee Change 'Member' role reference to 'member'
Keystone has changed the default role during bootstrap from 'Member'
to 'member' therefore the tests which operated under the assumption
of having the 'Member' role assigned are no longer passing.

This patch addresses this issue, however, long term, it's probably
better to create a role and assign it during the test rather than
assuming one already exists, but that's probably for another patch.

Change-Id: I6e4b7665ab046661dd763a1ff8c8e54e6db2dc18
2018-06-18 22:46:32 -04:00
Zuul 5ff6e314ac Merge "Switch to iterable version of server listing (and expose iterable method)" 2018-05-27 14:18:28 +00:00
Monty Taylor 60aafcc4bc Allow explicitly setting enable_snat to either value
The original code did not allow someone to set enable_snat to True.
Turns out on some clouds the default is the opposite.

Change-Id: Ic3bbfa35f7c6184faa60922b4dd796708de57136
2018-05-22 15:43:22 +02:00
Joshua Harlow ab28399aac Switch to iterable version of server listing (and expose iterable method)
This adds a new server listing mechanism that will yield back each server
block that is returned; it is useful for large clouds with projects with
a large number of active or deleted instances.

For example, in the current list servers it builds up a few lists which can grow
quite big (and consume a lot of memory) making it hard to list projects servers
that have existed for a long time, but with an iterable verion those lists do
not grow (since they just consume the current chunk).

Also the iterable version is more easily used in programs with progress bars
or anything like that that can take advantage of the knowledge that the listing
is at a given chunk (of many).

Change-Id: I84c15e664fb2f72f563618483f9df1c99d215faf
2018-05-21 13:27:40 -07:00
Monty Taylor dcbcfbf124
Fix recent pep8 issues
The latest version of pycodestyle got stricter with undefined objects
pulled in via * imports. It's a nice check and actually found a bug, so
instead of disabling it, fix the places where we're using a * import.

W503 on the other hand is terrible.

Change-Id: Id73599ca494545b0f6d2345236190fc13cf2a326
2018-05-12 10:31:30 -05:00
Monty Taylor 2b48637b67
Use openstack.config directly for config
shade depends on openstacksdk already. Let's go ahead and start
consuming it directly rather than through the os-client-config compat
layer. We have to leave the os-client-config depend for the legacy
client functions.

Change-Id: Ibe2176c16e29960e9cc2b569325e41d1a4e297a8
2018-05-04 11:30:09 -05:00
Monty Taylor b95b0c7f60
Make name setting in connect_as more resilient
This setting of the name is currently a hack and is mucking about
behing the OCC scenes. We need to be able to step out of it, but to
do that we need to muck about behind the SDK scenes for a second too.

Change-Id: I772a3d99c354f32c6341d8f409a8e839461af736
2018-04-27 13:35:35 -05:00
Zuul 2a59fb58f2 Merge "Backport connect_as fix from openstacksdk" 2018-04-27 12:56:11 +00:00
Monty Taylor 94c1ac842d
Backport connect_as fix from openstacksdk
The connect_as method tried overlaying things directly on top of the
'defaults' cloud config, but things get all messed up and it's actually
never what you want.

Instead, pull the existing CloudRegion from the local config object and
pass it in.

Change-Id: I54593bd8f6dd2124df854e923456c390e0d6a147
2018-04-26 16:05:40 -04:00
Paul Belanger f5e8ac20c7
Add get_volume_limits() support
Add the ability for a user (nodepool) to get the volume limits of a
cloud.

Change-Id: Id568d3ad408dd8976211c0576c45e8c9471d1849
Depends-On: https://review.openstack.org/564279
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-04-25 12:38:33 -04:00
Zuul 7d4458448c Merge "list_servers pagination support" 2018-03-31 13:47:17 +00:00
Antoni Segura Puimedon 6040b34345
list_servers pagination support
If we want to list the servers of a deployment with more servers than
nova.conf max_limit, currently we only return max_limit. This patch
makes it behave as expected, getting you all the servers.

Change-Id: I58844125ba37d81f1611e4e8ec641c1b8573884d
Signed-off-by: Antoni Segura Puimedon <antonisp@celebdor.com>
2018-03-23 18:31:29 +01:00
Romain Acciari f609876f2d create_subnet: Add filter on tenant_id if specified
Change-Id: I073b53c6bb34fa904e9595169b135fde7c393d0e
Task: 12102
Story: 2001744
2018-03-12 00:14:54 +01:00
Zuul 43419ef2c4 Merge "Adds toggle port security on network create" 2018-03-10 14:43:43 +00:00
Rarm Nagalingam 5b81959b32 Adds toggle port security on network create
Added a new property, 'port_security_enabled' which is a boolean to
enable or disable port_secuirty during network creation. The default
behavior will enable port security, security group and anti spoofing
will act as before. When the attribute is set to False, security
group and anti spoofing are disabled on the ports created on this
network.

Change-Id: If984a82ca5f6fb69ee644f4fa84333df09d7f8bc
2018-03-09 21:05:05 +11:00
Zuul d1451d022a Merge "Add proper return value for validate_node" 2018-03-08 23:09:15 +00:00
Olivier Bourdon 592ac2c7d0 Add proper return value for validate_node
Change-Id: I361c520e6697750137e86d32dc3890fd8af4a99c
2018-03-08 10:55:37 +01:00
Olivier Bourdon da6301bc46 Add extra failure codes to bad request exception
For instance on call to cloud.node_set_provision_state(uuid, 'dummy')
the API currently only returns:
(400) Client Error for url: http://127.0.0.1:6385/v1/nodes/<UUID>\
/states/provision Bad Request
whereas the HTTP request also contains some more valuable information
in the body:
The requested action "dummy" can not be performed on node "<UUID>" \
while it is in state "enroll".

Change-Id: I8412a2fd4ba78ad4f43ac2f13b1404dd3c151ce5
2018-03-08 07:35:20 +01:00
Monty Taylor 41538c865d
Fix private_v4 selection related to floating ip matching
The change to look for floating ips first when looking for the public
address exposed a latent bug in the private ip finding
code. When we get the mac address of the floating ip, we then look for
the corresponding fixed ip to return for private_v4. However, we weren't
specifying fixed before, so we were just getting the first one that
matched. That *happened* to be the fixed ip by accident.

Add in ext_tag='fixed' so that we look for the matching MAC from a fixed
interface.

We have to also add a second pass through the loop without the fixed
tag, as old nova network dicts do not have the fixed/floating tag like
that.

Includes a test which shows the breakage.

Story: 2001619
Change-Id: I60562a99f78c0c363f49106c285935448f804084
2018-03-02 08:51:26 +01:00
Tobias Brox 9282eb2fdd Functional test robustness and performance fix
In TestNetwork._cleanup_networks, pass network id instead of network name.
Task: 6594
Story: 2001617

Change-Id: I696d6b9a8abf3e2d064bd276d1865452ef1e4fea
2018-03-01 03:40:03 +00:00