Commit Graph

69 Commits

Author SHA1 Message Date
xuanyandong 44f22cf2a5 Remove six
Replace the following items with Python 3 style code.

- six.PY2
- six.text_type
- six.string_type
- six.binary_type
- six.iterkeys
- six.moves
- six.itervalues

Implements: blueprint six-removal

Change-Id: I6195ceeeed2ebe1586973eaeef7fa9f29698fec2
2022-07-05 17:37:44 +08:00
Andreas Jaeger 80bd123e50 Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found.

Update local hacking checks for new flake8.

Remove hacking and friends from lower-constraints, it's not needed
there.

Change-Id: I48fcd2521d99d0b77b976e87cc638fe5ac051315
2020-03-31 17:02:37 +02:00
jacky06 c27a8c4eae Update json module to jsonutils
oslo project provide jsonutils, and ec2api use it in many place[1],
this PS to update the remained json module to oslo jsonutils for consistency.

[1]: https://github.com/openstack/ec2-api/search?q=jsonutils&unscoped_q=jsonutils

Change-Id: I46e7cfe55bd5cfbca90177bc4a970c726872bab4
2019-09-13 02:00:39 +00:00
chenghuiyu c65cb663de Fix usage of isotime
The commit I34b12b96de3ea21beaf935ed8a9f6bae2fe0d0bc and
Ib384ae8130dcc6cbd47a837d11ca171ce02ef29e introduced the
deprecated oslo_utils.timeutils.isotime() is deprecated
as of 1.6.

The deprecation message says to use the datetime.datetime.isoformat()
instead, but the format of the string generated by isoformat isn't
the same as the format of the string generated by isotime. The string
is used in tokens and other public APIs and we can't change it
without potentially breaking clients.

So the workaround is to copy the current implementation from
oslo_utils.timeutils.isotime() to ec2api.api.ec2utils.

For more informations:
https://docs.openstack.org/oslo.utils/latest/reference/timeutils.html

Change-Id: Id62fb53264b04a7ea6ae3035a129353c5cfa040a
Closes-Bug: #1461251
2018-06-29 16:00:40 +03:00
Ngo Quoc Cuong 90ed02d682 Delete log translation functions and add hacking rule
The i18n team has decided not to translate the logs because it seems
like it not very useful; operators prefer to have them in English so
that they can search for those strings on the internet.

Since we have removed log translations completely, we should add hacking
rule to prevent future mistakes.

Change-Id: Ia7524308ef2675f8d41ac80b37dfc7e3787efd90
2017-07-03 04:14:44 -04:00
Ngo Quoc Cuong 335a0eb4ad Remove log translations
The i18n team has decided not to translate the logs because it seems
like it not very useful; operators prefer to have them in English so
that they can search for those strings on the internet.

See http://lists.openstack.org/pipermail/openstack-dev/2017-March/thread.html#113365.

Change-Id: I4418a388934ef58acb70c7f637b0236a6ccc7fe6
Closes-Bug: #1682101
2017-05-11 13:43:05 +07:00
Jenkins ceaca8338f Merge "Replace six.iteritems() with .items()" 2017-04-20 15:52:26 +00:00
M V P Nitesh 1cde1a7fe8 Replace six.iteritems() with .items()
1.As mentioned in [1], we should avoid using six.iteritems to achieve
iterators. We can use dict.items instead, as it will return iterators
in PY3 as well. And dict.items/keys will more readable.
2.In py2, the performance about list should be negligible, see the
link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html

Change-Id: Ib7ad61bac89e14561719434085b63cbad7e9a9be
2017-04-20 09:42:45 +00:00
Andrey Pavlov 6d30c69895 use glanceclient version '2'. fix keystone error message.
current devstack deploys glance version '2' only.
use it.

messages in keystone's exceptions was changed.
fix unit tests that checks it.

Change-Id: I7ed1f0ff518efa374a5e3b693c5785958c77340d
2017-04-19 13:10:40 +03:00
tikitavi 0ee47eb58e small fixes to default vpc implementation
Change-Id: I9c827ceec0ab6a2df0b221bc989a3c211c000200
2017-02-27 15:34:09 +03:00
tikitavi 61daf6a80f describe-security-groups by name in default VPC mode describes
security groups in default vpc only

changed check_and_create_default_vpc function to return default vpc

fix unit test of describe_security_groups

Change-Id: I50bad4a6a7152eb03599e3985a43162c81500049
2017-02-19 14:16:28 +03:00
tikitavi df67b88930 add instance in default-vpc by security-group-name
or private-ip-address

add unit test

Change-Id: Iff5421ca84d5d5f6c2f87c9aed636ead6aadef2f
2017-02-09 23:07:57 +03:00
tikitavi 449f7c741c create default vpc
check if default vpc exists, create if not,
use it in describe_vpcs function,
fix unit test
add unit tests for default vpc creation

Change-Id: I32411c55a9877854ef11e52f082496bb6656291f
2016-12-29 19:14:47 +03:00
Andrey Pavlov da6f97f291 rework certs stuff
remove ca_certs using for incoming requests
do small refactoring for keystone client using

Change-Id: I83dbb71248835cfc361eca691647deaa99023c8a
2015-11-12 11:19:06 +03:00
Andrey Pavlov f34d64747c py34: fix string conversion
fix working with strings for metadata server,
for some instance properties and for errors.
do some magic with types/encoding.

Change-Id: I2446fc48b51dd903a719aa4dbb1a5556af19e5df
2015-09-13 23:56:53 +03:00
Jenkins 8e6c1d2202 Merge "Create an image in a separate thread" 2015-09-11 09:52:30 +00:00
Feodor Tersin 90cbc6030f Create an image in a separate thread
Creating of an image by a running instance stops the instance. This
takes a time, and a client can resend the request due to timeout. The
resended request fails because instance is already not in appropriate
state (it's stoping, stopped, or scheduling at the moment).

Thus ec2api should make such images asynchronously. It should create a
DB item to reserve the image id, return it to a caller, and then stop
the instance and do other things. While image is asynchronously created
and no Glance image exists, ec2api should return the image in responses
for describe operation. Describe operation should return the image with
and without selective filter by the image id passed in.

Also it should consider a race when asynchronous task has not finished,
but Glance image is already created, and another client describe images.
In that case ec2api should not create a new db item, but reuse existed
one.

If the creation failed, ec2api should expose the image as failed and
allow to delete it.

Unit tests for auto_update_db and handle_unpaired_item will be added in
a separate review, because some refactoring is required for that.

Change-Id: Iba4ade5000bea4af35d97abd1e708bbbc57f372a
2015-09-10 20:31:53 +03:00
Andrey Pavlov f51ca94079 py34: fix iteritems/itervalues using
Change-Id: Id3e3e814448be22184270db57ff43f5f83c4d1c1
2015-09-09 17:16:09 +03:00
Feodor Tersin 07a7bd4b0c Fix comments in ec2utils.get_os_image_mappings
Change-Id: I6b2890478c05672706db067dd10d6369ba09294f
2015-08-24 21:53:59 +03:00
Feodor Tersin f938f1bd32 Use lxml to render EC2 response
Currently etree is a preferred library to work with xml. lxml
implementation is used for doing that in OpenStack.

Change-Id: I93ce6d1a4e8e59c70e74aa6634cdc8bc67cc5914
2015-08-14 10:44:47 +03:00
Feodor Tersin 2e40c234f8 Remove project_id from db.add_item interface
Since there is no part of code which adds an db item with other project,
this backdoor is not needed.

This patch removes this argument and its proxying by util functions.

This reverts commit b51eb4f350.

Change-Id: I70b5d2e7246152ac2b1014f51f89f44862acab44
2015-08-13 18:46:09 +03:00
Feodor Tersin 3adf3a1f50 Use id mapping in get_os_image
A public alien image can has id mapping only, but not a regular db item.
This leads to unexpected InvalidAMIID.NotFound error on certain
requests. An added test checks one such case.

Change-Id: Ie6fd0f877cbf4bdcf8b44cec82dc9cc8fb24918a
2015-08-13 17:27:30 +03:00
Feodor Tersin 16d9266c78 Extend db_api.get_items_ids to search by ec2 ids
db_api.get_items_ids return id mappings (id-os_id pairs) for specified
filters. Currently it doesn't allow filters by ec2 ids. This reduces its
scope of usage.

Change-Id: Ib855429dc2fc650ef7ecff07c637b8d4b45a07db
2015-07-28 18:39:56 +03:00
Feodor Tersin 5356a0ccc1 Move image root device name getter function to ec2utils
These functions are useful for various ec2api modules.

Change-Id: I319f21ced5b645c019497d54b0101b2a09a793de
2015-07-20 17:57:45 +03:00
Feodor Tersin 9be22b8ac8 Introduce a function to parse image device mappings
Images have two kinds of device mappings attribute: virtual (mappings)
and block (block_device_mapping). The last one may be stored in two
different formats (legacy and bdm v2). Device descriptions are merged
from this two attributes to a single list used for instance boot
operation.

We should have a function which parses and merges image device mappings
and returns the result device list of a certain format to use it in
various parts of ec2api.

This patch introduce the function which implements the requirements
above and returns device mappings in bdm v2 format.

Change-Id: I7ad8673dcb6ddba11c6e00e22aa68a35ea1ec0fd
2015-07-17 11:22:03 +03:00
Feodor Tersin 516d4a1ce5 Move strip/prepend dev functions to ec2utils
These functions are useful for various ec2api modules but do not have
any instance specificity.

Change-Id: Ib4539986ce3019d47d56817ec5b1bcf056074d72
2015-07-16 11:38:49 +03:00
Feodor Tersin 25c4e40f1b Rename dict_to_lxml to dict_to_xml
We're not going to have many libraries to work with xml.

Change-Id: I42f97b4fa7f1b18e14a897ccb19850586d2d0334
2015-05-28 19:41:39 +03:00
Feodor Tersin 76c7d45442 Move dict to xml conversion to ec2utils
This allows to use the conversion from other code

Change-Id: Iccc886c3a899f65cad2a097712b312ac18143e4c
2015-05-27 16:43:14 +03:00
Feodor Tersin 020e6c0ba1 Implement VPN connection storage
Change-Id: I3e73dca9ed4084a8d864b7d7495ac6321b623f10
2015-05-25 17:48:38 +03:00
Feodor Tersin a7aa7cb47b Attach router to external network for VPN gateway
Change-Id: Ie0bd6f1f38c898c26a16256e7d3f38c6c8bb00c2
2015-05-25 13:19:25 +03:00
Feodor Tersin 6843471ed1 Implement VPN gateways
Change-Id: I6d1d88c779321e875c16943a4aa162457543267c
2015-05-16 17:28:53 +03:00
Feodor Tersin 54690d1095 Implement customer gateway
Change-Id: I0e090f1d8dca70235615242f8993081c71ae6615
2015-05-16 17:28:48 +03:00
Jenkins a0b412ca70 Merge "Handle invalid setting of public network" 2015-04-22 10:40:24 +00:00
Andrey Pavlov b51eb4f350 fix bugs with project id in DB for public objects
- public ami images in DB had incorrect project id
- public aki,ari images had incorrect project id

Change-Id: I752d0f98fed150cfa7c5de792d50b091e00afae1
2015-04-21 20:06:59 +03:00
Andrey Pavlov b7d53fe5e6 add fixes to images
1. extend modify_image_attributes
2. fix create_image
3. add reset_image_attribute
4. add images' tests

Change-Id: I0e52dbe46e0a0f3e6257365014f9b35bb4a21ad9
2015-04-21 14:52:18 +03:00
Feodor Tersin dacb864ee5 Handle invalid setting of public network
Log error and raise an EC2-like exception.

Change-Id: I6190f68fd82eee1963c3a393b8134afe0d80dfc6
2015-04-20 10:30:51 +03:00
Feodor Tersin 80655cb362 Fix and rename get_item_ids
1 Rename to get_items_ids
2 Fix a bug of filtering
3 Add a test on it
4 Add a test on ec2utils.os_id_to_ec2_id

Change-Id: I0ae4b0d2cd937ffa810f7cea3d818a72e3fac845
2015-04-13 11:09:35 +03:00
Feodor Tersin 9e6e4f6542 Simplify usage of db_api.get_items_by_ids
Change-Id: I4053cfb5f1198dfee316e907ac64b9844199d83b
2015-03-01 00:46:25 +03:00
Jenkins d546e73253 Merge "Simplify usage of db_api.get_item_by_id" 2015-02-27 10:00:31 +00:00
Jenkins dc68a1b35e Merge "Simplify usage of ec2utils.get_db_item" 2015-02-26 23:17:44 +00:00
Feodor Tersin dcaf655611 Simplify usage of db_api.get_item_by_id
Change-Id: I7f839a2f845dfef707b9e6939b02e97e50f67927
2015-02-26 22:33:47 +03:00
Andrey Pavlov 67eae1069d follow oslo
move to oslo.db and some other oslo libraries

Change-Id: I57e4f01bd46339cdfe3ed2e4748eb1e923cea584
2015-02-26 22:04:09 +03:00
Feodor Tersin 2de2b74152 Simplify usage of ec2utils.get_db_item
At the moment we cann't completely get rid of validation of a kind of
an object ID, because common parameter validator doesn't validate
complex parameters. So this validation is still used in several cases.

Change-Id: Ibbe73a74306f2359f6554fcc8ef7bc31d356c167
2015-02-26 18:49:00 +03:00
Feodor Tersin 49c721f8c7 Use timeutils from oslo.utils instead of oslo incubator
Change-Id: I4ad485df22bb3779f855b17b74c012f380f12d8e
2015-02-26 13:38:39 +03:00
Feodor Tersin 345e89be51 Fix translation of an image id in register image
Change-Id: Ib2879bc51932e62c125d0994b1a97eb41e005e6b
2015-02-26 10:59:29 +03:00
Feodor Tersin d1cbe05cf7 Test for kernel and ramdisk images
Change-Id: If6a650190f2672a450c39fd9275290b03b47d6be
2015-01-22 17:35:01 +04:00
Feodor Tersin aaba8da605 Implement tests for tags
Change-Id: I581ab5868ae45d9ac1db4ac05cf463928a0c34e3
2015-01-21 12:43:31 +04:00
Feodor Tersin c748453028 Remove Glance to EC2 ID Nova mapping
Change-Id: I251597542805bd2ed2641041da7a65cd954a6233
2014-12-30 17:06:06 +04:00
Andrey Pavlov 4e508a1841 remove access to nova's instance id mapping table
Change-Id: I04475a22067d6892632e0e3b608b4bd1e789ab12
2014-12-30 08:53:50 +03:00
alevine fa1397fe10 Reworked initial parameters validation mechanism
Change-Id: Icf9be8a9d1a70365be21c40e684a3184095947c2
2014-12-29 12:36:33 +04:00