Commit Graph

361 Commits

Author SHA1 Message Date
OpenDev Sysadmins 2490d1e2cc OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:48:33 +00:00
Mike Fedosin 5ef746ab9e Allow to specify a place where to unpack zip data
Now unpack_zip_archive_in_memory always unpacks data in
folder called 'content'. To add more flexibility we add
new parameter that defines where to unpack the data.

Change-Id: Ieb2f24776293a1890d755cddc458511b16f8eb82
2017-07-25 14:29:56 +00:00
Idan Narotzki dd61d8fc53 Add unpack_zip_archive_in_memory func.
The purpose of the func:
unpacking zip archive in Ram, without using hard drive
memory.
The func will be called from the upload hook of
both ns_package artifact and hooks_artifact.

Change-Id: Id4287a3f56475540f58e2728e0333612077e931a
(cherry picked from commit 6d8356b0fb)
2017-07-25 10:06:06 +00:00
OpenStack Proposal Bot b3d8db07ec Updated from global requirements
Change-Id: I28b0a6d5db8b0646599e1a7352e31dfdd112a82d
2017-07-03 16:13:59 +00:00
Idan 598b4d5bfa Fix RequestEntityTooLarge exception from read func
The RequestEntityTooLarge that are thrown from
read function (common/utils.py) will include
only the limit parameter. The param that
indicated the number of blob's bytes was
removed.
we cannot be sure that we will have
the number of bytes of the user's blob.

Change-Id: I57a586dae413519238acfd826dcb6316732f8e43
(cherry picked from commit 4d31dd29f7)
2017-06-29 11:39:25 +00:00
Idan 17d4be2511 Add info to RequestEntityTooLarge exception.
Add 2 parameters to RequestEntityTooLarge exception:
blob_length - The size of the blob
limit - maximum number of bytes the reader should allow

Change-Id: I32bd4381f2b7b20f2fe642ffc71821872108471c
(cherry picked from commit 25992053e4)
2017-06-27 09:37:34 +00:00
Mike Fedosin d6309b6b9b Disable request_id absence warning
Glare prints warnings into logs when request_id is
not specified in user context. But when the service
is used outside of OpenStack it's normal behavior
and logs confuse operators.

So, it's better to remove this warning and prevent
logs clogging with such warnings.

Change-Id: Ia588d5a2f067ae1333284b2def9b6612bc493cea
2017-06-26 09:25:07 +00:00
Mike Fedosin 7d1466991c Fix activation when glare has uploading blobs
Currently user can't activate his artifact if glare
uploads data for other artifacts in the system.
This code fixes this behavior.

Change-Id: Ice4666b0368b86c1722651576116b08d99f6f045
Closes-bug: #1699503
(cherry picked from commit 84c31bac4c)
2017-06-21 14:43:04 +00:00
Mike Fedosin 88745fa433 Update default branch in .gitreview file
Change-Id: Id3f0f1b0408ede0d877dd378cea7c1251b568eb8
2017-06-20 20:40:57 +03:00
Jenkins 768360d064 Merge "Add basic documentation about glare framework" 2017-06-20 12:44:36 +00:00
Mike Fedosin 0c92788318 Add basic documentation about glare framework
Change-Id: I92daa0b1293881cc0474e0b13cb0b99049281f8e
Co-Authored-By: Idan Narotzki <idan.narotzki@nokia.com>
2017-06-20 14:00:18 +03:00
Jenkins a06d0efb64 Merge "Add new common utils tests" 2017-06-20 09:36:13 +00:00
Mike Fedosin b9c863e351 Add new common utils tests
Change-Id: I38ac7accf5fd90e402072dc6074c3ca2f2d46509
2017-06-20 12:00:54 +03:00
Jenkins a1d6947c1e Merge "Use https for urls in the readme file" 2017-06-19 20:08:20 +00:00
Jenkins 870682389a Merge "Add unit tests for validators" 2017-06-19 16:01:50 +00:00
Jenkins bf0b201af6 Merge "Fix LargeBinary column type for mysql" 2017-06-19 15:43:44 +00:00
Mike Fedosin 2d4e597ae4 Fix LargeBinary column type for mysql
Currently sqlalchemy has a bug with mysql and LargeBinary type
fails to create LongBlob and creates Blob instead [1].

As a workaround it's recommended to explicitly specify column
type and size for mysql deployments [2].

[1] https://bitbucket.org/zzzeek/sqlalchemy/issues/3883/largebinary-type-fails-to-create-longblob
[2] https://stackoverflow.com/questions/43791725/sqlalchemy-how-to-make-a-longblob-column-in-mysql

Change-Id: I921a5ae86ced07ac0a04ba7e5dafcea07afebdb0
2017-06-19 15:00:01 +00:00
Mike Fedosin 1f7a4ffeda Add database downgrade migration tests
Change-Id: I7cab62ab22fc313ed84a220352caffd841595746
2017-06-19 17:30:13 +03:00
Christian Berendt e71d3d6d75 Use https for urls in the readme file
Change-Id: Icc25b7efc39ce05657678cc6bb3d1c27bd7b8e0f
2017-06-19 11:33:02 +02:00
Mike Fedosin d9915469ab Add download hook
Change-Id: I46644ced4e2aaf64392be70133538b0a48478772
2017-06-17 16:26:06 +03:00
Jenkins eb1d518c21 Merge "Add field's properties documentation." 2017-06-15 16:05:39 +00:00
Jenkins 61879f07d1 Merge "Fix devstack deployment" 2017-06-15 09:15:42 +00:00
Jenkins ead2d541e4 Merge "Fix creation of artifact with ':' in names." 2017-06-13 15:54:24 +00:00
Mike Fedosin e3a3adaae2 Fix creation of artifact with ':' in names.
Currently to prevent name/version colisions we look
for artifact with the same name/version before creation.
To do this we use simple 'list' operation. Unfortunately,
if name or version contains ':', it is considered as filter_op
delimeter and Glare returns 400 for these cases.

This code fixes this and adds explicit operator 'eq:' for each
value.

Change-Id: I7760019cdbe1f6e8c5bf1c3b22a3cf07392aff19
Closes-bug: #1697698
2017-06-13 17:47:09 +03:00
Jenkins 3fad1ae6e5 Merge "Add new artifact: Secret" 2017-06-13 13:01:13 +00:00
Mike Fedosin 5aba8880d1 Fix devstack deployment
This patch adds two improvements in devstack shell script:

1. Now we use uncommented yaml file for default policies.

2. configure_auth_token_middleware function is used to setup
keystone_authtoken middleware instead of custom implementation.

Closes-bug: #1697686

Change-Id: I5448486019c9ed20a4eb656d48985b5d4958848b
2017-06-13 12:20:14 +00:00
Idan c1a5447c6b Add new artifact: Secret
The purpose Secret artifact, is to enable the user to store
'secret' data such as: Private key, Certificate, Password, SSH keys Etc.

Change-Id: I89df9f7eba576a9086cdae4c4f3379caec2b8df7
2017-06-12 18:55:31 +00:00
Idan f630661cea Add field's properties documentation.
Change-Id: Ic2951378b8039945dab5c7521ac7a8b98ca3f7e2
2017-06-12 18:52:32 +00:00
Mike Fedosin e4c7031a38 Add unit tests for validators
This code adds unit tests for validators.

Besides, it fixes various issues in 'validators'
module, related to logging, json-schema generation,
dead code removing and many other things.

Finally, it implements a long waiting feature for
json-schema generation: now validator strings
are applied not only to basic fields, but also
to dict and list elements. This improves detalization
of artifact type schema.

Change-Id: Ifee4b355bc5b9db3d597920c6aeb047cfc59c6ab
2017-06-09 08:05:37 +00:00
Van Hung Pham 8ef98671cb Replace LOG.warn with LOG.warning
logging.warn is deprecated in Python 3.[1]
[1] https://docs.python.org/3/library/logging.html#logging.warning

Change-Id: Ic197980b926fbdcd5fdd1be9c0aea59b27738d54
2017-06-08 22:15:20 +07:00
Jenkins 9fe4385a74 Merge "Add missing unit tests" 2017-06-07 10:11:38 +00:00
Mike Fedosin b6e5633b7b Replace six.iteritems() with .items()
1.As mentioned in [1], we should avoid using six.iteritems to achieve
iterators. We can use dict.items instead, as it will return iterators
in PY3 as well. And dict.items/keys will more readable.
2.In py2, the performance about list should be negligible, see the
link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html

Change-Id: I9b1f1496375f5b5c6dbc5dadbb57d729680ff3fd
2017-06-07 12:25:41 +03:00
Jenkins 89fe91ce7a Merge "Rename 'attributes' to 'fields'" 2017-06-07 09:03:51 +00:00
Mike Fedosin 9682a0036f Add missing unit tests
This patch adds additional unit tests for updating, listing,
uploading and deleteing of artifacts.

Plus several tests were added for trusted auth middleware.

Finally some dead code was removed from the system.

Change-Id: Ia570d02251946b0a9c28b34678239109a125abcf
2017-06-06 16:06:53 +03:00
Mike Fedosin 2199963911 Rename 'attributes' to 'fields'
Currently there is a confusion in naming, because in
different parts of the code there are names: 'fields' and
'attributes', which represent the same things.

To avoid this, new convention is suggested:
1. Entities that define artifact structure are called
'fields'. Example of fields: 'id', 'version', 'tags',
'owner'.

2. There are two types of fields: basic and custom. Basic
are fields from Base artifact type and each artifact
type has them. Custom ones belong to specific artifact type
only.
Example of custom fields: 'disk_format' for images, 'package'
for murano_package.

3. Each field has properties. Example of field properties:
sortable, nullable, required_on_activate.

This patch renames all ocasions of attributes into fields.
For backward compatibility classes and modules can be
alternatevely accessed by their previous names.

Change-Id: Iec521112020f7cb7badd010bb21be130bf5825c0
2017-06-06 14:43:26 +03:00
Jenkins 7da6ebf6b2 Merge "Use assertIsNone instead of assertEqual(None, ***)" 2017-06-06 08:18:36 +00:00
Jenkins f6926bbd27 Merge "Fix html_last_updated_fmt for Python3" 2017-06-06 08:18:26 +00:00
lingyongxu 09338d876c Use assertIsNone instead of assertEqual(None, ***)
Following OpenStack Style Guidelines: [1]
Unit test assertions tend to give better messages for more
specific assertions. As a result, assertIsNone(xxx) is
preferred over assertEqual(None, xxx) and assertIs(xxx, None)

[1] http://docs.openstack.org/developer/hacking/#unit-tests-and-assertraises

Change-Id: I580dba2cc3c8ec80e080d6d0972ab342c0105943
2017-06-06 15:00:55 +08:00
Vu Cong Tuan ecae1938eb Fix html_last_updated_fmt for Python3
html_last_updated_fmt option is interpreted as a
byte string in python3, causing Sphinx build to break.
This patch makes it utf-8 string.

Changing Popen to .check_output because of 3 reasons:
1. check_output() will raise CalledProcessError if
the called process returns a non-zero return code.
2. For consistency with keystone [1] and cinder [2]
3. It makes the code look much better.

[1] https://review.openstack.org/#/c/457142/
[2] https://review.openstack.org/#/c/433081

Change-Id: I2da5d4cc708c06f7d03f2f2805f6340895c9f5ac
2017-06-06 09:49:35 +07:00
Jenkins 80ae6e2c90 Merge "Add Link management tests" 2017-06-05 14:59:13 +00:00
Jenkins f910db5cc6 Merge "Fix default filter and sort parameters for list" 2017-06-05 14:54:45 +00:00
Jenkins 0b982b3b12 Merge "Remove log translations" 2017-06-05 14:52:02 +00:00
Mike Fedosin d5af363595 Add Link management tests
Change-Id: Iaf25f6fe010cedf2bc5a397c2199ccf5f87af419
2017-06-01 22:14:25 +03:00
Mike Fedosin a44eedfdf6 Remove log translations
The i18n team has decided not to translate the logs because it seems
like it not very useful; operators prefer to have them in English so
that they can search for those strings on the internet.

Change-Id: Id9dea38dbfd15f12dd3f25a97829016a242277f4
2017-06-01 22:04:48 +03:00
Jenkins a3edb98e5f Merge "Deprecate 4-bytes unicode characters in sql-based dbs" 2017-06-01 18:40:32 +00:00
Jenkins 39a9469ba5 Merge "Add download unit tests" 2017-06-01 15:07:05 +00:00
Idan 610752fe35 Add download unit tests
Change-Id: I41a08901c0673182f0c82ee00e619e7be52ba4f0
2017-06-01 12:52:09 +00:00
Mike Fedosin 73c12cffa3 Fix default filter and sort parameters for list
Default sort=created_at:desc,id:asc
Before sort parameters direction was also considered, which is wrong.
So, defaults applied even if user wanted to sort by created_at but in asc
order:
sort=created_at:asc,created_at:desc,id:asc, but it should be just
sort=created_at:asc,id:asc.
This patch fixes it. Now default sort parameters are added only if sort
name not in the list, regardless of the direction.

For filters now it's checked that default filter not in the list.
For example: if user wanted to filter by status=neq:deleted, the same default
filter will be applied in addition anyway, i.e. there will be two equal
filters, one from user, one is default.
This patch fixes it. Now default filters only added if they are not already
in the list.

Change-Id: I1b23bd51d943fa2a489b948a5609fcced6ae79b1
2017-06-01 13:23:10 +03:00
Mike Fedosin ad2cac447e Add tests for adding custom location to blob
Change-Id: Id8ceec283174ede650836ea081228d406c0c74b5
2017-05-28 20:20:21 +00:00
Mike Fedosin 18b81f369e Close db connection after modifying operation is done
Change-Id: I3f23793fa68def56704bd4518ebf370072e0ad9c
2017-05-26 15:05:44 +03:00