Commit Graph

420 Commits

Author SHA1 Message Date
Tony Breeds 519b1b1b8e Retire Packaging Deb project repos
This commit is part of a series to retire the Packaging Deb
project. Step 2 is to remove all content from the project
repos, replacing it with a README notification where to find
ongoing work, and how to recover the repo if needed at some
future point (as in
https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project).

Change-Id: I999443d9fd240490df5a2b232e6c9c6b55a04a34
2017-09-12 15:38:36 -06:00
Jenkins 9b9592ea1e Merge "Create new policy for downloading deactivated artifacts" 2017-07-25 15:37:35 +00:00
Jenkins eef2f35190 Merge "Raise NotFound if blob data is not found" 2017-07-25 15:35:49 +00:00
Mike Fedosin dfeab5255c Do not return second parameter in unpack_zip_archive_in_memory
Change-Id: Ic2d54f7e52d1fcfb0041384b281be8053b23f477
2017-07-25 18:12:50 +03:00
Mike Fedosin dbdd286346 Start using unpack_zip_archive_in_memory in hooks artifact
Change-Id: I4a15581600f9163397ec5442cf8705b7afc807de
2017-07-25 17:36:18 +03:00
Mike Fedosin 0172abb9ae Allow to specify a place where to unpack zip data
Now unpack_zip_archive_in_memory always unpacks data in
folder called 'content'. To add more flexibility we add
new parameter that defines where to unpack the data.

Change-Id: Ieb2f24776293a1890d755cddc458511b16f8eb82
2017-07-25 17:18:40 +03:00
Idan Narotzki 9337f11bca Add warning to unpack_zip_archive_in_memory function.
Change-Id: Ia8dfd95df82a5ba7f72426be65f9521ed161d5a8
2017-07-25 10:34:36 +00:00
Idan Narotzki 6d8356b0fb Add unpack_zip_archive_in_memory func.
The purpose of the func:
unpacking zip archive in Ram, without using hard drive
memory.
The func will be called from the upload hook of
both ns_package artifact and hooks_artifact.

Change-Id: Id4287a3f56475540f58e2728e0333612077e931a
2017-07-25 08:20:27 +00:00
Jenkins 9d80949fb3 Merge "Stop using deprecated 'message' attribute in Exception" 2017-07-24 15:28:33 +00:00
Idan Narotzki 0d8aad964c Fix validate_upload function in hooks_artifact.
when in_memory_proccessing param is true
we want to ensure that upload will be
executed in memory.

Change-Id: Ifaa5fbd0444f3ff626ae806b2aa567151490a22e
2017-07-24 13:10:22 +00:00
bhavani.cr 56d9a6bd85 Stop using deprecated 'message' attribute in Exception
The 'message' attribute has been deprecated and removed
from Python3.
For more details, please check:
https://www.python.org/dev/peps/pep-0352/

Change-Id: I5d801bec8ada2303457c8a87c1965b146f60da81
2017-07-24 15:08:17 +05:30
Jenkins dc1055f951 Merge "Add devstack hook stubs" 2017-07-23 15:35:57 +00:00
OpenStack Proposal Bot c334d62f0f Updated from global requirements
Change-Id: Ie23b14246d0409ef81d40c8294bea5f30d7986bd
2017-07-23 13:41:34 +00:00
Mike Fedosin 139d735749 Add devstack hook stubs
Change-Id: I4cab55be457dcd7f6509940297c9f289e3785463
2017-07-23 12:13:43 +03:00
OpenStack Proposal Bot f27b990d25 Updated from global requirements
Change-Id: Ic1360d3bb444a930da8444786c46a2d50f036564
2017-07-22 16:28:16 +00:00
Jenkins 47f0bd5476 Merge "Allow to specify custom modules and artifact types in devstack" 2017-07-21 18:58:33 +00:00
Mike Fedosin 8faf9090d9 Create new policy for downloading deactivated artifacts
Now there is a check in the engine, that verifies that
only admins can download deactivated artifacts.
This is not recommended and it's much wiser to create a
policy to check this.

Change-Id: I26fb4be064d2860ace5d7423d3103f2fa33360cd
2017-07-20 21:26:36 +03:00
Mike Fedosin b67f0538e4 Raise NotFound if blob data is not found
This is api change, that raises NotFound(404) instead of
BadRequest(400) if blob data is not found for user request.

Also small refactoring implemented.

Change-Id: I5ebe5f5611540b886ee82b22376408b916d28759
2017-07-20 20:57:02 +03:00
Mike Fedosin 4c0b88e8eb Implement in-memory data processing for upload hooks
This code adds a possibility to unpack zip archives
in ram for 'hooks_artifact' type.

Change-Id: I40417d96593f6545f205e51af40535151c2aea96
2017-07-12 17:07:12 +03:00
Mike Fedosin 5da4af347c Add validation hooks tests
Change-Id: I44260f412c81a19a71f5a854f355dbcc8d8f9d56
2017-07-12 12:42:24 +03:00
Mike Fedosin 6b67968cbc Allow to specify custom modules and artifact types in devstack
This patch adds a possibility to set third-party modules
and enable additional artifact types during devstack deployment.

Change-Id: I84fcf741f20e061c73515c979d0fe89da460f467
2017-07-11 19:01:39 +03:00
Mike Fedosin 14a0fb1c90 Implement per artifact type configurations
Now we can specify some parameters per artifact
type in section [<type_name], like [images] or
[heat_templates] in glare.conf file.

It increases a flexibility of the system and
allows to tune the system more precisely.

Change-Id: I236716f88409169732661ade12168c560e2be112
2017-07-06 15:32:13 +03:00
Mike Fedosin 991dcdca21 Massive refactoring
All the tricks are done by professionals.
Please, don't try this at home!

Change-Id: I9000065fde57043a4e0de3a409505cf66ac3360d
2017-07-06 11:56:51 +00:00
Mike Fedosin 0c6cef3d47 Rename methods from functional testing infrastructure
To prevent code duplication we combined 3 different functions
in one - 'admin_action'

Change-Id: I62b366e38ee3aadd42272e144171d39252f8151f
2017-07-05 15:12:48 +00:00
Jenkins f119a615a3 Merge "Rework link validation" 2017-07-05 13:11:23 +00:00
Mike Fedosin c69a3eba6d Rework link validation
Currently links are validating in two places: in 'update'
method of Base artifact type and in the Link type 'coerce' method.
This looks ugly and it's better to combine all the validations
in one place, where they supposed to be - in the 'coerce' method.

Change-Id: I7cf74c111f7a41fdda2925f232718693110c26de
2017-07-05 11:30:23 +00:00
iswarya_vakati 979233cfb1 Replaces uuid.uuid4 with uuidutils.generate_uuid()
Change-Id: I27f8babfc8a995f3433cc1f94526b87bd70a4757
2017-07-05 11:29:26 +00:00
Jenkins 514aeb9e6d Merge "Add changes to Base AT structure" 2017-07-05 08:57:59 +00:00
Jenkins fb8d17bc6b Merge "Don't add MaxStrLen validator if AllowedValues applied" 2017-07-05 08:57:53 +00:00
Jenkins 81745d2933 Merge "Enable H904 check" 2017-07-05 07:59:00 +00:00
Jenkins 5dc189d4e5 Merge "Fix active status webapi documentation" 2017-07-05 07:43:09 +00:00
Ngo Quoc Cuong 4e615d6da6 Enable H904 check
H904  String interpolation should be delayed to be handled by the
logging code, rather than being done at the point of the logging call.
Use ',' instead of '%'.

See: https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html#adding-variables-to-log-messages

Change-Id: I66ea69fe877fa57a54e06c22221cdd1f3414ef16
Related-Bug: #1596829
2017-07-05 03:39:22 -04:00
Idan 27fddd5c51 Fix active status webapi documentation
Change-Id: Ieb9daa51760fe94a522031a421b03a012d72062d
2017-07-05 07:32:11 +00:00
Jenkins 714e77bcad Merge "Update log translation hacking rule" 2017-07-04 18:00:41 +00:00
Mike Fedosin b0c2a93aa5 Add changes to Base AT structure
Fields 'status', 'visibility', 'updated_at' should be
declared as mutable (because their values can be changed
after activation).

Name must contain at least 1 character and cannot be an
empty string.

Change-Id: Iafdce6f24fd20d0db61a6f022ff2534b4d01e306
2017-07-04 20:38:56 +03:00
Mike Fedosin 405ae23beb Don't add MaxStrLen validator if AllowedValues applied
Currently MaxStrLen is added by default to all string
fields. But it's not necessary if we've already specified
a list of allowed values.

Change-Id: I09b101a42089792220ec8814ce153cf20bf0aaf3
2017-07-04 20:38:48 +03:00
Ngo Quoc Cuong 7539e4510b Update log translation hacking rule
Starting with the Pike series, OpenStack no longer supports log
translation.
Update hacking rule to prevent log translation in all log level instead
of only debug level.

Change-Id: I00d1930ba2bb0e0278e0d33147e4d15c59615e74
2017-07-04 03:51:57 -04:00
Mike Fedosin 4b2c4260dc Fix calculation of the folder size
If folder contains blobs with external links
their sizes will be None, which leads to errors,
like

TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

Closes-bug: #1702113

Change-Id: I0bf431b1cded7848fe17fa69aad1e34ec11eeb87
2017-07-03 17:45:02 +03:00
Jenkins 9b50f5d839 Merge "Add max_folder_size" 2017-07-03 14:13:11 +00:00
Idan 010726cd47 Add max_folder_size
while max_blob_size limits one specific blob,
max_folder_size will limit the whole folder size.
i.e. if max_folder_size = 100Mb, you can upload 100
blobs with size 1Mb, or a big one with size 100Mb.

Default max folder size is 2550 megabytes.

Change-Id: I845c493db14d5d1884ace29cbcf6b5707a48250a
2017-07-03 13:55:27 +00:00
bhavani.cr fc3d0657be Update .gitignore to ignore .eggs
because egg* already ignores egg-info [1]

[1] https://review.openstack.org/#/c/477476/

Change-Id: I628c0cd404dfd9b97f5d3829a6b032a8bc952fd3
2017-07-03 18:16:11 +05:30
bhavani.cr f1f1d8a428 Remove log translations
Log messages are no longer being translated. This removes all use of the
_LE, _LI, and _LW translation markers to simplify logging and to avoid
confusion with new contributions.

See:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html

Change-Id: I098d7e44c8cfafe2d9f21dd51ea942e9a2833752
2017-07-03 17:50:01 +05:30
Jenkins e8e182e687 Merge "Implement scrubber util" 2017-07-03 07:30:20 +00:00
Mike Fedosin fb60b14159 Implement scrubber util
Scrubber is a utility, that cleans up artifacts that have been marked
as deleted, when delayed_delete mechanism is activated. Enabling
delayed_delete option means that after delete API call, the artifact
will change its status to deleted, but it's data will not be deleted
immediately. Afterwards (according scrubber settings), scrubber will
detect the artifacts with 'deleted' statuses and delete them completely.

The scrubber can be run as a background periodically task or once as
a standalone tool.

Implements-blueprint: scrubber

Change-Id: Icada538521be927bf8e42512fa2d0be9c97c73ff
2017-07-03 07:14:22 +00:00
Jenkins 81e259fac9 Merge "Allow to install python-glareclient from pypi in devstack" 2017-06-29 11:47:41 +00:00
Mike Fedosin f27d396be5 Allow to install python-glareclient from pypi in devstack
Change-Id: I8532fb7c709349868c1160a7e3dcbb5fe3c631dc
2017-06-27 23:41:15 +03:00
OpenStack Proposal Bot 9dd190ccf9 Updated from global requirements
Change-Id: Ic64d90998fb110266ee239f3e96d8c7e87db39db
2017-06-27 17:42:33 +00:00
Jenkins 5eb44ecdb3 Merge "Fix RequestEntityTooLarge exception from read func" 2017-06-27 16:47:05 +00:00
Idan 4d31dd29f7 Fix RequestEntityTooLarge exception from read func
The RequestEntityTooLarge that are thrown from
read function (common/utils.py) will include
only the limit parameter. The param that
indicated the number of blob's bytes was
removed.
we cannot be sure that we will have
the number of bytes of the user's blob.

Change-Id: I57a586dae413519238acfd826dcb6316732f8e43
2017-06-27 15:20:27 +00:00
Mike Fedosin 144ca45efb Change "rm" command path in tox.ini
tox command shows WARNING message as follow.
"WARNING:test command found but not installed in testenv"
If we use "find" command in tox.ini, we should use the full path.
This patch changes fix command path and remove the Warning message.

Change-Id: I6b8f487afc84ef5ecb5e0e4ec865ab0bb2cf2688
2017-06-27 18:15:12 +03:00