Commit Graph

483 Commits

Author SHA1 Message Date
OpenDev Sysadmins e795f8cc25 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:50:09 +00:00
Ian Wienand 1ec6ea0b40 Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I095bf012d264025f84702140cb96f10428e237fb
2019-03-24 20:33:41 +00:00
Andreas Jaeger da21019109 Import legacy-glare-dsvm
Import legacy-glare-dsvm job and rename to glare-devstack following
naming policy.

Change-Id: I74aa695df50887d99ccf2b0348f1912a90e94725
(cherry picked from commit ac4eb163bd)
(cherry picked from commit d48d8439cf)
2018-08-25 18:55:39 +02:00
Mike Fedosin 9d30f3d709 Disable access to artifact with different artifact type
Now it's possible to get artifact info by providing
different artifact type.
For example, if artifact art_1 has type images, then users can
access it by url /artifacts/heat_templates/art_1 and any other
enabled artifact type.
For sure it should be accessed only by its own type, or by
metatype like 'all'. In all other cases 404 code should be returned.

Change-Id: I6a6bac273572671ae50528b44fb9c39503851b87
Closes-bug: #1743969
(cherry picked from commit c2cbdf076e)
2018-02-06 16:51:31 +00:00
Mike Fedosin 0183c47f50 Fix BlobIterator
Now we do not update index during iteration which leads
to the fact that BlobIterator returns incorrect data if
a file size exceeds chunk size (default 65536 bytes)

Change-Id: Id2b5f9efe29518106e097d237827ec481f22127b
(cherry picked from commit d880fce89e)
2018-01-12 17:00:48 +00:00
Idan Narotzki e28355bb9f Make keyclock auth support HTTP 400.
Make keyclock_auth to return exception.BadRequest
instead of exception.GlareException.

Keyclock may return 400 Http status For some cases.

In that case glare throws:
"code": 500, "error": {"message": "{\"error\":\"invalid_request\",
\"error_description\":\"User session not found\"}"

This change came to make glare reflect that this
is a token issue.

Change-Id: I084f05754d25722039039280826dc5dca4d58b95
2017-11-12 14:33:33 +00:00
Zuul 304ca08c11 Merge "Allow to define format for 'all' metatype" into stable/pike 2017-10-20 19:30:34 +00:00
Mike Fedosin 194a379f7e Allow to define format for 'all' metatype
This patch add new class method 'format_all' that can
be used to change the output format for 'all' metatype.

Change-Id: I67d82d921c5f5a9bad59384f11ed66f4fcecf7a9
(cherry picked from commit 3f1376552a)
2017-10-18 14:10:05 +00:00
Mike Fedosin 1e31fe877f Optimize 'get_artifact_type' from ArtifactRegistry
Now to get an artifact type object by its name we iterate
by all available classes and compare their names. It leads
to complexity O(n).

It's better to generate a dict of available types and get
them directly from there with complexity O(1).

Change-Id: Icc13cd6f844bc93c8d5a80ba963068cde3ef133e
(cherry picked from commit 7ac7bb092c)
2017-10-18 14:09:06 +00:00
Mike Fedosin d18cbc18ea Optimize quota calculations when they are disabled
Currently glare calculates the amount of data and artifacts
even if the related quotas are disabled (value -1 is set).
It's not optimal, because additional requests to db are
required.

This code changes this and implements new behaviour when
requsts to bd are performed only if quotas are enabled.

Also it disables quotas by default.

Change-Id: Ie297aa1bc6f2be989bcc29353c8fec0101cf2e5b
(cherry picked from commit 8796021ca6)
2017-10-09 13:28:42 +00:00
Mike Fedosin 8df33b01dc Allow to run tests in offline mode
Currently for several unit and functional tests it is
required to download data from external source, which
prevents their execution in offline mode.

This commit eliminates the drawback.

Change-Id: I35bfe8fba0a4d332c132a718b1845b3a09d6f68b
Closes-bug: #1721290
(cherry picked from commit 80be8dfbc2)
2017-10-06 20:56:57 +00:00
Mike Fedosin 5c0139f2bc Revert scoped lock format
Recently scoped lock format was mistakenly changed
from "type_name:name:version" to "owner". New format
prevents simultaneuos modifications of different 
artifacts by one tenant, because it locks the whole
tenant scope, not just a single artifact.

For this reason it's suggested to revert the format as
it was before.

Closes-bug: #1720139

Change-Id: I9dc26da85f424a4ced1ca232d1f33c548b602f0f
(cherry picked from commit d2ce390ca8)
2017-10-02 10:47:13 +00:00
Mike Fedosin 1f798c3116 Allow eventlet hub to use monotonic clock
Import oslo_service first, so that it makes eventlet
hub use a monotonic clock to avoid issues with drifts
of system time.

Change-Id: Ia260a5eb52bacdf202e5204d5b7b2bd281174a4a
Closes-bug: #1719178
Related-bug: #1510234
2017-09-24 12:49:48 +00:00
Mike Fedosin 70a559a48a Fix tempest plugin
This code fixes importing configured admin credentials,
adds a help message to 'catalog_type' config parameter
and removes unused custom tempest.conf

Change-Id: I60ba1852d3686141132ae11362f51bf65ddfc618
Closes-bug: #1718669
2017-09-21 19:47:55 +00:00
OpenStack Proposal Bot c3939469be Updated from global requirements
Change-Id: Ied7ceb9be52f8df39f2e18d60045eb54ecd7d6aa
2017-09-21 00:57:33 +00:00
Mike Fedosin 2a2c7f432e Optimize zip unpacking
Now each file in zip archive requires 2 or 3 calls to db:
1. initialize blob instanse, 2.(optional) upload data if
database store is used, 3. activate blob instance
So, for 100 files there are 300 requests, and complexity
is O(n).

With this commit we perform requests for all files in batch
mode: 1. initialize all blob instances, 2.(optional) upload
all data if database store is used, 3.activate all blob
instances.
So, for any number of files in archive there will be only 2 or
3 requests to db. Complexity is O(1).

Change-Id: I87ea9d1f87dc6519956808a1d1e764f16e6dc611
2017-09-20 15:48:54 +00:00
Bernhard M. Wiedemann 32b91f3a0e Avoid restarting a child when terminating
When sending a SIGTERM to the main glare-api process, it was sending a SIGTERM to its children but then also re-spawning its first dead child.

Related-bug: #1714240

Change-Id: Ibef426c198d287bbdac4e764fd654edba4b7c2d7
2017-09-19 13:40:49 +02:00
Jenkins e93e29bb50 Merge "Run upload hooks in separate threads" 2017-09-11 20:24:50 +00:00
Jenkins df81a6ddaf Merge "Fix some reST field lists in docstrings" 2017-09-08 09:45:51 +00:00
lingyongxu 05e093efbe Fix some reST field lists in docstrings
Probably the most common format for documenting arguments is reST field
lists [1]. This change updates some docstrings to comply with the field
lists syntax.

[1] http://sphinx-doc.org/domains.html#info-field-lists

Change-Id: I54ac5a1cff963f301ac1536887f7f0866ada0d22
2017-09-08 02:11:30 +00:00
Jenkins 77c31fa285 Merge "Enable Glare API 1.1" 2017-09-07 19:39:37 +00:00
Mike Fedosin f309a494aa Run upload hooks in separate threads
To prevent possible freezes in upload hook due to cpu intensive tasks
we should execute it in a separate thread.

Change-Id: Id3556d0d6f1973c7b9755428d2587822beddd70f
2017-09-07 20:54:16 +03:00
Mike Fedosin c1ed166a38 Update logging messages
Change-Id: I33be28339b0f86472aafb85f53c8adb17c9ceece
2017-09-05 23:23:22 +03:00
Jenkins ef89cae691 Merge "Improve engine code" 2017-09-05 14:12:30 +00:00
Jenkins 10db41bf4f Merge "Enable tempest tests for Glare" 2017-09-04 17:23:34 +00:00
Mike Fedosin c028791852 Enable tempest tests for Glare
Change-Id: Ia4d8014b04c5a39b59f6af8f87e9195cd93e5e80
2017-09-04 19:03:36 +03:00
Mike Fedosin 3330ca04d7 Improve engine code
This patches optimizes engine code, adds docstrings
and fixes typos.

Change-Id: Ifee9acaffba9efa8ac8371defba73d4f8e0dac0a
2017-09-04 16:10:03 +03:00
Mike Fedosin b93aea2e3a Rework glare hooks
This patch adds pre and post hooks for all artifact modifying operations:
create, update, activate, deactivate, reactivate, publish, upload,
add_location, download, delete.
Example: 'pre_create_hook' and 'post_create_hook'.

Previous hook calls with 'validate_' prefix are removed except
'validate_upload' which is marked as depricated.

Change-Id: I746ceaacd357e332790c84ffe50e712cfa6cecda
2017-08-31 11:27:41 +03:00
Mike Fedosin 408e77dc5c Extend blob key length to satisfy db model
Change-Id: Ia16d99bf17c17753080b8a17a947dfdff47d5a1d
2017-08-30 10:48:18 +03:00
Jenkins 9dd17acea2 Merge "Use '_prepare_json_response' to serialize list response" 2017-08-29 15:04:04 +00:00
Jenkins 7780a20d0f Merge "Add Regex string validator" 2017-08-28 08:25:21 +00:00
Mike Fedosin 2a71c7888a Enable Glare API 1.1
This commit marks 1.0 API as stable and 1.1 as experimental

Change-Id: Idd5644a6f338656d4b4ca59ce62f26860e9d20a3
2017-08-28 10:58:39 +03:00
Mike Fedosin 8212ea423b Use '_prepare_json_response' to serialize list response
Change-Id: I79b6dcbba387e2ee1f133873469ecfb2eea70326
2017-08-27 18:25:49 +03:00
Mike Fedosin 1281a8b4db Update quotas API
Change-Id: I36016050c94cf8baf7bbfa73cd9073f86fab76bc
2017-08-27 14:18:06 +03:00
OpenStack Proposal Bot bbba25a0fc Updated from global requirements
Change-Id: I44e8da1c3519377e306babd8245c4b3acb79ca1c
2017-08-24 05:49:09 +00:00
Mike Fedosin e08ca30792 Add Regex string validator
Change-Id: I8dde3bd70cd03fcef4d246cb489e0e638d7cf7b2
2017-08-23 14:16:51 +03:00
Idan Narotzki 0626284d54 Fix wsgi bug
Today we cannot exit normally from glare app.
This patch should fix that.

Change-Id: Icd236bd995ef3b391a95a2a9a5f2bdd068d0902b
2017-08-22 11:56:17 +00:00
Jenkins 515cae2bad Merge "Add ssl options to keycloak auth middleware" 2017-08-18 14:40:57 +00:00
Jenkins dd931c87c4 Merge "Add read_data function to store_api" 2017-08-18 14:40:48 +00:00
Jenkins 567f0b2789 Merge "Updated from global requirements" 2017-08-18 14:40:34 +00:00
OpenStack Proposal Bot acd1e92694 Updated from global requirements
Change-Id: I0a2fd1e0cb262cee1be9751c39fc402bdd207a17
2017-08-18 11:29:49 +00:00
Mike Fedosin 8675c3950d Specify a way to register artifact conf options
Now to register new config options artifact type
should have 'artifact_type_opts' class attribute,
which represents a list with config options.

After that this options will be placed to
[artifact_type:{type_name}] config section. For
example [artifact_type:images].

Change-Id: I3969b08ad36b37932dbad4919ca3c0fab119344e
2017-08-17 23:18:08 +03:00
Mike Fedosin 7062a59a1e Add read_data function to store_api
Change-Id: Id51f97a1b0828235618c6805c619ac5e3a9ea91a
2017-08-17 22:20:48 +03:00
Mike Fedosin ce02f1b77b Add ssl options to keycloak auth middleware
Change-Id: I48264b3a3fbd25555155a4d744d2f7332e7e4697
2017-08-17 21:13:33 +03:00
Jenkins ce0bfd1170 Merge "Remove unnecessary exception handling in db api" 2017-08-15 11:09:51 +00:00
Jenkins b0f690898b Merge "Implement dynamic quotas" 2017-08-15 10:55:31 +00:00
Jenkins a4842086e2 Merge "Implement static quotas" 2017-08-15 10:53:57 +00:00
Jenkins d4497958cf Merge "Create db functions for quotas" 2017-08-15 10:51:08 +00:00
Mike Fedosin 8f7d615e94 Implement dynamic quotas
This patch adds 3 new api calls

PUT /quotas - set new quotas for specific projects
GET /quotas - get info about all available quotas
GET /quotas/{project_id} - get info about quotas for
 specific project

Implements: blueprint glare-quotas

Change-Id: Icd0c960bac897d48b8502dc39d77b6cffad0d6e4
2017-08-14 13:20:54 +03:00
Mike Fedosin eb1eb29b74 Change default values for glance_store
By default glance and glare will share common place to store data.
For example, for 'filesystem' backend it is '/var/lib/glance/images'.

To prevent possible collisions we have to redefine 4 glance_store default
values:
 * filesystem_store_datadir='/var/lib/glare/artifacts',
 * rbd_store_pool='artifacts',
 * vmware_store_image_dir='/openstack_glare',
 * swift_store_container='glare'

Change-Id: I37093eb491e13b1bd553bfc9788b89d9c95df937
2017-08-13 16:29:24 +03:00