Commit Graph

855 Commits

Author SHA1 Message Date
Clark Boylan b502ec35b0 Omnibus fixups for storyboard testing and CI
Linting fails due to two different hacking rules: H214 and H216. We fix
H214 by using assertNotIn and assertIn over assertFalse(foo in bar) and
assertTrue(foo in bar). We ignore H216 as it says don't use the mock
library but storyboard actually does use mock and updating to use
unittest.mock appears to be a fair bit of work.

In the sqlaclhemy models we move some classes around to address this
error:
  sqlalchemy.exc.InvalidRequestError: When initializing mapper mapped
  class Story->stories, expression 'TimeLineEvent' failed to locate a
  name ('TimeLineEvent'). If this is a class name, consider adding this
  relationship() to the <class 'storyboard.db.models.Story'> class after
  both dependent classes have been defined.

In the project loader we switch to using yaml.safe_load() to avoid the
missing Loader argument to yaml.load().

We pin jinja2 to < 3.0.0 as the old sphinx we've pinned to uses
contextfunction which appears to have been replaced in jinja2 3.0.0.
We should probably attempt to update the doc toolchain for storyboard
when we have time.

We pin MarkupSafe for similar reasons to the jinja2 pin. Version 2.1.0
of MarkupSafe removed the soft_unicode name which is used by sphinx.

Finally we run the python 3.6 and 3.7 jobs on ubuntu-bionic.

Change-Id: Ifcb6660ce2c89c21ba867aa88d942461bcff125d
2022-04-18 16:00:35 -07:00
Adam Coldrick 9ec8875c4c Allow Project Groups to be filtered by project
Currently there is no easy way to find the set of Project Groups that a
specific Project belongs to. This commit adds a parameter to the Project
Groups browse endpoint to enable filtering by Project ID to add this
functionality.

Change-Id: I7b09665bb554c19e8cc4074ce0d1813f3adc4218
2020-09-22 18:42:36 +01:00
Adam Coldrick 63a003af7f Optimise the Story browsing query
The `story_get_all` function currently constructs a query which perfoms
very poorly, making search in StoryBoard painful to the point of being
almost unusable with only ~30000 stories in the database. There are a
number of reasons for the poor performance, but the most impactful is
the fact that we join two subqueries, one of which contains the entire
stories table, and the other which contains a filtered list of story
IDs to implement most of the filtering.

This commit removes all of the subquerying, and just does simple
filtering in a single WHERE clause. Basic comparison of the two queries
shows around an 85% - 90% speedup in the execution time using the new
approach, from ~100ms for the current query, and ~10-15ms for the new
query (using a test dataset containing 10000 stories).

This approach adds a bit of code complexity around filtering stories by
story status, but this is a cost worth paying for the performance
improvement. The new approach will also allow extra flexibility in the
meaning of the story statuses, making it easier to have per-project
statuses for example.

Story: 2005845
Task: 33627
Change-Id: I7c19372506017726e4fc61ffb09d3cfb9c6b36d3
2020-09-11 12:27:55 +01:00
Adam Coldrick 983bb8e2d7 Unpin webob and hacking to fix compatibility issues
The pinned versions of webob and hacking weren't functioning properly
with modern Python, so they are unpinned.

Unpinning hacking brought a number of extra rules into use, so this
commit also contains a number of code style fixes.

There was also an incompatibility in the test_setup.sh script, which
meant it doesn't work with MySQL 8.0 (which is in Ubuntu 20.04). This
is also fixed in this commit.

Change-Id: I0889bc35f8babfeec42f6f577d302bb5fbce1c95
2020-09-11 12:26:47 +01:00
OpenStack Proposal Bot c1e79c0d6f Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I1138db23404b9c74385d57d1e6d15d51035e7af8
2020-01-17 06:14:34 +00:00
Adam Coldrick ad12f5ad7b Add attachments API availability to /v1/systeminfo
Add a boolean representing whether or not attachments are enabled (and
therefore the API endpoints are available) to the /v1/systeminfo
endpoint. This allows clients to check whether attachments can be used
without having to have separate backend-specific configuration
themselves.

Change-Id: I949200d35e8591d1d670fd987ae7be7911aab983
2019-11-16 21:03:37 +00:00
Adam Coldrick 4d018b2eaf Add support for creating attachments
This commit adds a method to the attachments endpoint to handle POST
requests for creating a record of an attachment on a particular story.

Change-Id: Id0c653d15b18644a05daa1aaf881982eb594f18e
Story: 2000679
Task: 29122
2019-11-16 21:03:37 +00:00
Adam Coldrick c1037655e4 Add support for retrieving attachment metadata
This commit adds support for getting details about a specific
attachment, and for getting a list of attachments related to a
specific story, to the attachments endpoint.

Change-Id: Ic4c945167fad77c304f46efc677eaec04426d49b
Story: 2000679
Task: 29116
2019-11-16 21:03:37 +00:00
Adam Coldrick 7662d13857 Add a WSME model for attachments
Change-Id: I8369f7b9ad6c4f8c7807adcac3a13e88db21f0c9
2019-11-16 21:03:37 +00:00
Adam Coldrick 29428572cc Add a table for attachments
This commit adds a model and migration for storing links to and metadata
about attachments in the StoryBoard database.

Change-Id: Ib2e291f5cc80222b226627d076ba3e1f5b88a454
Story: 2000679
Task: 3146
2019-11-16 21:03:37 +00:00
Adam Coldrick 6b805aafce Add an attachments endpoint to stories
This endpoint currently supports returning an upload URL
for the configured storage backend.

Change-Id: I637cf5897f281a4e6b66be6495938aa999a8d3ac
Story: 2000679
Task: 29115
2019-11-16 21:03:37 +00:00
Adam Coldrick 385a34fa77 Add a Swift storage backend implementation
This implements the storage backend interface added in the previous
commit using Swift. It also adds some example configuration to the
sample config file, and some setup to initialise a storage backend
if one is configured.

Change-Id: I8467486ed42f8674e2b1db635789e88bf4113850
2019-11-16 21:03:37 +00:00
Adam Coldrick c11547badd Include enabled features in /v1/systeminfo output
Currently the /v1/systeminfo endpoint is largely useless and just
returns the version of the API as specified in setup.cfg. This commit
adds functionality to also return information about which features are
enabled in a specific backend's configuration.

Change-Id: I9934f656643bb764edc986231b198d92ddc66934
2019-11-16 20:59:38 +00:00
Zuul 8c86a4f09b Merge "Regular expression support for CORS and OAuth ACLs" 2019-10-25 23:14:06 +00:00
Jeremy Stanley 3e4e956ff8 Regular expression support for CORS and OAuth ACLs
Make it possible for allowed_origins and valid_oauth_clients to
include regular expressions, for cases where part or all of the
domain/URL cannot be predicted or easily enumerated.

Change-Id: I9cfc729547560438e0fa1e47cc90cd5579168c73
2019-10-25 22:22:51 +00:00
Adam Coldrick 8c7604ffe0 Add support for POST requests to /v1/openid/authorize_return
Some OpenID providers make a POST request to the `return_to` URL if the
query string would be too long. This commit adds support for POST
requests to this endpoint.

The change in notification_hook.py is to stop attempting to parse the
empty string as JSON when receiving this POST request.

Change-Id: I34d7032d795a5d36799bffc51864e63e585c6eb1
2019-10-25 21:44:28 +01:00
OpenStack Proposal Bot 4c1c7c0cfc Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: Id29d45594f2973eba3e49631617f0668776e6656
2019-10-11 06:14:18 +00:00
Adam Coldrick e859370a9b Correct team iterator lists in board creation
Boards were also iterating over lists of User IDs to populate the team
permissions. This commit fixes them to use the correct variables too.

Change-Id: I2c9bc645a5e02a307f3ff3549adf00a72b5d2f26
2019-06-25 07:54:50 +01:00
Jeremy Stanley dd39dc1b23 Correct team iterator lists in worklist creation
The team_get() method needs to iterate over our list of team owners
and team users, so pass in elements from the correct list names.

Change-Id: I7b18378577e818989fc89118d6b1d947cc7a5682
2019-06-24 23:43:52 +00:00
Adam Coldrick 07098d17ba Use correct package name in /v1/systeminfo
Change-Id: Id3bfec6fcaeab23c24eb646d93344ef35bc5ae54
2019-06-16 17:24:28 +01:00
Zuul c425165b8e Merge "Retry on DB deadlock when updating story" 2019-06-13 17:55:05 +00:00
Zuul 43c716524d Merge "Change the dist name to 'storyboard-api'" 2019-06-13 17:07:55 +00:00
Tony Breeds 79886df354 Change the dist name to 'storyboard-api'
In order to be able to upload to PyPI we need to avoid the name
clash with another existing "storyboard" project which is still
active there, in use and is understandably disinclined consider
renaming (they were there first). We can eventually consider
renaming our project fully to storyboard-api along with its Git
repository and all, but for now simply publishing under an
alternative distribution name will suffice.

While we're in there, add some additional Python package metadata,
correct some more, and remove an invalid trove classifier which
would have prevented uploading. Also mark the resulting wheel as
"universal" (supporting both Python 2 and 3). Bump the minimum PBR
version to accommodate the project_urls and description-content-type
options.

Change-Id: I365ad340ec875f8603e088fa114e7de8aff191c9
2019-06-10 21:34:56 +00:00
Zuul 46beecb00a Merge "Clean up private story filtering code" 2019-06-09 15:42:47 +00:00
Zuul cb42ebd1cb Merge "Allow permissions to be set for teams in worklists and boards" 2019-06-09 15:42:47 +00:00
Zuul 1919d5414a Merge "Rationalise code for filtering private Boards and Worklists" 2019-06-09 14:46:13 +00:00
Zuul eefdeae09b Merge "Add RESTful endpoints for Story permissions" 2019-06-05 21:07:56 +00:00
Zuul 6e8c3dd5f2 Merge "Add a security flag to Stories" 2019-06-05 21:07:50 +00:00
Zuul fe8909e473 Merge "Allow Teams to be filtered by project_id" 2019-06-05 21:07:49 +00:00
Zuul 13b64ab673 Merge "Add an endpoint for getting Projects related to a Team" 2019-06-05 21:07:47 +00:00
Zuul 27e802e4d3 Merge "Add a table mapping Teams to Projects" 2019-06-05 18:55:29 +00:00
Zuul 4dcf96d324 Merge "Add a 'security' flag to Teams" 2019-06-05 18:34:16 +00:00
Zuul 4ff5d9a3b2 Merge "Project Group UI should permit removing project group even if it is not empty." 2019-06-05 18:09:34 +00:00
Adam Coldrick 1e0338c5dc Add RESTful endpoints for Story permissions
Having to update the whole Story in order to add/remove a single
permission entry is quite limiting. This commit addresses this
flaw by adding subcontrollers to retrieve and modify the lists of
Users and Teams who have access to a Story, without having to
modify the whole Story.

This allows much simpler use for clients in situations where
only the ACL needs modification.

Change-Id: I2ecee2c38456c5a23ae1dc7bdecb94efb2daac04
2019-06-04 17:54:57 +01:00
Adam Coldrick b7e1b2e2ae Add a security flag to Stories
This is more specific than "private" and can therefore be sensibly used
for automating ACLs based on projects/teams affected by the story.

Change-Id: Ib8ff53d11ba3977b15d735cef420ac44fd9aa288
Story: 2000568
Task: 29891
2019-06-04 17:53:58 +01:00
Adam Coldrick 30337674c5 Allow Teams to be filtered by project_id
Change-Id: I46b482b68b58288c183abe7446531d6686ad91c3
2019-06-04 17:53:05 +01:00
Adam Coldrick 84518d0ccc Add an endpoint for getting Projects related to a Team
This adds a subcontroller under teams, to expose the mapping of Teams to
Projects. It also provides methods to add and remove Projects from a
Team.

Change-Id: I064082001a2a2d3080c35a1e937d420fd473d96c
2019-06-04 17:53:04 +01:00
Adam Coldrick e7fe27fa22 Add a table mapping Teams to Projects
Change-Id: I4821d4d204a35abab6b43e1037424bdc5c9606a1
2019-06-04 17:52:47 +01:00
Adam Coldrick 81e7f115bb Add a 'security' flag to Teams
This will be used to determine whether or not a Team is responsible
for (and therefore should be automatically subscribed to) security
issues.

Change-Id: I0869e9acc366f3a5546b4e39d736a229762999c2
2019-06-04 17:51:26 +01:00
Zuul 6db2fe34cb Merge "update StoriesController so users subsribe to stories they create" 2019-05-16 02:24:14 +00:00
Zuul af301beaed Merge "Remove simple priority support from the API" 2019-05-02 18:26:07 +00:00
Zuul 6aba5d7f5c Merge "Make the first user a superuser" 2019-05-02 17:15:41 +00:00
Zuul 363c64f47c Merge "Add some tests for checking private story behaviour" 2019-04-10 21:53:59 +00:00
Zuul db126c6403 Merge "Use one database per test class rather than per test" 2019-04-10 21:48:30 +00:00
Ian Wienand 45f7369173 Merge "Replace openstack.org git:// URLs with https://" 2019-04-10 01:34:34 +00:00
Zuul bb0fd27372 Merge "Search bars handle queries as if they have a * at the end" 2019-04-09 21:40:52 +00:00
Riju19 5ad890854c Retry on DB deadlock when updating story
Updating the updated_at time of a story when creating a new task seems to be prone to deadlocks, which results in the task not getting created. This patch retries the update thrice.

Change-Id: I1e657d9eb64717f1dfbaf53d843675ee686c774d
Task: 4141
Signed-off-by: Riju19 <19.riju@gmail.com>
2019-04-08 11:12:42 +05:30
Riju19 a2a3df3d19 Search bars handle queries as if they have a * at the end
Change-Id: Iac8a6a46e7898edd523e3f7c026cbdd134bbff21
Task: 2260
Signed-off-by: Riju19 <19.riju@gmail.com>
2019-04-01 22:04:38 +05:30
Ian Wienand 3f070fe44a Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: Icbbdd5b70854fe740de1b0ebc9de203a2b501500
2019-03-26 13:51:43 +00:00
Ankita Bansal 3292db6bdf Project Group UI should permit removing project group even if it is not empty.
Story: 2000164
Task: 2274

Change-Id: I8099d6fef1fe56d3330fbdf9391100add6afdeca
Signed-off-by: Ankita Bansal <ankitabansal2798@gmail.com>
2019-03-25 21:14:36 +05:30