This implements the storage backend interface added in the previous
commit using Swift. It also adds some example configuration to the
sample config file, and some setup to initialise a storage backend
if one is configured.
Change-Id: I8467486ed42f8674e2b1db635789e88bf4113850
Make it possible for allowed_origins and valid_oauth_clients to
include regular expressions, for cases where part or all of the
domain/URL cannot be predicted or easily enumerated.
Change-Id: I9cfc729547560438e0fa1e47cc90cd5579168c73
To make full use of the new 4-byte character set in columns for
MySQL, connections to the database need to also specify
charset=utf8mb4. The easiest way to do that is by tacking it on as a
variable in the DBURI string. Do this both in the test migration
script and in the documented sample and test configs.
Change-Id: Icf5f74543f6d062cafe2166d384f37f46394e964
This will allow the fixing of an issue where changes created
by things using the API directly, such as the Gerrit plugin,
cause broken URLs in notification emails.
Change-Id: Ibad10fa35b7acaaa59cbb28b0960571480f80552
Workers instantly exit if rabbitmq server is not available. This changes
use connection_attemps and retry_delay pika connection parameters to try to
reconnect before failing.
Change-Id: I00ff950270360405da092803c1d251552c837831
This commit adds a configuration option which must be set in order
to enable users to edit their comments, and enable superusers to
delete comments.
The option, `enable_editable_comments`, is False by default, meaning
that the editing and deletion functionality must be opted into. If
it is False, then only a database admin can delete or edit comments,
by modifying the database directly.
Change-Id: Iabf598eae3aa35e4e53aadfe514fb2b0da37cefc
This middleware will automatically redirect any clients that
express a desire for text/html content away from the API and to
the configured webclient. It assumes that the webclient will have
an identical URL structure after the API root, or have sane 404
handling.
This feature is being added to simplify generating a URL in email.
We point the user at the API, and the API will redirect the user
to the appropriate location. If, instead, the client sends a
different accept header, then this will not trigger and return
the raw data instead.
Example:
GET https://storyboard.openstack.org/api/v1/projects
303 https://storyboard.openstack.org/#!/projects
Future work on this feature will intelligently check for search
engine bots, and return a scrapeable html page for SEO.
Change-Id: Id98e12f85ce1523ab3982d070c438583d51ce9cb
This patch removes page_size_default. This is done as a followup
to removing page_size_maximum, because the justification for
removing that defines the use case of getting all records. With
page_size_default in place, getting all records is not strictly
possible, as an API consumer must always declare a 'large' page
size in order to ensure they get everything, and even in that
case it's not guaranteed. Thus, making 'limit' an optional
parameter switches the default API behavior to getting all records.
Change-Id: I4a2ffdfbb91dab79bdbfa440e40c1fc8fe62637c
This patch removes maximum page size from all API calls. I've
realized that it is not for the API to force the clients to
implement paging. Some clients may find it easier to grab single,
large requests, while others want to filter and page their
requests. There are cases in which result limits are beneficial,
and some in which they are not - it is up to the client to decide
which use case is best for them.
Furthermore, this patch sets the minimum size of a page to 0. This
is to support queries for information only (how much do I have).
Change-Id: Ie6c45fad43756dae762b8f973b30439c3340f558
According to spec in https://review.openstack.org/150743,
integration tests will be launched from the backend.
Add new install_test_backend.sh and destroy_test_backend.sh
scripts that will be called from each integration tests.
The workflow will be:
- install_test_backend
- run integration test
- destroy_test_backend
Change-Id: Ie676d3152cc4cd854ab6644e5fd281de9c2d70fd
python-crontab is GPL'd, so we can't use it. This replaces all
related functionality with apscheduler. Notable API changes: Since
the APScheduler guarantees single execution environments, we no
longer have to provide execution time windows for our plugins.
Note, pymysql 0.6.4 seems to have issues with some unicode characters.
Change-Id: Ie8b3473ac316c8c661f7ffe1cdf069e7e822f23b
mysql-python undesireable over PyMySQL for a few reasons. Mainly, it
does not support Py3* and does not work well with eventlet.
There is also a bug in using oslo.db with PyMySQL where reference errors
are not properly detected. This fixed in oslo.db
ab20754db71e55b79b9e71e36ad86d9befc89a92 but we should still work around
this unless global-requirements are bumped for oslo.db.
Also adding pymysql to requirements since it is a pure python library
and having a codebase that can 'just work' is awesome.
Change-Id: I0058193c7cbd329731ade37108614fa6eb19d0f7
This patch adds the configuration options that will allow us to
specify the smtp server which we are connecting to. These options are
drawn directly from smtplib, and will be used in subsequent patches
to generate a context-aware smtp sender.
Change-Id: I6fff2575d12e00f54b5733f990b1535649c8280d
This patch adds the valid_oauth_clients configuration option, a list
of string ID's that limits what the client may use to identify itself.
This is a breaking change for downstream consumers, as the domain on
which storyboard is hosted will need to be added to this
configuration option before logins will again be permitted.
Change-Id: I411fdf67b56f5be0d151152471e5a00a75daa51e
A TTL was added to the authorizationcode table, and the validator now
checks against that TTL to make sure that outdated authorization codes
are no longer honored.
Change-Id: Id793a6dd219c4f83b66fb6a0fbf79d2079f2d936
The OAuth configuration elements were moved into a single block
within storyboard.conf, so that relevant other components could
add their own configuration. The OAuth Token Cleaner was then provided
with its own configuration flag so that it can be disabled.
Change-Id: I9c02ea913f184c0734e2b694469d36c5e96339b5
This adds a crontab plugin hook to StoryBoard, allowing a plugin
developer to run periodic events. Example use cases include:
- Summary emails.
- Periodic report generation.
- Synchronization check points.
Plugins are expected to provide their own execution interval and
configuration indicator. The management of cron workers is
implemented as its own cron plugin as a sample, and unit tests
for all components are provided.
Change-Id: I3aa466e183f1faede9493123510ee11feb55e7aa
This patch creates a configurable working directory which storyboard may
use to persist files on disk. Example uses may be file uploads, deferred
processing of emails, storing pid files, and others. The library also
provides an API by which a plugin may request their own, isolated
working directory.
The default configuration is ~/.storyboard, as we have a reasonable
expectation that the current user has the ability to create their own
directories. More advanced installs may want to put this in
/var/lib/storyboard or /opt/storyboard.
Change-Id: I41adefbbba7466ab82240b3c079f1e43e261dfbb
Added project group import to the project import script. Note that
the group import does not behave quite the same as projects do-
orphaned projects are not deleted (as per previous behavior),
however orphaned project groups are.
Change-Id: Id3f57a9154ba828e63bb67bec8f1cb1ce0ee3443
Added new middleware that intercepts and decorates all requests in
accordance to the Cross Origin Resource Sharing W3C specification
found at http://www.w3.org/TR/cors/. Permitted origins and cache max
age are configurable, however headers and methods are hard-coded since
those are application-specific. The notification hook was also
updated to only trigger on POST, PUT, and DELETE.
Motivation for this change is to allow js-draft builds of the
storyboard webclient to break the browser sandbox, and become fully
operational by accessing the production storyboard api. Reviewers
interested in seeing a functioning webclient for UI review will no
longer have to download and run their own client.
Patch to make the webclient build support a configurable API backend
is forthcoming.
Change-Id: I7a825820e4edf48cd9552b2c1c656bc7e664a25a
AMQP message broadcasting did not reconnect when, for some reason,
the server disconnected (ex: restart). This change builds on the
ConnectionService class to make it a generic, self-healing
connection manager that may be extended. The publisher and
subscriber logic was then built on top of ConnectionService to
take advantage of connection management.
- New self-healing, lazy initializing connection service.
- Publisher extends connection service.
- Subscriber extends connection service.
- ResourceHook was moved into notifications as the NotificationHook.
- Configuration options for explicit exchange and queue naming added.
Change-Id: Ib57c56a38574a0c70db9066625aef75ff8891c93
For each action, except for creation of a new project, two messages will
be sent to rabbitmq. One with the main resource; resource_id; user_id;
method and the other with the event_id; user_id; a faked method POST and
a faked resource TIMELINE_EVENT
Publisher creates an exchange called 'storyboard' and publishes the
messages to it. Subscriber creates the queues with different binding_keys
which bind themself to the storyboard exchange and start consuming the
messages from the exchange.
The consumed messages are used by CR 113016.
Co-Authored-By: Nikita Konovalov <nkonovalov@mirantis.com>
Change-Id: Ia573437302dc2d0b1a68d2343e83f9dd397fac04
This reverts commit 9a5a5b59d6.
As agreed on #openstack-infra, this change is being made so
proper credit may be given to the actual authors of this patch.
Change-Id: I07fda79ca0bdd2cfd2a1a45d25fd0a1fc692b6f4
For each action, except for creation of a new project, two messages will
be sent to rabbitmq. One with the main resource; resource_id; user_id;
method and the other with the event_id; user_id; a faked method POST and
a faked resource TIMELINE_EVENT
Publisher creates an exchange called 'storyboard' and publishes the
messages to it. Subscriber creates the queues with different binding_keys
which bind themself to the storyboard exchange and start consuming the
messages from the exchange.
The consumed messages will be printed on the console for now which will
be modified later.
Do not merge until storyboard has Rabbitmq running.
Change-Id: Ic4697f79aaab82dadf1fb1ae66f414a90ae28dac
The refresh token may now be used to obtain a new access token if
needed. The refresh token can be used only once.
Sample request:
POST /v1/auth/token?grant_type=refresh_token&refresh_token=<the_token>
The response will be similar with the one if authroization code is used.
Introduced expiration for refresh tokens.
The migration deletes existing refresh tokens as they do not have a
valide expiration information set. Probably noone should notice that
because client is not using refresh tokens currently.
Change-Id: Ie0924888f66ca956caa43b04b8501e6fa8e9371e
The option use-storyboard should be set to True. Otherwise the
load_projects script would ignore that Project.
Change-Id: I283229cca5fea26337424f41c167c6e4bdf7c8bc
This time should be configurable, so that it may be increased untill the
refresh token is supported. The defalult value is still 3600 secons to
match the standard.
This change also fixes small issue with oslo.config in storyboard-api
entry point.
Change-Id: Ib4d31c90e42e5180a9b891fe0d2c9eeb780ca7ee
In order to accomodate large lists of projects, stories, and tasks
in a way that response quickly to user requests, I've added limited
support for oslo's paginate_query method. Search responses now also
include metadata headers that allow us to manage paging in our
result set: X-Limit, X-Marker, X-Total.
Using Markers, instead of offsets, comes with a tradeoff - calculating
the _current_ page of the result can only be done in client memory by
loading all previous records and determining where the marker lives.
It probably makes sense to also permit offset, or to only allow offset
and determine the marker record based on that.
The benefit of using marker-style paging is that - rather than using
a paging metaphor, we can implement an 'infinite scroll' UI on long
lists of records. Whether that's a good idea remains to be seen.
Additional changes -
- Page size maximum and default is configurable
- I had to change getAllStories to a distinct subselect, because
left joins were screwing up our result sets.
Change-Id: I058a4182d2b454edbbfb7db3493d94b3bad07b36
The idea is the same as for projects.
Usage:
storyboard-db-manage --config-file <config> load_superusers
<superusers.yaml>
Change-Id: Ia7f3f7415ffedb5e449bf7b9ca76a28d0cf31824
Endpoints added.
OpenId support added.
Memory storage added for tokens and authorization codes.
Tbd in next commits:
* Add a middleware for token validation
Change-Id: I1805bc645428bc9301dc3447537fd9792afe781d
load_projects command added.
This command should be initialize project_groups and projects.
usage:
storyboard-db-manage --config-file <config> load_projects <projects_file>
Example file added etc/projects.yaml.sample
Tests added.
Change-Id: Ib5cb32064aa0629c45c3249e08a0ca2281acd8f0
storyboard.conf added to .gitignore
api section remove from config.
To configure API host and port use bind_host and bind_port in default
section.
Change-Id: I2e2686faa063e1182e37f2bf869d0cd7d54ef352
This is initial commit adding pecan/wsme framework.
Example operations are:
* GET /v1/project_groups
* GET /v1/project_groups/<group_name>
* GET /v1/projects
* GET /v1/projects/<project_name>
* GET /v1/teams
* GET /v1/teams/<team_name>
* POST /v1/teams
* POST /v1/teams/add_user
* GET /v1/users
* GET /v1/users/<username>
* POST /v1/users
* PUT /v1/users/<username>
* GET /v1/stories
* GET /v1/stories/<story_id>
* POST /v1/stories
* PUT /v1/stories
* POST /v1/stories/add_task
* POST /v1/stories/add_comment
* GET /v1/tasks
* GET /v1/tasks/<task_id>
* PUT /v1/tasks
More detailed documentation will be added later to a wiki page.
Tests will be added in a separate CR.
Auth stuff will be added in a separate CR after it is dicussed.
Change-Id: Ibace8cf7dd5bb933b0d2484b1d57b79bb8441a28
As a step towards continuous deployment and having a pecan/WSME REST
interface, split the database out into SQLalchemy-based model using
Alembic for migrations. To support that, also pull in oslo.db and use
oslo.config for config files.
Change-Id: I33a1e72700be14e28255aaa52faed70c4686a3ec
Adam Coldrick