Commit Graph

44 Commits

Author SHA1 Message Date
Kristi Nikolla 39e4c36344 Same line logs
When logging, headers, url, method, etc, were in different lines
making it hard to distinguish what belonged to the same request
or response. This puts everything in the same line.

Change-Id: I9aa8ce64fa98a13d5d26f80f4e690f2a8eda847b
2018-06-15 18:09:04 -04:00
Kristi Nikolla 61a8444c27 Don't put project_id in path if it wasn't there
This fixes the case that when a request looking like
http://example.com/resource/volume
and containing a token comes to the proxy, we translate it to
http://cinder.local/volume/<project_id>

This happens because we populate the project_id information of
a request either from the url, or from the token.

Change-Id: Ie83fec4910eff750ba15f5485428c23ec3505e3b
2018-06-07 00:27:15 -04:00
Jeremy Freudberg 955f28d397 Extend-a-network
Create a new extension to the proxy, which will allow networks to be
extended across clouds.

Additionally, provide lots of documentation for this new feature.

Change-Id: I9088e3509f71fb363ddc7f504cbb96f94932cc1e
2018-05-15 18:19:26 +00:00
Lohith 902c033af8 Avoid request loops between mixmatch proxies
Closes-Bug:#1763549

Change-Id: I9ad9a2363024ce722c1cb2b1bac27be755fc6888
2018-05-02 16:26:59 -04:00
Kristi Nikolla 5c42bde257 Remove hack for lack of service type
Change-Id: I1fbce415369f8d7ee6edd58299df02ad10d86818
2018-02-16 21:47:46 +00:00
Kristi Nikolla be10840d86 More sophisticated extension matching
Rewrote the extension matching code to work with variables. Now
using the Routes library.

Change-Id: I361e6e597421bc71ed2cf9ed82516ad8b7bd3a8c
2017-10-25 13:10:49 -04:00
Jeremy Freudberg a1a9cad038 Request headers are case insensitive
Per RFC2616. Within the codebase itself we represent headers as
uppercase strings, but now they can be passed with any capitalization
style. (Including whatever keystoneauth or requests chooses to send.)

Change-Id: Ia4e932a91dec030b9efeb947759ceebdb7a426fc
Closes-Bug: #1720433
2017-10-10 17:45:25 +00:00
Jeremy Freudberg b23bf3cefa More attributes of flask.request in RequestDetails
To simplify the writing of extensions, and to reduce reliance on
importing the flask.request global object, store pertinent attributes
of flask.request in the RequestDetails class.

Change-Id: Idff737155a6110d0520a74e36a5b0c23b5b5d93c
2017-09-11 19:47:09 +00:00
Kristi Nikolla 1cb775fcf4 Add mixmatch-manage cli command with db_sync
This adds a cli for management commands that is run through
`mixmatch-manage`.

Also added `db_sync` which is creates the tables in the database
based on the model. No upgrade functionality exists yet.

Change-Id: I18187967bb62865529712c83ef60482c3d1ed8ca
2017-09-05 11:46:08 -04:00
Leo Tomatsu 06543ccbed RequestDetail
The self.detail is now a class rather than a dictionary.

Change-Id: Ia33583613e91707f16de638023e93899528473e2
2017-08-08 19:28:14 +00:00
Jenkins e860a37ea8 Merge "Don't stream json" 2017-08-02 20:42:08 +00:00
Eric Juma 87f89d1f74 Don't stream json
Makes json data that would normally be streamed not streamed
so that it can be modified.

Change-Id: I9690ee1841258e4e6a4d1bfea68e6505484f2216
2017-08-01 18:12:22 +00:00
Jeremy Freudberg c23c48fe8e Neutron pass-through
Network federation preliminary work

Depends-On: I9dcd30fe6669bdf570149ed317b16005c21f6291
Change-Id: Ic5e694259544c2c6ffe717d560fc18391f64a1a0
2017-07-31 17:55:38 +00:00
Jenkins 4130ec8ca6 Merge "Retry request" 2017-07-24 16:00:57 +00:00
Kristi Nikolla dafed90251 Retry request
Will try multiple times on connection errors and on 500, 502, 504.

Change-Id: I16d6eb8a00e452aeaf5ee4a51d423271684bf7eb
2017-07-24 10:31:26 -04:00
Jenkins 5205cc8b3d Merge "Catch invalid MM-SERVICE-PROVIDER early" 2017-07-24 14:18:35 +00:00
Jenkins a4ddfd04f0 Merge "Preliminary extension support + Name routing" 2017-07-21 21:40:28 +00:00
Jeremy Freudberg c1815c66ff Catch invalid MM-SERVICE-PROVIDER early
Now junk in MM-SERVICE-PROVIDER will lead to a client side error,
instead of a server-side one.

Change-Id: Ic930b4093fe005c3abc6c8561ee1114846fb80a6
Closes-Bug: #1705773
2017-07-21 20:17:24 +00:00
Kristi Nikolla 6673e2e6f2 Preliminary extension support + Name routing
This introduces preliminary support for extensions.
Extensions are registered to routes, and if the
request matches the registered route they will be
called to handle the request and the response.
The interface is still not finalized and will
progress as more extensions with their different
requirements are added.

Also included dependency on oslo_serialization for
loading the json response. Follow up patch will
change all instances of json.loads to
jsonutils.loads

Change-Id: I9c573ce1d4ebe85c07c8ff219f384e3c6c67b39a
2017-07-21 16:05:13 -04:00
Kristi Nikolla b488c79e5a Combine aggregate_forward and search_forward
Also included a new decorator utils.CachedProperty.

Change-Id: I3fe6072866e7d5716f6d19cde86684d5b0eb8bff
2017-07-21 11:05:24 -04:00
Jenkins 6f5a885fa4 Merge "The Volume API now supports user messages" 2017-07-06 14:47:15 +00:00
Kristi Nikolla 4bb9dd6841 The Volume API now supports user messages
This tentatively adds support for that API by letting the
microversion header OpenStack-API-Version go through. Also
added a few unit tests just to check that we're properly routing
the messages path.

Change-Id: Ie21b15ee5d33085e0515a129ea4ef89836ca8d57
2017-07-05 17:04:19 +00:00
Jenkins f7aec5ccf5 Merge "Return 404 rather than ValueError" 2017-07-05 16:51:47 +00:00
ltomatsu 199dbbaea8 Return 404 rather than ValueError
The get_service function returns ValueError
when the service response is neither cinder
nor glance. Switch this over to abort 404.

Closes-Bug: #1700144
Change-Id: If2cf585dd555eab4771a27fe6f272842a3d498fe
2017-07-05 15:24:53 +00:00
Jeremy Freudberg 22f45add29 Return most specific error instead of 404
Particularly for client-side errors, we ought to at least return the
actual error status instead of a generic 404.

Enablement of negative Tempest tests.

Change-Id: Ieaf658c9bd202dbb5000c10c43bdd1afa15a2aa1
2017-07-03 16:26:59 +00:00
Kristi Nikolla c49aacec27 Config refactoring
This splits the configuration module into a package with multiple
modules.

Additionally adds support for oslo-config-generator.
Additionally adds mixmatch.conf to .gitignore

Change-Id: I735c6e852ef3cddb6dff654e1b8777ba239b91ab
2017-06-13 12:20:28 -04:00
Kristi Nikolla 6e042d2e1f Move utility functions to utils.py
Some general utility functions such as safe_get and safe_pop were
moved to utils.py.

Change-Id: I14f4fce68369746542abade39b7dc02d3ce272bf
2017-05-30 21:26:13 +00:00
Jenkins 9125a215fb Merge "Fix for Volumev1 Aggregation" 2017-05-15 22:34:10 +00:00
wjdan.alharthi 217f7c3a02 Fix for Volumev1 Aggregation
Takes into account the differences between volumes details
in volume v1 and v2

Co-Authored-By: Kristi Nikolla <knikolla@bu.edu>

Change-Id: Ied7f6f1cca752eb300a284ca1267ad1ef79e6db5
Closes-bug: 1646520
2017-05-15 16:52:46 -04:00
Kristi Nikolla 6b5f16c85c Rename tenant_id to project_id
Tenant_id is old keystone terminology. Replace all instances of
tenant in the model with project. The listener still uses tenant_id
when listening to messages in the message queue. Investigate if
it is safe to rename those too to project_id.

Change-Id: I5126238eb5c4e5ade8effbd9fdc9b73a09b4ea90
2017-05-15 19:32:35 +00:00
Eric Juma e7e56b018d Add more logging
Adds more logging of requests and responses to and from the proxy.
Strips tokens from logs.

Change-Id: Ic08a3adfeda83390c16bb8195d68b7e780e3081f
2017-03-24 19:41:21 +00:00
Eric Juma 8180e9e8fe Ignore service tokens
If service token passed to the proxy, ignore it and do not
forward it in calls to SPs. Service tokens are not currently used
as part of authentication.

Change-Id: I2b830cb4cc9d3fbb62be93211a630d2a859322e3
2017-03-20 17:04:49 -04:00
Kristi Nikolla 4d44d0bb0d Pop limit and marker only during aggregation
Change-Id: Id0eeb99c46f340dc1a5a151690cb2580c6f15ccb
2017-02-10 11:44:48 -05:00
Kristi Nikolla ec18c37e56 Refactoring of proxy.py, stage one
This reduces the usage of member variables in favor of a
dictionary containing details. Now almost all the details are set
in a new function get_details.

Though this is a first good step, more work needs to be done in
follow-up patches.

Change-Id: I73552277255cf3cdb52ebc85b89a7f32f4355038
2017-02-01 16:06:36 -05:00
Eric Juma e1fab7c7ce Allow normal response to unactioned calls if aggregation disabled
Also some mild refactoring in proxy.py. Now unversioned/unactioned calls
can go through without authentication. Renamed self.detailed to the more
descriptive strip_details.

Co-Authored-By: Kristi Nikolla <knikolla@bu.edu>

Change-Id: I5c3a0ab7c020fba480319f11cd1637d5c1f07e11
2017-02-01 14:01:37 -05:00
Eric Juma 755da47df3 Add config option to enable/disable services for each sp
Modified proxy.py and config.py to support the option.

Change-Id: Ibd7d701f4efb1fc952cd4750cd39d98c9eb98a79
2016-12-20 21:58:47 +00:00
Jenkins 840d176f44 Merge "Use get_projects_at_sp in case no MM-PROJECT-ID header" 2016-12-05 21:21:06 +00:00
Kristi Nikolla e517507b00 Use get_projects_at_sp in case no MM-PROJECT-ID header
Change-Id: Ifd99221eabdf9455e405cb6f73f4acf8e93a2760
2016-12-05 16:13:21 -05:00
wjdan.alharthi 12a27eac33 Fix for Volumev2 Pagination
Constrcuted volume pagination properly with 'volumes_links' in
services.py aggregate function. Used extra parameters and changed
the tests accordingly. Added a test for volumes pagination.

Change-Id: I3755b11a274aa66fe3096233cb161ca41080464c
Closes-bug: 1646523
2016-12-02 19:23:41 +00:00
Jenkins c9f7da99a3 Merge "Move configuration options from [proxy] to [DEFAULT]" 2016-11-28 21:55:01 +00:00
Eric Juma 6e9dd7b02b Move configuration options from [proxy] to [DEFAULT]
Move configuration options listed in the [proxy] configuration
group to the [DEFAULT] configuration group for consistency with
other openstack configuration files. Update references to the
configuration group to match the change, including unit tests.

Change-Id: I0ed5a9210952efb91379dcdab2c078f0923649b0
Closes-bug: 1640232
2016-11-28 21:47:50 +00:00
Eric Juma f372c0fc45 Change token logging from INFO to DEBUG
Change token logging from INFO to DEBUG, so that tokens are not
printed by default, only when the logging level is set to DEBUG.

Change-Id: I4f2e1ee84c82edc41edde93386d6f28f815aa09f
Closes-bug: 1640234
2016-11-28 17:28:58 +00:00
Kristi Nikolla a859704206 Fix pep8 issues
Fixed pep8 issues for E126, H238, E241, H405, H403, E121, H201.
H306 and H301 which relate to the way imports are done is still
unfixed.

Change-Id: Ib2668c0af16a75e33e93ec3d699ce2d3fffade38
2016-11-07 15:50:28 -05:00
Kristi Nikolla dcc8b3aa76 Initial commit of the code
This is the initial commit of the code for the project. Previous
development was ongoing in https://github.com/CCI-MOC/k2k-proxy.git

Change-Id: I5cce38e22581e1f0a82c2c76a64e7bbf2cd7490b
Co-Authored-By: George Silvis, III <george.iii.silvis@gmail.com>
Co-Authored-By: Wjdan Alharthi <walharth@bu.edu>
2016-11-07 09:49:52 -05:00