Some of the fixed endpoint configs caused errors.
Magnum and Zun explicitly call for grabbing the API endpoint from
catalog. Switch all the JSON ingester configs to use API endpoint
from service catalog to avoid incorrect config.
Change-Id: I50d2755317f68928f41d3770e25dfe67ba088969
Closes-bug: 1819556
Fixes a bug where json ingester repeatedly creates redundant indexes
each time Congress restarts.
Mitigates a problem where indexing fails when a field exceeds the
postgres indexable limit of 2712 bytes. Indexing is made optional.
Fixes a bug where unexpected behavior occurs when the backend DB
default isolation level is not the expected read_committed. The
isolation level is now set explicitly per connection.
Change-Id: I514aa9b96e4efbffe8880cce775dc2259eca4648
Closes-bug: 1819987
Closes-bug: 1819988
Closes-bug: 1819985
Allow config to specify a port to use in making the API calls to
ingest JSON.
Also added additional sample configs: neutron, keystone, monasca.
Change-Id: I7d852742d0943c3857ee4e04d5a0d77b6bafeed7
partially-implements: bp json-data-model
Add the 'api_default_headers' field which specifies a hash/dict
of headers to pass with each API call w.r.t. that JSON ingester.
Main usage is to specify API microversion. For example:
api_default_headers:
X-OpenStack-Nova-API-Version: "2.26"
Change-Id: Ie859e45ea03135aa21196fe19767f28627e76c3b
partially-implements: bp json-data-model
Allows json ingester config YAMLs to use the !ref tag to reference
previously defined reusable structures, which makes deployment much
more convenient.
Allows devstack plugin to optionally enable JSON ingester feature.
Renamed and reorganized the JSON ingester config options.
Adds CI job which sets up JSON ingester.
partially-implements: bp json-data-model
Change-Id: I6391224c99249d16fe943b8f00fe12d1e6b7d8e6
This commit registers default policies in code. For the
Register and Document Policy in Code community goal.
bp policy-and-docs-in-code
Closes-Bug: 1724714
Change-Id: I1eea53adc19247d8b982c3e22184e8a1e2fb875c
This is the agent part of the blueprint.
Implementation of a datasource that transcribes the content
of configuration files managed by oslo-config in Congress
tables.
The datasource uses a set of agents deployed on the nodes
to access the configuration files.
Change-Id: I56750cfd72ad43d8af123d151f70d1d76568a456
Implements: blueprint configuration-files-validation
Co-Authored-By: Valentin Matton <vmatt.openstack@gmail.com>
Co-Authored-By: Pierre Crégut <pierre.cregut@orange.com>
The purpose of this middleware is to set up the request URL correctly in
case there is a proxy in front of congress.
The HTTPProxyToWSGI is off by default and needs to be enabled via a
configuration value.
Change-Id: Ib0aaebbd42daa94e93ba1daefd3e84241f95e92a
Instead of defaulting to kombu+memory messaging driver, make the
option configurable for the end user.
Changing the config option rpc_backend will change to the
corresponding messaging driver.
Eg: rpc_backend=kombu+memory or rpc_backend=rabbit etc
Change-Id: If1bd27b6cf7d0f732e01bf3a9614f94e49b523cb
The default values needed for congress's implementation of cors
middleware have been moved from paste.ini into the configuration
hooks provided by oslo.config. Furthermore, these values have been
added to the default initialization procedure. This ensures
that if a value remains unset in the configuration file, it will
fallback to using sane defaults. It also ensures that an operator
modifying the configuration will be presented with that same
set of defaults.
Change-Id: I2edc24e4d47c4d664dd31c407d46e42fefdb1488
Closes-Bug: #1551836
This adds the CORS support middleware to Congress, allowing a deployer
to optionally configure rules under which a javascript client may
break the single-origin policy and access the API directly.
For Congress, cors_middleware was added to the paste pipeline on all API's,
and config generation was added where appropriate. Tests were not added,
as the fake_wsgi bootstrap does not actually load a true wsgi pipeline.
OpenStack CrossProject Spec:
http://specs.openstack.org/openstack/openstack-specs/specs/cors-support.html
Oslo_Middleware Docs:
http://docs.openstack.org/developer/oslo.middleware/cors.html
OpenStack Cloud Admin Guide:
http://docs.openstack.org/admin-guide-cloud/cross_project_cors.html
DocImpact: Add link to CORS configuration in admin cloud guide.
Change-Id: Ie6577efe76fa2e7f8a14be33e43facf4f66369d7
In recent release of oslo.middleware
(https://launchpad.net/oslo.middleware/+milestone/3.0.0)
removed the usage of oslo namespace, hence causing
gating errors as we still have one reference to
oslo.middleware. This commit fixes the same.
UpgradeImpact
Change-Id: I6bb68c22dffc370a9745538d4d26db5137f29a7e
Purpose: help users avoid common errors in stand-alone install
Changes: Added instructions for upgrading virtualenv to resolve
common errors
Change-Id: I5ed968ba12d77a205337a8ad3c54c346f208ff1d
congress.conf.sample is out of date, some important options don't
present, like: api_workers, and some option name are wrong, like:
datasource_drivers
We should use oslo-config-generator to generate all the options
automatically.
Change-Id: Id898840667116278c55e4a7568cf80497db4833f
Closes-Bug: #1488405
This patch introduces a new configuration to Congress.
The configuration manages whether Congress can execute any action
or not.
Change-Id: I9d487d0cb70a7e7064399bf7a3f72b598f732c15
Implements: blueprint control-for-execution
The Congress API should be able to list
current supported API versions. This patch
implement the feature.
Change-Id: I307a295369e5ac04567293b680edf202d0711b4b
Implements: blueprint api-version
- Add oslo.policy to requirements.txt
- Update Enforcer constructor to account for new config parameter
- Update policy_file references to account for new config section
- Remove in-tree copy of oslo.policy
Partial-Bug: #1458945
Change-Id: Ifad815a98d6d8efc9c84bd03cb31a40bbfeff948
As discussed in the Liberty Design Summit "Moving apps to Python 3"
cross-project workshop, the way forward in the near future is to
switch to the pure-python PyMySQL library as a default.
https://etherpad.openstack.org/p/liberty-cross-project-python3
Change-Id: I5d266fad182d701456f57e85fa93bfbd5c991bf1
The snapshot file support bypasses the database when inserting rules into the
policy engine, so it makes it possible to insert rules into the engine without
the rules being in the db. That means if Congress tries to synchronize rules
with the db, the synchronizer will delete rules from the snapshot because the
rules do not exist in the db.
Change-Id: Icbd6d1e1f1233aa00157d036f697933da07347f7
The datasource synchronizer reads from the configuration database on a
periodic basis and checks whether the active datasources match the
configuration in the database. If the two do to match, the
synchronizer changes the active datasources to match the database.
By default, datasource synchronization is off. To enable it, add a
line like this
datasource_sync_period = 60
to the congress.conf configuration file.
This change includes a change to the datasource status API semantics.
The previous version of the status handler first checks that the
datasource exists in the database; if the datasource does not the
handler responds to the client with and error because the running
state in the server always matches the database content because there
can be just one Congress server modifying the database. With the
basic high-availability solution, one congress server replica can
modify the database without a second replica knowing, so the running
state of the datasources can become out of sync with the database.
The new version of the datasource status API call returns the status
of the datasource if the datasource is running in the server, even if
the database does not contain the datasource.
This change also makes the list-datasources API call update the
running datasources to match the database after fetching the
datasource configuration from the database.
This also adds a tempest test to verify that datasource
synchronization works.
Implements blueprint: basic-high-availability
Change-Id: I732c965c616f0f11d4d30f7f90b0cce980f9342d
This patch allows datasources to be configured via the API. It exposes a new
api call /v1/drivers/<driver> which returns the configuration
needed by a driver in order to configure it. Then, a user will make a post to
/v1/data-sources with the config to create the datasource.
This patch makes a few structural changes to congress the most noted that it
makes d6Cage a singleton which is needed to be able to dynamically add new
datasources to the message bus.
Also, lots of the API code was changed in order to start moving toward an API
that supports multi tenancy which should now be easy to add.
I apologize that this patch set is so big though the scope of changes needed
to be down to support a configurable API were large and through out many
places in the code. Several cleanup patches will be push on top of this which
should be easier to review.
Implements blueprint: api-configurable-datasources
Change-Id: If82b505e1835254216e19eb1c55a035d2c0b8a3e
This patch adds a datasource driver to congress that integrates with
cloudfoundry. This driver exports the following tables with cloudfoundry
data: organizations, apps, spaces.
Implements blueprint: cloudfoundry-datasource-driver
Change-Id: I632fbf95f6a24ec975448aaa4929fa8a290c3cc3
Co-authored-by: Sabha Parameswaran <sabhap@pivotal.io>
Adds driver to populate datasource tables based on Murano object
model. Excludes python-muranoclient from requirements.txt.
Replaces it with a mock in unit test.
Implements: blueprint murano-driver
Change-Id: I2a9b411bf841e58b7fce4f234f834eb099a332ab
This patch adds a neutronv2 driver. This driver allows one to write policy
against neutron and exposes an improved schema from the first version of the
driver (i.e no more subtables). This patch also exposes security-group-rules
from neutron which were not exposed in the previous driver.
Note: This patch does not remove the original datasource driver. I plan to
remove this in a follow up patchset. The reason why I didn't remove it in this
one is that test_congress uses the neutron datasource driver for several of
its tests.
Implements blueprint: neutron-datasource-driver-refactor
Change-Id: I2124e449d524eea3ab293e0a592b9850f0896bd1
Description:
Data source driver for Swift added.
Also configuration changes and unit test code has been added.
This driver provides swift(object storage) containers list
and objects list in each container
Change-Id: I956ca6bf46ad7f8fba445d271e48d47b7da570fd
The driver provides 3 lists :
1. Meters
2. Alarms
3. Events
The patch also includes test for the above lists.
Change-Id: Iab4f96fdf3f21ec6fab06f08aeaf3eb96bcf4d13
Congress is much more useful when it can see everything which requires
running as the admin user so this patch changes it to do so.
Change-Id: Ibabb33637ebbe16e6758b70dda06dc4839bcb425
Closes-bug: 1375431
This driver uses three keystone API calls to populate Congress tables users,
roles, and tenants. These are the columns for these three tables:
columns ('username', 'name', enabled', 'tenantId', 'id', 'email').
roles ('id', 'name')
tenants ('enabled', 'description', 'name', 'id')
The keystone driver includes a test.
blueprint: refactor-drivers
Change-Id: I8b77576a72e953971bd99bd7cb0a947bf5dc5aa9
This patch just adds /v1/ to each api method so we can have a versioned
api url and updates the docs.
Change-Id: Iea633f2c0ae22a575fcc8f581409923b713468d9
Closes-bug: 1343542