Commit Graph

58 Commits

Author SHA1 Message Date
Nate Johnston bba805af02 Retire the Congress project
Recently the TC has worked on determining the criteria for when an
OpenStack project should be retired.  When there was not a PTL nominee
for the Congress project, that triggered the TC to review the project
health per [1], and the TC has determined [2] that development work on
the project has ceased.  This decision was announced in the
openstack-discuss mailing list in April 2020 [3].

This commit retires the repository per the process for governance
removal in the Victoria cycle as specified in the Mandatory Repository
Retirement resolution [4] and detailed in the infra manual [5].

Should interest in developing Congress as part of OpenStack revive,
please revert this commit to have the project rejoin the list of active
projects.

The community wishes to express our thanks and appreciation to all of
those who have contributed to the Congress project over the years.

[1] https://governance.openstack.org/tc/reference/dropping-projects.html
[2] http://eavesdrop.openstack.org/irclogs/%23openstack-tc/latest.log.html#t2020-04-20T15:36:59
[3] http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014292.html
[4] https://governance.openstack.org/tc/resolutions/20190711-mandatory-repository-retirement.html
[5] https://docs.opendev.org/opendev/infra-manual/latest/drivers.html#retiring-a-project

Change-Id: I21c9ab9820f78cf76adf11c5f0591c60f76372a8
2020-05-28 18:12:16 +02:00
Eric K 8adf6535e1 Fix glance json ingester api path
The /v2/ part previously ommitted. Added here.

Change-Id: I013d3c1e3dfa7f478a228d8d13af86ef37b3771b
2019-05-08 18:39:26 +00:00
Zuul c1a9d2a095 Merge "Replace fixed endpoint config by dynamic in devstack plugin" 2019-03-19 19:30:20 +00:00
Eric K ea1fb9c3ed Replace fixed endpoint config by dynamic in devstack plugin
Some of the fixed endpoint configs caused errors.
Magnum and Zun explicitly call for grabbing the API endpoint from
catalog. Switch all the JSON ingester configs to use API endpoint
from service catalog to avoid incorrect config.

Change-Id: I50d2755317f68928f41d3770e25dfe67ba088969
Closes-bug: 1819556
2019-03-15 13:22:39 -07:00
Eric K a1dfbdec00 Fix redundant index, unindexable, wrong isolation in JSON ingester
Fixes a bug where json ingester repeatedly creates redundant indexes
each time Congress restarts.

Mitigates a problem where indexing fails when a field exceeds the
postgres indexable limit of 2712 bytes. Indexing is made optional.

Fixes a bug where unexpected behavior occurs when the backend DB
default isolation level is not the expected read_committed. The
isolation level is now set explicitly per connection.

Change-Id: I514aa9b96e4efbffe8880cce775dc2259eca4648
Closes-bug: 1819987
Closes-bug: 1819988
Closes-bug: 1819985
2019-03-14 17:18:58 -07:00
Eric Kao bd3a30ae90 More sample configs for JSON ingester
glance, heat, magnum, masakari, mistral, tacker, zun.

Change-Id: I7ebb94c08c993d20b57396197409eaf9a72230ba
partially-implements: bp json-data-model
2019-03-07 15:12:58 -08:00
Eric Kao 8788b5b31d Add cinder config to sample JSON ingester configs
Change-Id: Ic867beeb2aaae6cf754bedc02d25c53bbb77e9c5
partially-implements: bp json-data-model
2019-03-06 23:17:38 -08:00
Eric Kao 7676be3cb6 Allow port in JSON ingester config
Allow config to specify a port to use in making the API calls to
ingest JSON.

Also added additional sample configs: neutron, keystone, monasca.

Change-Id: I7d852742d0943c3857ee4e04d5a0d77b6bafeed7
partially-implements: bp json-data-model
2019-03-06 23:15:29 -08:00
Eric Kao 86a778ca59 Add API header config to JSON ingester
Add the 'api_default_headers' field which specifies a hash/dict
of headers to pass with each API call w.r.t. that JSON ingester.
Main usage is to specify API microversion. For example:

api_default_headers:
  X-OpenStack-Nova-API-Version: "2.26"

Change-Id: Ie859e45ea03135aa21196fe19767f28627e76c3b
partially-implements: bp json-data-model
2019-03-06 11:33:17 -08:00
Eric Kao 5da6e463ff JSON ingester deployment enhancements
Allows json ingester config YAMLs to use the !ref tag to reference
previously defined reusable structures, which makes deployment much
more convenient.

Allows devstack plugin to optionally enable JSON ingester feature.

Renamed and reorganized the JSON ingester config options.

Adds CI job which sets up JSON ingester.

partially-implements: bp json-data-model
Change-Id: I6391224c99249d16fe943b8f00fe12d1e6b7d8e6
2019-03-06 08:35:23 -08:00
Zuul 964c3d4c1f Merge "Agent for configuration file validation" 2018-01-04 21:42:44 +00:00
Lance Bragstad 3ac806a41c Move policy into code
This commit registers default policies in code. For the
Register and Document Policy in Code community goal.

bp policy-and-docs-in-code

Closes-Bug: 1724714

Change-Id: I1eea53adc19247d8b982c3e22184e8a1e2fb875c
2017-12-05 11:36:20 -08:00
Valentin Matton 872f1b9419 Agent for configuration file validation
This is the agent part of the blueprint.

Implementation of a datasource that transcribes the content
of configuration files managed by oslo-config in Congress
tables.

The datasource uses a set of agents deployed on the nodes
to access the configuration files.

Change-Id: I56750cfd72ad43d8af123d151f70d1d76568a456
Implements: blueprint configuration-files-validation
Co-Authored-By: Valentin Matton <vmatt.openstack@gmail.com>
Co-Authored-By: Pierre Crégut <pierre.cregut@orange.com>
2017-12-05 18:17:59 +01:00
shashi.kant 46620ea447 Set HTTPProxyTOWSGI middleware in front of congress.
The purpose of this middleware is to set up the request URL correctly in
case there is a proxy in front of congress.

The HTTPProxyToWSGI is off by default and needs to be enabled via a
configuration value.

Change-Id: Ib0aaebbd42daa94e93ba1daefd3e84241f95e92a
2016-10-10 15:49:24 +05:30
Anusha Ramineni 3f14dd561a Make the messaging driver configurable
Instead of defaulting to kombu+memory messaging driver, make the
option configurable for the end user.

Changing the config option rpc_backend will change to the
corresponding messaging driver.
Eg: rpc_backend=kombu+memory or rpc_backend=rabbit etc

Change-Id: If1bd27b6cf7d0f732e01bf3a9614f94e49b523cb
2016-06-02 00:58:12 +09:00
Tin Lam c596b89c48 Moved CORS middleware configuration into oslo-config-generator
The default values needed for congress's implementation of cors
middleware have been moved from paste.ini into the configuration
hooks provided by oslo.config. Furthermore, these values have been
added to the default initialization procedure. This ensures
that if a value remains unset in the configuration file, it will
fallback to using sane defaults. It also ensures that an operator
modifying the configuration will be presented with that same
set of defaults.

Change-Id: I2edc24e4d47c4d664dd31c407d46e42fefdb1488
Closes-Bug: #1551836
2016-03-14 10:55:49 -07:00
Michael Krotscheck d39b89f90d Added CORS support to Congress
This adds the CORS support middleware to Congress, allowing a deployer
to optionally configure rules under which a javascript client may
break the single-origin policy and access the API directly.

For Congress, cors_middleware was added to the paste pipeline on all API's,
and config generation was added where appropriate. Tests were not added,
as the fake_wsgi bootstrap does not actually load a true wsgi pipeline.

OpenStack CrossProject Spec:
   http://specs.openstack.org/openstack/openstack-specs/specs/cors-support.html
Oslo_Middleware Docs:
   http://docs.openstack.org/developer/oslo.middleware/cors.html
OpenStack Cloud Admin Guide:
   http://docs.openstack.org/admin-guide-cloud/cross_project_cors.html
DocImpact: Add link to CORS configuration in admin cloud guide.

Change-Id: Ie6577efe76fa2e7f8a14be33e43facf4f66369d7
2016-01-08 08:55:16 -08:00
Anusha Ramineni 0fcf75e829 Unblocks Gate-Use oslo_middleware
In recent release of oslo.middleware
(https://launchpad.net/oslo.middleware/+milestone/3.0.0)
removed the usage of oslo namespace, hence causing 
gating errors as we still have one reference to
oslo.middleware. This commit fixes the same.

UpgradeImpact
Change-Id: I6bb68c22dffc370a9745538d4d26db5137f29a7e
2015-11-25 09:21:38 +00:00
Eric K e6bab1272f Added sample conf instructions to resolve errors
Purpose: help users avoid common errors in stand-alone install
Changes: Added instructions for upgrading virtualenv to resolve
 common errors

Change-Id: I5ed968ba12d77a205337a8ad3c54c346f208ff1d
2015-10-16 13:28:22 -07:00
Hidekazu Nakamura 32a2773fca Fix typo in /etc/README-congress.conf.txt
nova -> congress

Change-Id: I8571a9bba06ebd619c2a09a3e0486ea4697ad392
2015-09-09 19:39:58 +09:00
Rui Chen 8f7932cfdf Using oslo to generate congress.conf.sample
congress.conf.sample is out of date, some important options don't
present, like: api_workers, and some option name are wrong, like:
datasource_drivers

We should use oslo-config-generator to generate all the options
automatically.

Change-Id: Id898840667116278c55e4a7568cf80497db4833f
Closes-Bug: #1488405
2015-08-27 19:05:59 +08:00
Jenkins b04ee756d5 Merge "Add execution controll option" 2015-07-28 16:47:48 +00:00
Masahito Muroi 60f1d71a5b Add execution controll option
This patch introduces a new configuration to Congress.
The configuration manages whether Congress can execute any action
or not.

Change-Id: I9d487d0cb70a7e7064399bf7a3f72b598f732c15
Implements: blueprint control-for-execution
2015-07-28 07:06:09 +00:00
Rui Chen 4edd65a907 Support version list API
The Congress API should be able to list
current supported API versions. This patch
implement the feature.

Change-Id: I307a295369e5ac04567293b680edf202d0711b4b
Implements: blueprint api-version
2015-07-27 11:15:57 +08:00
Jenkins 7cb67aa68f Merge "Switch from MySQL-python to PyMySQL" 2015-06-26 20:21:20 +00:00
Peter Balland 6da0511be3 Use graduated oslo.policy
- Add oslo.policy to requirements.txt
- Update Enforcer constructor to account for new config parameter
- Update policy_file references to account for new config section
- Remove in-tree copy of oslo.policy

Partial-Bug: #1458945
Change-Id: Ifad815a98d6d8efc9c84bd03cb31a40bbfeff948
2015-06-03 17:18:07 -07:00
Jeremy Stanley 2ea687898a Switch from MySQL-python to PyMySQL
As discussed in the Liberty Design Summit "Moving apps to Python 3"
cross-project workshop, the way forward in the near future is to
switch to the pure-python PyMySQL library as a default.

https://etherpad.openstack.org/p/liberty-cross-project-python3

Change-Id: I5d266fad182d701456f57e85fa93bfbd5c991bf1
2015-05-20 01:03:58 +00:00
Alexander Yip 893c5ad784 Remove support for policy snapshot file
The snapshot file support bypasses the database when inserting rules into the
policy engine, so it makes it possible to insert rules into the engine without
the rules being in the db.  That means if Congress tries to synchronize rules
with the db, the synchronizer will delete rules from the snapshot because the
rules do not exist in the db.

Change-Id: Icbd6d1e1f1233aa00157d036f697933da07347f7
2015-04-20 16:49:56 -07:00
Alexander Yip edfbbe4c21 Add datasource synchronizer
The datasource synchronizer reads from the configuration database on a
periodic basis and checks whether the active datasources match the
configuration in the database.  If the two do to match, the
synchronizer changes the active datasources to match the database.

By default, datasource synchronization is off.  To enable it, add a
line like this
   datasource_sync_period = 60
to the congress.conf configuration file.

This change includes a change to the datasource status API semantics.
The previous version of the status handler first checks that the
datasource exists in the database; if the datasource does not the
handler responds to the client with and error because the running
state in the server always matches the database content because there
can be just one Congress server modifying the database.  With the
basic high-availability solution, one congress server replica can
modify the database without a second replica knowing, so the running
state of the datasources can become out of sync with the database.

The new version of the datasource status API call returns the status
of the datasource if the datasource is running in the server, even if
the database does not contain the datasource.

This change also makes the list-datasources API call update the
running datasources to match the database after fetching the
datasource configuration from the database.

This also adds a tempest test to verify that datasource
synchronization works.

Implements blueprint: basic-high-availability
Change-Id: I732c965c616f0f11d4d30f7f90b0cce980f9342d
2015-04-07 13:30:28 -07:00
Aaron Rosen 6fc440828d Remove datasource.conf from needing to exist
This patch refactors out the datasource.conf file from the project.

Change-Id: I322974704fc909430b044898fb17afeb01208105
2015-02-11 11:53:03 -08:00
Aaron Rosen b7a950bebf Make datasources configurable via the API
This patch allows datasources to be configured via the API. It exposes a new
api call /v1/drivers/<driver> which returns the configuration
needed by a driver in order to configure it. Then, a user will make a post to
/v1/data-sources with the config to create the datasource.

This patch  makes a few structural changes to congress the most noted that it
makes d6Cage a singleton which is needed to be able to dynamically add new
datasources to the message bus.

Also, lots of the API code was changed in order to start moving toward an API
that supports multi tenancy which should now be easy to add.

I apologize that this patch set is so big though the scope of changes needed
to be down to support a configurable API were large and through out many
places in the code. Several cleanup patches will be push on top of this which
should be easier to review.

Implements blueprint: api-configurable-datasources

Change-Id: If82b505e1835254216e19eb1c55a035d2c0b8a3e
2015-02-10 13:09:56 -08:00
Aaron Rosen cf89c453a2 Add cloudfoundry datasource driver
This patch adds a datasource driver to congress that integrates with
cloudfoundry. This driver exports the following tables with cloudfoundry
data: organizations, apps, spaces.

Implements blueprint: cloudfoundry-datasource-driver

Change-Id: I632fbf95f6a24ec975448aaa4929fa8a290c3cc3
Co-authored-by: Sabha Parameswaran <sabhap@pivotal.io>
2015-02-04 20:42:33 -08:00
Steven Tran 833c823a71 Adds driver for Murano datasource
Adds driver to populate datasource tables based on Murano object
model. Excludes python-muranoclient from requirements.txt.
Replaces it with a mock in unit test.

Implements: blueprint murano-driver

Change-Id: I2a9b411bf841e58b7fce4f234f834eb099a332ab
2015-02-03 16:00:20 -08:00
Aaron Rosen d6f4105f01 Implement NeutronV2 Driver
This patch adds a neutronv2 driver. This driver allows one to write policy
against neutron and exposes an improved schema from the first version of the
driver (i.e no more subtables). This patch also exposes security-group-rules
from neutron which were not exposed in the previous driver.

Note: This patch does not remove the original datasource driver. I plan to
remove this in a follow up patchset. The reason why I didn't remove it in this
one is that test_congress uses the neutron datasource driver for several of
its tests.

Implements blueprint: neutron-datasource-driver-refactor
Change-Id: I2124e449d524eea3ab293e0a592b9850f0896bd1
2015-01-14 10:59:54 -08:00
Conner Ferguson 75a0d511a9 Add vCenter driver
Implements blueprint: vcenter-driver

Change-Id: Ifc9334d79295fa5edaaebc66a0484492977a1e36
2014-12-30 22:21:49 -05:00
Abhishek Chanda 1eea4579b9 Migrate to oslo.middleware
The incubator library has graduated and will be removed soon

Change-Id: I6cba0375047a5aad5cd3c76532c9bb76e3388250
Closes-Bug: #1398639
2014-12-13 15:14:34 +00:00
Conner Ferguson 046516109b Added Plexxi Driver
Change-Id: Iee629b199ce8c4251d7c2aea27877fc88b46990d
2014-11-21 20:07:23 +00:00
Jenkins 5d3baaedc0 Merge "Add missing database section to congress.conf.sample" 2014-11-20 20:23:37 +00:00
Aaron Rosen 87e72094a8 Add glancev2 datasource driver
Implements blueprint glance-data-source-driver

Change-Id: I7b6dd7604a0cbcb5ab0962a37e68d995315be0c3
2014-11-19 16:33:31 -08:00
Aaron Rosen 3a53d9e3a2 Add missing database section to congress.conf.sample
Change-Id: Icca22301f9aafd0e94e9cf592346362a97e78036
closes-bug: #1392556
2014-11-13 18:18:45 -08:00
Srinivasa Rao Ragolu f7d5ee8c1d Datasource driver for Swift
Description:

Data source driver for Swift added.
Also configuration changes and unit test code has been added.
This driver provides swift(object storage) containers list
and objects list in each container

Change-Id: I956ca6bf46ad7f8fba445d271e48d47b7da570fd
2014-11-11 14:34:35 +05:30
Samta fc92c7cfa1 Data source driver for Cinder
Following lists are supported by this driver:
- Volumes
- Snapshots
- Services

Change-Id: I66bd1e8e86f6f9e7ea61b8318dc8c958043ba0e9
2014-10-24 11:39:15 +05:30
Aaron Rosen aae285bafb Trival whitespace cleanup
Change-Id: I50b5a807935c41bdd12064ef8f085fd57cb12928
2014-10-06 13:50:35 -07:00
Aaron Rosen ce15d88890 Ceilometer should also connect as admin
Change-Id: I00c51c40b9108987e2337c7e866b552e0209f730
2014-10-06 13:47:21 -07:00
Jenkins 49f3783a5b Merge "Data source driver for ceilometer" 2014-10-06 19:53:04 +00:00
Madhu Mohan Nelemane ab03f13d4e Data source driver for ceilometer
The driver provides 3 lists :
1. Meters
2. Alarms
3. Events
The patch also includes test for the above lists.

Change-Id: Iab4f96fdf3f21ec6fab06f08aeaf3eb96bcf4d13
2014-10-04 15:28:45 +05:30
Aaron Rosen be2b407921 Make default creds for datasources admin
Congress is much more useful when it can see everything which requires
running as the admin user so this patch changes it to do so.

Change-Id: Ibabb33637ebbe16e6758b70dda06dc4839bcb425
Closes-bug: 1375431
2014-09-30 15:50:06 -07:00
Alexander Yip 4c5c3d71b8 Add a Congress driver for Keystone
This driver uses three keystone API calls to populate Congress tables users,
roles, and tenants.  These are the columns for these three tables:

columns ('username', 'name', enabled', 'tenantId', 'id', 'email').
roles ('id', 'name')
tenants ('enabled', 'description', 'name', 'id')

The keystone driver includes a test.

blueprint: refactor-drivers

Change-Id: I8b77576a72e953971bd99bd7cb0a947bf5dc5aa9
2014-09-25 15:09:40 -07:00
Aaron Rosen 586c164805 Fix bind_port in sample config to be correct
Change-Id: Ic6b21223680122ad4270167f3ef9eacfe9279c5b
2014-09-15 16:18:21 -07:00
Aaron Rosen ba7fb8864e Add version /v1 to url in congress api
This patch just adds /v1/ to each api method so we can have a versioned
api url and updates the docs.

Change-Id: Iea633f2c0ae22a575fcc8f581409923b713468d9
Closes-bug: 1343542
2014-09-10 17:02:09 -07:00