The original Alembic migration script was autogenerated, then
subsequently modified by hand. Between these two aspects, some
correspondence to the corresponding SQLAlchemy models was lost:
* Columns were specified independently of corresponding foreign key
and primary key constraints; besides making it more difficult to
follow the code between models and the migration script, there was a
loss of integrity constraints: a variable association must exist for
every entity that mixes in VariableMixin as a base was not being
enforced at the database level with an appropriate not-null; this
causes problems for use of variables for any such object that is not
constructed by models code, most notably via direct usage of the
database. (See specifically
https://bugs.launchpad.net/craton/+bug/1668251, fixed with this
change.)
* The column created_at would always be set by model-using code; but
this was not enforced at the database level by ensuring it was not
null.
* Children should be deleted if the parent is deleted (we do not
support "soft delete" in Craton).
In addition, all constraints are now named (the original intent of
this change, so as to ensure the feasibility of future migrations),
and they are also named in a consistent fashion.
Note that this change does not support many-to-many deletions (as seen
in https://bugs.launchpad.net/craton/+bug/1668308), but SQLAlchemy
does provide support for this at the model level. A future change can
address this without requiring Alembic support.
There's no direct testing of this change, but it is implicitly, and
rather strongly, tested by the overall functional tests that are used;
and to a lesser extent by some of the unit tests. This is in part
because this change has strengthened constraints, not weakened them.
Change-Id: I1f84c29610127de12c292a210fd003ae07bd6462
Closes-bug: 1665066
Closes-bug: 1668251
This patch implements the ability to query for values
in nested variable documents. For instance, if we had
hosts with a variables that look like:
{
"hardware_profiles": {
"disks": [
{
"manufacturer": "seagate",
"capacity_quantity": 2,
"capacity_unit": "TB"
},
{
"manufacturer": "western",
"capacity_quantity": 3,
"capacity_unit": "TB"
}
]
}
}
and we wanted to grab all of them with a disk that was
manufactured by Seagate, we would like to be able to
query like so:
GET /v1/hosts?vars=hardware_profiles.disks[*].manufacturer:"seagate"
This does modify the variables query to expect valid JSON
values. So, strings need to be quoted, for instance.
Change-Id: Id597d3e57d2e28766fecd1b314f53176543e1b9d
Closes-Bug: 1671116
Adding wrapper functions to the tools directory. Updating development docs
advising users to clone the sample config to a dev config. The dev config
allows users to launch craton directly and in docker without constant config
changes.
Change-Id: I6c32c86ef84fdcf02077fea226e20110826b1f21
Closes-Bug: #1675113
So far we have been using external mysql command
to insert project/user to bootstrap the process.
This caused much frustration since operators needed
to insert records in the db. This patch adds commands
to dbsync to allow creating project/users. This method
also takes care of association_id.
To initiate a bootstrap:
craton-dbsync --config-file=craton.conf bootstrap
which will create a project with a root user.
Users using Dockerfile can look at docker logs
after creating the container to find this info.
Closes Bug: 1670561
Change-Id: I9372961ca6623d530d7844b9f38aade544d961e8
Adds labels to device and container creation
such that we can use --label query to filter
by labels.
Change-Id: Icd47cbaca0067fad66e0f3b75150653b6a5bde3d
This change moves craton to a port that is not in use according to
/etc/services and is not being consumed by any other OpenStack service
as defined here: [
https://docs.openstack.org/newton/config-reference/firewalls-default-ports.html
]. This should allow other deployers to leverage craton in an way that
does not conflict with other OpenStack services already running in
production.
Change-Id: I2d0432b4c608cf11538d7b8f267eaf7be763204c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch includes the Cloud resource to allow
for a project o have multiple clouds under it. This
modifies the environment hierarchy to look like:
Project -> Cloud -> Region (-> Cell) -> ...
Change-Id: Ib1c16a504430760f2a7234221f91429a7ea72596
Closes-Bug: 1662576
For our default CI jobs, we can add a test-setup.sh script that will run
before OpenStack CI revokes sudo for the "jenkins" user.
At the moment, we cannot add the jenkins user to the docker group to
allow the jenkins user to use docker. Instead, we have to make docker
launch itself so that the jenkins group can access the socket.
Change-Id: I732d144e4635dffe8034c0d374eee591df078f52
This patch re-applies RBAC-lite to the /projects
endpoint. For the top-level /projects endpoint,
permissions have been narrowed to only root
project users. Anything pertaining to a specific
project has been narrowed to project admin users.
This patch implements a decorator to afford us the
same features for the generic resources functions
that @require_project_admin_context and
@require_admin_context decorators afforded us
piecemeal for other DB API functions.
Change-Id: I875a9a8cb68c76d27f613bac6f4b87f3a1e2b411
Closes-Bug: 1665237
Closes-Bug: 1648626
Includes Variables support, which is already present
in several other Craton resources, in the Projects
resource as well.
Project variables will serve as a top-most scope
of variables for resources that belong to a specific
project.
Partial-Bug: 1648626
Closes-Bug: 1661670
Change-Id: I574e834ce409efd8ceafb15fe30a86ae93df2a80
This commit updates POST responses based on API-WG guidelines [1] and
RFC 2616 [2]. Specifically:
- POST requests have been updated to return '201 (Created)' on success
(previously returned 200)
- a Location header field has been added to responses (previously not
included in response)
In addition:
- The unit tests have been updated to deal with 201 return codes and
to check for the presense of the Location header
- The generate_fake_data script has been updated to deal with 201
return codes
- POST requests in the documentation have been updated to the
new status code
[1] https://specs.openstack.org/openstack/api-wg/guidelines/http.html
[2] https://tools.ietf.org/html/rfc2616#section-9.5
Change-Id: I2246ff5edfa6fccd9711b67efb73504fb1eb2653
Closes-bug: 1658800
Updates fake data generation such that network devices are created
with the required attribute 'name'. It had previously been called
'hostname', but the renaming in a recent commit had not been applied
to this tool, which is used for testing (and demos!).
Change-Id: I9bd859c1481fc8db02cac04d8005ee61370925e1
Closes-bug: #1660510
Parts of the code assumed that a network interface had an IP address,
however the network interface model did not manage IP addresses. This
code updates the appropriate database model and API schema to add an IP
address. The IP address has been defined as a required property.
In addition, the tests have been updated to ensure the response data
always includes the IP address and requests can include IP addresses.
Change-Id: I935b0d27248199d037a37468df821c12f0068c5a
Closes-bug: 1630296
The URLs defined in the routes have been updated to replace underscores
with hyphens. The tests have been updated to account for this change. In
addition tests have been added to ensure that the URLs meet the
API-WG guidelines [1], namely that they contain only lowercase characters
and do not contain punctuation, with the exception of hyphens.
[1] https://specs.openstack.org/openstack/api-wg/guidelines/naming.html
Change-Id: I77c02de2973e12c6fbcdcc925fa6686aa1f7a094
Closes-Bug: 1657889
Naming of network endpoints should be consistent:
/networks
/net_interfaces -> /network_interfaces
/netdevices -> /network_devices
Also corresponding classes, methods and DB table names should follow the
changed endpoint naming.
Closes Bug: 1643356
Change-Id: Ib2b9d8fe11145944d050b55f2f965f1a90002639
- Currently we are not properly checking or normalizing
variables (data) during creates or responses. This fixes
this by adding proper schema for data and also adding
some code for data normalization. For data normalization
we dont care to match the properties, since its not easy
to have a set format of data, therefore, we have pattern
properties such that any key:value object is allowed.
Change-Id: I1c476f89046a6cfc996837cb8204b17d8f93ca61
We add the necessary Paste filters for Keystone middleware and context
usage inside of Craton so that operators can use Keystone for Identity
and Access Management of Craton.
We also clean up our Keystone Context Middleware to access the
attributes that the keystonemiddleware library set on a request object
so we can determine if the token sent has been validated and what the
roles the user own.
Due to the effort exerted to get this working, this also adds
documentation around how to set up Craton to use Keystone for identity
and adds a tiny script in the `tools/` directory so that one can explore
the API interactively with keystone authentication.
Change-Id: Ida86881f1e6097b1a1f6dec912f094c53c2f4cc2
Previously, we were requiring X-Auth-Project headers for authentication
and a project_id attribute in JSON bodies. This allows for several
things.
Closes-bug: 1628845
Closes-bug: 1628917
Change-Id: I40d0e8b6a3260ff11c493268c8a0553eba62512b
When looking at integrating craton with Keystone, it became apparent
that we could not store project_id's from Keystone in Craton's Database.
All of our project_id columns are Integers and Keystone uses UUIDs by
default. As a result, we've switched Craton to use UUIDs for Project IDs
and tweaked the docker tooling and install documentation.
Change-Id: I4b84f6ac23ffd128e0060fd929c6f0c60ee16d52
Closes-bug: 1634191
- Adds a unique constraint to ensure only unique
names are allowed for networks per project_id per region.
Closes Bug: 1623100
Change-Id: Ie2ffd21fe58eb335a68599dfefd4566aeac4c3b2
We need to update setup.cfg, docker_run.sh and install guide
to update the entry points for craton api server.
Change-Id: Ia1ae1a4f9467b352b45ba53f93eca0751191cdbb
Closes-bug: #1619076
- Since its not just inventory, and the service
now includes everything craton the file names
should reflect the same.
Fixes Bug: 1630784
Change-Id: I0fb77942a92bbf2aacbf2c4a7d0c3104ed8bebfe
This change removes the following boilerplate:
* Most filter setup, along with output to a JSON response, is now
captured in the new craton.api.v1.base.filtered_context decorator.
* Provides standard exception to HTTP status code mapping using the
new craton.api.v1.base.http_codes decorator.
There are additional opportunities for boilerplate removal, especially
in scenarios where we have multiple ways of looking up a specific cell
or region; but this boilerplate is not addressed in this fix.
This change does not implement related opportunities for cleanup, such
as import reordering or doc strings. These should be in separate
changes, per discussion on Gerrit.
In addition, this change now returns 204 upon DELETE in the API; and
adds version info to requirements.txt for Flask to conform to Jenkins
gate check.
Change-Id: I7da8e9620c2fb07ebd3953cc6d76d47c08651fe7
Partial-Bug: 1623095
Closes-Bug: 1628050
- Adds a client side script to simply call ansible
inventory endpoint. The script is an example of how
to run playbooks using Craton inventory.
Change-Id: Ied8059fd518f044f244ef5edc38c06dc9a235b33
Fixing file path on sample config and script
to ensure docker always finds the right path
when run under any evnironment.
Change-Id: Ida678f152eb585be54031b6ecba3aaf64574be6f
In order to try or test the api it is very
useful to have some dataset ready in the inventory.
This patch adds a script to generate the fake data.
Also, adds a comment on the Dockerfile to inform users
to use it with the image.
Change-Id: I3401b2b1ff4784a2505cd9d14c56e3d53ca8d5a0
Currently, docker_run.sh is not executable by default,
this patch will make it an executable file.
Change-Id: I81d5eadd760042382d55bd0517f1ecb7424e4c13
Closes-Bug: #1607418
Andreas Jaeger