Commit Graph

135 Commits

Author SHA1 Message Date
Luigi Toscano f3fea3abb7 Fix Flask 2.1 compatibility, disable unstable job
Since Flask 2.1, the same blueprint cannot be registered multiple
times with different url_prefix values, unless the 'name' parameter
is set. Otherwise the following error is reported: "The same blueprint
cannot be registered with the same name. Use name= when registering
to specify a unique name."

At the same time, disable the openstack-ansible job,
which started to fail again, but at this point the priority
is to unblock the gates.

Change-Id: I50df2fec3fd312082a14ef2a5b6774a2515afb44
2022-04-15 17:17:48 +02:00
ruifaling 5d2f114b42 delete validation error cluster
Change-Id: I1ec693c17a43c09383fa83556cd3275b75ca90fd
story: 2008898
task: 42465
2021-05-17 01:16:03 +00:00
Ghanshyam Mann 9a69cc52b6 [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:

1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.

2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.

Also replace policy.json to policy.yaml ref from doc and tests.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: Ib2101f13171940857fe81f64dd9798dfe489743a
2021-03-30 04:02:11 +00:00
Luigi Toscano b95503f38f Use authorize instead of enforce for policies
After fully implementing policies in code, the authorize method
can be safely used and its a safeguard against introducing
policies for some API which are not properly defined in the code.

Change-Id: I499f13c34027b217bf1de905f829f36ef919e3b8
2019-03-08 15:26:58 +01:00
Zuul 193a604839 Merge "Declare APIv2 stable and CURRENT" 2019-03-07 16:04:25 +00:00
Zuul f5e047ec2c Merge "Fixing policies inconsistencies" 2019-03-06 20:01:33 +00:00
Telles Nobrega f422b31b81 Fixing policies inconsistencies
Fixing some jobs/job-templates/job-executions policy inconsistencies.

Change-Id: I05223981bd44d5f467808576113c29fbb37a3191
Story: #2004675
Task: #28666
2019-03-06 13:50:28 -03:00
Jeremy Freudberg 1c16c48b71 Fix version discovery for Python 3
bytes issue

Change-Id: I27bf5f0a0a9f5afb1e06190c9f8a4140c215f877
2019-01-16 15:02:28 -05:00
Jeremy Freudberg a83c97f4b0 Declare APIv2 stable and CURRENT
Change-Id: Ic551d77f83bc746932fce7cd70d889b77717d24f
2019-01-10 10:10:55 -05:00
Jeremy Freudberg 33489a1f9f Give the illusion of microversion support
Understand and react to microversions in accordance with
http://specs.openstack.org/openstack/api-wg/guidelines/microversion_specification.html

The actual mechanism allowing for new microversions of APIv2 will come
later.

Story: 2002178
Task: 20044

Change-Id: I2b664189e45ac4ffd02c3a176787b4bfb78b3871
2019-01-10 09:54:56 -05:00
Jeremy Freudberg 69d74c1a66 Some polish for APIv2
- update_keypair now only in v2 schema
- tenant_id->project_id in cluster provision steps
- tenant_id->project_id in referenced job binaries in job templates
- proper check for job template existence, to fail early (as intended)
- hadoop_version->plugin_version for query string filter
- unbreak some data source stuff (related to tenant_id->project_id)
- fix omission of show_progress from cluster GET query string whitelist
- job_id->job_template_id for v2 jobs
- add missing release note info for strict query string checking
- release notes for all the rest

Change-Id: Idea117c406b5ab9b8d85ccf8adb175053416d6ff
Story: 2004505
Task: 28822
2019-01-10 10:01:04 -03:00
Zuul 71efb19ebc Merge "API v2: fix "local variable 'c' referenced before assignment"" 2019-01-09 22:47:42 +00:00
Zuul 78090dee69 Merge "APIv2 - Fix 500 on malformed query string on" 2019-01-09 22:20:40 +00:00
Luigi Toscano 4bfd800ff4 API v2: fix "local variable 'c' referenced before assignment"
The subject explain it all.

Change-Id: If35d56e010d203c96769e7123414a6486d9b789f
2019-01-09 20:17:13 +01:00
Telles Nobrega 7105a891c2 APIv2 - Fix 500 on malformed query string on
In order to improve return information to clients we are adding a check
to verify parameters before each call and return a more appropriate
message to the users.

Change-Id: I9923601d0903e415a3fe30bec9bdc8fc34b91ff6
Story: #2004506
Task: #28228
2019-01-09 15:52:22 -03:00
Telles Nobrega 9bf50ce571 APIv2 Changing return payload to project_id
As part of the APIv2 work we need to change all tenant_id references
to project_id on the return payload.

Story: #2004505
Taks: #28227

Change-Id: I94bca161aa4f7bdd56d5528bae92fa81af188a43
2018-12-12 07:09:51 -03:00
Telles Nobrega bb7c039936 Another small fix for cluster creation on APIv2
Small fix that was blocking creation of multiple clusters using api v2.

Change-Id: I6c1db62a4bee3d1b064333b7b2d8b8e2d6ad50f1
2018-08-03 14:54:07 -03:00
Telles Nobrega 76549404c4 Fixing cluster creation on APIv2
We missed this detail on the review. Right now the creation of a single cluster
fails.

Change-Id: I56a94d9045284259b0f0a8b3998106b754187c4f
2018-07-31 14:26:01 -03:00
Zuul 1e0276926f Merge "Switch make_json_error back to being a function" 2018-07-12 13:16:31 +00:00
Jeremy Freudberg 9aab4b5e47 Switch make_json_error back to being a function
This was originally a function, and then I changed it to a dict because
Flask complained. Now Flask complains that is a dict, so change it back
to being a function.

Change-Id: Iec30b4cf5023be711ac070def00b77e91978d992
2018-07-11 10:44:45 -04:00
Jeremy Freudberg cb0e5ad7ca Final fixup to APIv2 responses
Some inconsistencies still remained.

Change-Id: Ic54af5433f6498488f457d8e49dae74e684e2bd2
Story: 1745294
Task: 8392
2018-07-09 12:00:43 -04:00
Corey Bryant 72b493a40e Use register_error_handler to register make_json_error
error_handler_spec[None] doesn't exist by default in flask
versions >= 1.0. It doesn't appear to be the right way to
register an error handler anyway, assuming that's what it is
doing. Use register_error_handler instead.

Change-Id: I6a56e1edf277c5652a876658f8cfbb5550a73ada
Story: 2002617
Task: 22246
2018-06-19 15:18:54 -04:00
Zuul b7c90ff987 Merge "Improve force delete" 2018-06-13 01:04:37 +00:00
Jeremy Freudberg ed59cfef84 Improve force delete
* Drop use of stack abandon: just use a regular delete instead
* Return stack name from force delete API call

Change-Id: I33ee7323fade1b237957abb8f7c79b87eb20148f
2018-06-12 14:20:39 -04:00
Jeremy Freudberg f96709a59e [APIv2]Consolidate cluster creation endpoints
Creation of a single cluster and creation of multiple clusters will now
share an API endpoint (in the APIv2 case). More specifically, the
original single-cluster endpoint will accept a `count` parameter in the
request and the multiple-cluster endpoint has been removed.

We can make this kind of change because APIv2 is still experimental.

Also, when creating multiple clusters, the response will now contain
all details about the clusters; previously, the response simply gave
cluster IDs.

Change-Id: I90faf4956a8ea4b4ae31a29382732771fdfddecb
Story: 2002099
Task: 19777
2018-06-04 15:31:39 -04:00
Zuul ddfdb0c925 Merge "Fix Flask error_handler_spec" 2018-01-25 22:15:13 +00:00
Zuul 80eb38629e Merge "Various server-side fixes to APIv2" 2018-01-25 22:15:12 +00:00
Zuul e5a7c9035f Merge "Tweak Sahara to make version discovery easier" 2018-01-25 19:14:29 +00:00
Jeremy Freudberg b1503b36c4 Tweak Sahara to make version discovery easier
Or at least, tweak Sahara to be more accomodating to keystoneauth's way
of doing version discovery.

* Don't require auth to do version discovery
* Make project ID in URL optional for APIv1 (and also for APIv2, but
  don't go around advertising that... *wink*)

Change-Id: Idb6f734aee26cab5bd629963a66ba01c92760864
Closes-Bug: #1744350
2018-01-25 05:56:15 +00:00
Jeremy Freudberg ccb9167e18 Various server-side fixes to APIv2
* Check if plugin_version key is in dict before doing something to it
* Fix references to non-existent policies
* Generally, correct how some responses get tweaked (in many cases the
  old way was throwing an error)
* Fix an incorrect schema

Change-Id: I6b4802a614e4b58678343f12856dd531827dc7b2
2018-01-25 05:23:22 +00:00
Jeremy Freudberg 87e176f882 Fix Flask error_handler_spec
The error_handler_spec is supposed to be a dict of dict of dicts, but
we had it as a dict of dict of functions, for unknown (historical?)
reasons.

Fix that, so that errors which occur in the Sahara API, but are not
otherwise caught by `sahara.utils.api.Rest.route` get handled correctly
by Flask instead of leading to some arcane error far down the stack.

Change-Id: I1e9d5f3fa00308baa7eee101c1f3b5a666cae4aa
Closes-Bug: #1745236
2018-01-24 21:42:40 +00:00
Jeremy Freudberg 6850bb8ba8 Force deletion of clusters
* Add force delete operation to engine using stack abandon
* Basic unit tests for force delete
* Necessary removal of "direct engine" code
* Change schema of APIv2 cluster delete to allow force delete
* Unit tests for new cluster delete schema
* Basic docs about force delete
* Release note

bp sahara-force-delete
Partial-Bug: #1647411

Change-Id: Ida72677c0a4110cb78edf9d62d8330cd4608ff76
2018-01-06 17:18:21 +00:00
Telles Nobrega b001ef2a55 Decommission of a specific node
Adding the option to decommission specific node from cluster.

Partially implements bp: decommission-specific-instance

Change-Id: I1a858fecc1b32f91c76aea6db14e0d5a419211d1
2018-01-02 15:17:51 -03:00
Jeremy Freudberg 84b77958a1 [APIv2]Add ability to export templates to APIv2
The feature should be supported in both API versions.

bp v2-api-experimental-impl
bp portable-node-group-and-cluster-templates

Change-Id: I102fb2ad16d0256e3a9aa364586332a13826cc90
Co-Authored-By: Iwona Kotlarska <pinus314@gmail.com>
2017-12-07 19:29:02 +00:00
Jeremy Freudberg 3525108d5a [APIv2]Nix custom OpenStack-Project-ID header
We don't need to be special/annoying. Other services whose APIs don't
need project ID in the URL do just fine trusting keystonemiddleware to
provide that info.

bp v2-api-experimental-impl

Change-Id: I8397ed7c134f0742de8c38466ed3f9035d8103d4
2017-12-03 03:13:56 +00:00
Zuul c546fa497e Merge "[APIv2] Remove job-binary-internal endpoint" 2017-12-02 15:52:08 +00:00
Marianne Linhares Monteiro bb1fcb3989 [APIv2] Remove job-binary-internal endpoint
By not including support for job binary internal in APIv2 at all, there
will be a clearer path towards the eventual removal of the JBI feature.

Since JBI will be an APIv1 exclusive, that means we can safely remove
support JBI upon APIv1's removal without affecting APIv2 users.

Change-Id: Iee70e9f6b4471b50105974c7c77169303c4159de
Partial-Implements: bp v2-api-experimental-impl
Partial-Implements: bp remove-job-binary-internal
2017-12-01 03:30:16 +00:00
Telles Nobrega 5d208ed486 Policy in code for Sahara
This patch is part of the community goal to move policies into code.

https://governance.openstack.org/tc/goals/queens/policy-in-code.html

Change-Id: I8f228ca40acc589b9aa41fc1e80a1c0308f179cf
2017-10-23 11:46:31 -03:00
Jenkins e8d071a8fb Merge "Add export of cluster templates" 2017-08-31 15:49:41 +00:00
Iwona Kotlarska 62a869ac1d Add export of cluster templates
Partially-Implements: bp portable-node-group-and-cluster-templates

This change adds functions to sahara to enable export of ct to JSON.

Change-Id: Ib5a2df4013840050b8034bcefd68040f508d0b31
2017-08-30 21:39:07 +02:00
Iwona Kotlarska 59abd1e301 Fix export of node group templates
Change-Id: Iee59b84765c50ec47370f4c72a1d59b24ac5ddb2
Depends-on: I7a2ef7e5cff70e6034c1222252fbf7c5c35a7e1c
2017-07-26 19:09:57 +02:00
Iwona Kotlarska b3d3793d15 Add export of node group templates
Partially-Implements: bp portable-node-group-and-cluster-templates

This change adds function to REST api to enable user to export node
group template to JSON, so that it can be later imported on another
deployment.

Change-Id: I7a2ef7e5cff70e6034c1222252fbf7c5c35a7e1c
2017-07-10 21:42:32 +02:00
Jenkins a1db392af7 Merge "[APIv2] Refactor job cancel operation" 2017-05-05 13:14:42 +00:00
Jenkins 00eae679d3 Merge "[APIv2] Refactor job refresh status" 2017-05-05 13:14:37 +00:00
Jenkins a0a953f004 Merge "[APIv2] Rename hadoop_version" 2017-05-04 14:44:15 +00:00
Shu Yingya 3c24a9b00e [APIv2] Refactor job cancel operation
Remove endpoint "/jobs/{job_id}/cancel" for cancelling a job.
we can request a cancelled state on a PATCH to the new /jobs/{job_id}
endpoint to make code more consistency.
The request body should be: {"info": {"status": "cancel"}}

Change-Id: I4a427d99e533979be42a84d388324ec2f933acc2
Partial-Implements: bp v2-api-experimental-impl
2017-05-04 12:44:46 +08:00
Shu Yingya e0477d161c [APIv2] Refactor job refresh status
The endpoint /jobs/{job_id}/refresh-status has been removed in favor
of using a GET on the new /jobs/{job_id} endpoint with parameter.
To refresh job status, we can GET on:
http://{host:ip}/v2/jobs/{job_id}?refresh_status=True

Change-Id: I8674a9f26512702f1732c0fefb230a1c0ab90e1e
Partial-Implements: bp v2-api-experimental-impl
2017-05-04 12:44:39 +08:00
Marianne Linhares Monteiro 21045d3c39 [APIv2] Rename oozie_job_id
oozie_job_id should be changed to engine_job_id.

This patch only changes json responses, the complete
change should be done when APIv2 is stable
and APIv1 deprecated. Along with this should be a
data model change as well.

Change-Id: I2ecfbb56d1e53d6b005ae6a52c70576238341da2
Partial-Implements: bp v2-api-experimental-impl
2017-04-27 17:02:13 -03:00
Jenkins 2a3f32be69 Merge "[APIv2] Refactor job execute endpoint" 2017-04-27 19:51:23 +00:00
lcsong 0bdd434add Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, _LW and _LC translation markers to simplify logging and to
avoid confusion with new contributions.

See:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html

It will be a giant patch if all these _LE, _LI, _LW and _LC being deleted
within one patch, so this patch only delete _LW,  _LC and _LE has been
handled in previous patch; deletion of _LI will be in the following patches
related with this one.

Change-Id: Ia696ee76f22e488b9a219305f4f220eca2161cee
2017-04-10 16:02:05 +08:00