* Create S3 data source type for EDP
* Support storing S3 secret key in Castellan
* Unit tests for new data source type
* Document new data source type and related ideas
* Add support of S3 configs into Spark and Oozie workflows
* Hide S3 credentials in job execution info, like for Swift
* Release note
Change-Id: I3ae5b9879b54f81d34bc7cd6a6f754347ce82f33
Inefficient validation checks, to check unique object name Sahara
retrieves all entries from database.
Change-Id: I6494d3ac3f5d793ec4a47dce598dc42ca06a303e
Closes-Bug: 1212225
Add run_scheduled_job in base_engine, and implement it in oozie engine.
Implements bp: enable-scheduled-edp-jobs
Change-Id: I2a0b3724396b4bed5cd2a4bc1392f849eb902e3e
This change will allow data sources with urls of the form
"manila://share-id/path", similar to manila urls for job binaries.
The Sahara native url will be logged in the JobExecution, but
the true runtime url (file:///mnt/path) for manila shares will
be used in the cluster.
Partial-implements: blueprint manila-as-a-data-source
Change-Id: I0b43491decbe6cb0ec0b84314cf9b407b9e3fb4a
Now EDP engine is fully responsible on validation of data for
job execution.
Other changes:
* Removed API calls from validation to remove circular dependancy
* Removed plugins patching in validation to allow non-vanilla
plugins testing
* Renamed job_executor to job_execution
Change-Id: I14c86f33b355cb4317e96a70109d8d72d52d3c00
Closes-Bug: #1357512
The proposed fix compares input and output data sources. If they
have the same types and urls, a validation error is thrown.
Closes-bug: #1291380
Change-Id: I09956c2e29c1e12d9e5c28a5c7953f97197b3565
Rename the subdirectory and replace all instances
of "import savanna" with "import sahara" and all
instances of "from savanna" with "from sahara".
* Replaced mock patches like mock.patch('savanna...
* Updated config generator script
* Renamed entry points in setup.cfg
* Hacking checks also fixed
* Manual renaming in alembic scripts to force work migrations
* Fix doc building
* Renamed itests directories
* Some changes in gitignore
* Removed locale dir after rebase
Co-Authored-By: Alexander Ignatov <aignatov@mirantis.com>
Change-Id: Ia77252c24046c3e7283c0a7b96d11636020b949c
Partially implements: blueprint savanna-renaming-service
Jeremy Freudberg