sahara

Commit Graph

Author	SHA1	Message	Date
Jeremy Freudberg	a449558ac0	S3 data source * Create S3 data source type for EDP * Support storing S3 secret key in Castellan * Unit tests for new data source type * Document new data source type and related ideas * Add support of S3 configs into Spark and Oozie workflows * Hide S3 credentials in job execution info, like for Swift * Release note Change-Id: I3ae5b9879b54f81d34bc7cd6a6f754347ce82f33	2018-07-02 14:27:46 -04:00
Jeremy Freudberg	d05bf66592	S3 job binary and binary retriever * Create common module for managing S3 job binaries * Add new dependency on botocore * Use common S3 library to create S3 job binary type for EDP * Use common S3 library to create S3 job binary retriever * Support storing S3 secret key in Castellan * Document new job binary type (and foreshadow the S3 data source type) * Unit tests for new code Change-Id: I6781203d802305446ba1418ed6999186db4dfe9b Partially-Implements: bp sahara-support-s3	2017-12-28 20:20:37 +00:00

Author

SHA1

Message

Date

Jeremy Freudberg

a449558ac0

S3 data source

* Create S3 data source type for EDP
* Support storing S3 secret key in Castellan
* Unit tests for new data source type
* Document new data source type and related ideas
* Add support of S3 configs into Spark and Oozie workflows
* Hide S3 credentials in job execution info, like for Swift
* Release note

Change-Id: I3ae5b9879b54f81d34bc7cd6a6f754347ce82f33

2018-07-02 14:27:46 -04:00

Jeremy Freudberg

d05bf66592

S3 job binary and binary retriever

* Create common module for managing S3 job binaries
* Add new dependency on botocore
* Use common S3 library to create S3 job binary type for EDP
* Use common S3 library to create S3 job binary retriever
* Support storing S3 secret key in Castellan
* Document new job binary type (and foreshadow the S3 data source type)
* Unit tests for new code

Change-Id: I6781203d802305446ba1418ed6999186db4dfe9b
Partially-Implements: bp sahara-support-s3

2017-12-28 20:20:37 +00:00

2 Commits