flake8 new release 3.8.0 added new checks and gate pep8
job start failing. hacking 3.0.1 fix the pinning of flake8 to
avoid bringing in a new version with new checks.
Though it is fixed in latest hacking but 2.0 and 3.0 has cap for
flake8 as <4.0.0 which mean flake8 new version 3.9.0 can also
break the pep8 job if new check are added.
To avoid similar gate break in future, we need to bump the hacking min
version.
- http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014828.html
Change-Id: Icb5106877ee4e323314009a1e69842d17302ac65
- Updated CONTRIBUTING.rst per goal guidelines
- The repository does not have docs, so rely on main sahara docs
Change-Id: I1317b2acdbd3ae56bb9f6db305e6404969da3644
Make a few cleanups:
- Remove python 2.7 stanza from setup.py
- Add requires on python >= 3.6 to setup.cfg so that pypi and pip
know about the requirement
- Remove obsolete sections from setup.cfg:
* Wheel is not needed for python 3 only repo
* Some other sections are obsolete
- tox.ini: Move constraints into deps, remove install_command since
the default is fine
Change-Id: Ifc4c123f30451ab6844d8b2f704b930760ebcef8
Thorough replacement of git.openstack.org URLs with their opendev.org
counterparts.
(bonus point: fix also a leftover github URL.)
Change-Id: Ie24652d525b1d2859594b271500a4d32aa5b2d34
The new openstack-discuss mailing list is going to replace few
mailing lists, including openstack-dev.
Change-Id: Ib64c192f2dcef47929de3c1a31554c719974c048
This is a mechanically generated patch to complete step 1 of moving
the zuul job settings out of project-config and into each project
repository.
Because there will be a separate patch on each branch, the branch
specifiers for branch-specific jobs have been removed.
Because this patch is generated by a script, there may be some
cosmetic changes to the layout of the YAML file(s) as the contents are
normalized.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: Ia71e9ef1b97568715ef07ab02b0ff28d39559fee
Story: #2002586
Task: #24332
Use the (hopefully correct) directory where the artifacts are
really stored.
This should fix the newly merged sahara-extra-publish-artifacts
job, which dit not test proper testing (sorry).
Change-Id: I3d6f9fd7c39633c6b3ba24bab369f50ba50d8fbc
The new jobs replace (and move in-tree):
- legacy-sahara-extra-artifacts
- legacy-post-sahara-extra-artifacts
Unfortunately the new publish job can't be tested
beforehand, but we can fix it later.
Story: 2001686
Task: 8660
Depends-On: https://review.openstack.org/581087
Change-Id: I5b1ad485dfb23ce164473569b6bbc77fb37871ac
Host these jars, and also provide tooling to recreate them.
These are needed for proper S3-as-a-datasource capability.
Change-Id: Ied75211044b8f51623d87d36aa0588632645eea3
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: Id456fc88170c4466a434100934e150f403e7ecd9
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Download the binary format package of hadoop 2.7.5, and create
tarball with hadoop native libs (hadoop-2.7.5/lib/native)
Change-Id: I4eb9906a4cd4e1c41ff01a4b88fdb5a7199ecf5a
Without these, users will encounter messages like "Unable to load
native-hadoop library for your platform...", which is no fun at all.
For posterity's sake, you can note that this artifact was built by
following the instructions given in the README of the "hadoop" element
in SIE, with the additional note that newer Linux distros do not
package protobuf 2.5.0 which must be built from source instead.
Change-Id: I51a9343bd5804254e36b4d52334644ac6f3f8cef
This patch include some modification to build hadoop 2.8.2
required packages.
1. Modify the hadoop-swiftfs/pom.xml to fix dependency error
while building hadoop-openstack.jar. It is caused by change
in hadoop upstream [1],[2]
2. Add hadoop version 2.8.2 into sahara-extra/tools/ to build
related packages
3. make some changes on build-oozie.sh so that we don't need
to replace hadoop.version in pom.xml. we can specify
hadoop.version directly in the command.
4. Apply a workaround patch to oozie 4.3.0 to solve WebUI issue [3]
5. Insert a dependency into oozie/sharelib
[1]: fa440a39b7
[2]: 12aa184479
[3]: https://issues.apache.org/jira/browse/OOZIE-2533
Change-Id: I186cf5e01907cfab4a34507be464a0da170ebb5d
The Hadoop 2.7.1 native libs found on sahara-files host were actually
the wrong thing; additionally, we are no longer using that host.
Instructions for building these libs are found in the README of the
"hadoop" element in SIE.
Change-Id: I07ed1164266a2d2d2ad63ef12c8364935c9b95cc
Partial-Bug: #1705942
Related-Bug: #1304995
Remove the reference to repository.codehaus.org, which is still
still references by the pom.xml file for oozie 4.2; see
https://issues.apache.org/jira/browse/OOZIE-2417
Change-Id: I7420cd27bd732932bd113f18060ca89c2a882ff8
Since pbr already landed and the old version of hacking seems not
work very well with pbr>=2, we should update it to match global
requirement.
Partial-Bug: #1668848
Change-Id: I70ab80335d0cc25ed31e2b9557e4363b6f152148
To be co-installable with other OpenStack projects and have consistent
testing it's a good idea (tm) to use constraints when creating tox
environments.
Change-Id: I2996f6fb97f9511ab13e8f5c4460014b286df7a8
Can make the query couple of 10 times faster.
In order to debug a Presto query performance issue, I observed the
seeking in Sahara-extra is expensive and sometimes even unnecessary.
The best way to avoid the overhead and unnecessary calls of seeking
is to do it only when the client really needs the data.
After this changes, the same query in Presto able to run 30 times faster.
Both Presto and S3 clients have added the similar changes too.
Change-Id: I8586af0d481fd08d48620e699467280f7b93150a
The files section defines the install location of files
in the package using three fundamental keys: "packages",
"namespace_packages", and "data_files".
The "elements/*" in sahra-extra has been removed, we also
delete this section
Change-Id: Ifcc0904844d910eaa75ee5138853c74a480f47fe
This dir has been cleaned after oslo.incubator deprecated.
But still exists in tox test configure file. Delet it.
I am also thinking that this project doesn't contain any
python file except "setup.py". Why do we still execute pep8
check? Can we remove it to reduce the work load of jenkins?
This result would looks like "openstack-dev/devstack".
Change-Id: Ib6d28fabeeab662c4b447c6a0027afa2b921ec54