Commit Graph

196 Commits

Author SHA1 Message Date
guolimin.lc 1d0bdedd51 Fix deprecated zuul queue syntax
Change-Id: I34d1f3d596caf9f7588dffb62eba36452c09868a
2023-11-02 17:12:36 +08:00
Ghanshyam Mann cf5762829e Fix hacking min version to 3.0.1
flake8 new release 3.8.0 added new checks and gate pep8
job start failing. hacking 3.0.1 fix the pinning of flake8 to
avoid bringing in a new version with new checks.

Though it is fixed in latest hacking but 2.0 and 3.0 has cap for
flake8 as <4.0.0 which mean flake8 new version 3.9.0 can also
break the pep8 job if new check are added.

To avoid similar gate break in future, we need to bump the hacking min
version.

- http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014828.html

Change-Id: Icb5106877ee4e323314009a1e69842d17302ac65
2020-05-12 20:59:19 -05:00
Jeremy Freudberg c45a707373 Ussuri contributor docs community goal
- Updated CONTRIBUTING.rst per goal guidelines
- The repository does not have docs, so rely on main sahara docs

Change-Id: I1317b2acdbd3ae56bb9f6db305e6404969da3644
2020-04-22 13:04:47 -04:00
Andreas Jaeger 117326ee16 Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Change-Id: I4abcc8f3ca66aece0c6726b1e1770f486492d3c8
2020-04-07 20:15:28 +02:00
Andreas Jaeger 8bc0f11366 Cleanup py27 support
Make a few cleanups:
- Remove python 2.7 stanza from setup.py
- Add requires on python >= 3.6 to setup.cfg so that pypi and pip
  know about the requirement
- Remove obsolete sections from setup.cfg:
  * Wheel is not needed for python 3 only repo
  * Some other sections are obsolete
- tox.ini: Move constraints into deps, remove install_command since
  the default is fine

Change-Id: Ifc4c123f30451ab6844d8b2f704b930760ebcef8
2020-04-07 20:15:05 +02:00
Jeremy Freudberg f2a8f4aa7d Fix oozie building
http->https

Change-Id: Ibc1ac6fdaa4f9384dcb6c07acd3fd9aaaf0c8429
2020-04-07 13:11:42 -04:00
Ghanshyam Mann 29feb954d0 [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in ussuri cycle.

sahara-extra is ready with python 3 and ok to drop the
python 2.7 support.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: Ib096b763835837544da0cd6e013de83b3125889b
2019-11-17 14:27:05 +00:00
pengyuesheng 7d09ba71ca Update the constraints url
For more detail, see http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006478.html

Change-Id: I5b3865609f8d03af02ebebfb120efe5604bb04af
2019-09-29 10:37:36 +08:00
Luigi Toscano b7c24df858 Replace git.openstack.org URLs with opendev.org URLs
Thorough replacement of git.openstack.org URLs with their opendev.org
counterparts.
(bonus point: fix also a leftover github URL.)

Change-Id: Ie24652d525b1d2859594b271500a4d32aa5b2d34
2019-04-20 13:51:25 +02:00
OpenDev Sysadmins 633b026870 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:34:51 +00:00
jacky06 40a9ccae89 Update the home-page link to latest
Change-Id: I5723b455ae9d256fde9d4a349b6f4f9badbf22a4
2018-11-25 16:28:38 -05:00
Luigi Toscano 71e4c11cbe Update devel info: mailing list
The new openstack-discuss mailing list is going to replace few
mailing lists, including openstack-dev.

Change-Id: Ib64c192f2dcef47929de3c1a31554c719974c048
2018-11-21 18:47:50 +01:00
zhouxinyong d792bed39f Replacing the link in HACKING.rst
Change-Id: Ibc29760e05841c6f3cf7c548601fdb63b3f671f6
2018-11-16 08:55:39 +08:00
Doug Hellmann 77c5fb0255 import zuul job settings from project-config
This is a mechanically generated patch to complete step 1 of moving
the zuul job settings out of project-config and into each project
repository.

Because there will be a separate patch on each branch, the branch
specifiers for branch-specific jobs have been removed.

Because this patch is generated by a script, there may be some
cosmetic changes to the layout of the YAML file(s) as the contents are
normalized.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: Ia71e9ef1b97568715ef07ab02b0ff28d39559fee
Story: #2002586
Task: #24332
2018-08-20 22:00:08 +00:00
Luigi Toscano 5370554703 Fix the post-publishing job: correct src directory
Use the (hopefully correct) directory where the artifacts are
really stored.
This should fix the newly merged sahara-extra-publish-artifacts
job, which dit not test proper testing (sorry).

Change-Id: I3d6f9fd7c39633c6b3ba24bab369f50ba50d8fbc
2018-07-13 21:51:53 +02:00
Luigi Toscano 90c2430bc9 Add native Zuul v3 build and publishing jobs
The new jobs replace (and move in-tree):
- legacy-sahara-extra-artifacts
- legacy-post-sahara-extra-artifacts

Unfortunately the new publish job can't be tested
beforehand, but we can fix it later.

Story: 2001686
Task: 8660

Depends-On: https://review.openstack.org/581087
Change-Id: I5b1ad485dfb23ce164473569b6bbc77fb37871ac
2018-07-09 20:03:59 +02:00
Zuul c091d778b6 Merge "Host some patched jars in common artifacts" 2018-06-26 14:59:53 +00:00
Jeremy Freudberg d4e774245b Host some patched jars in common artifacts
Host these jars, and also provide tooling to recreate them.

These are needed for proper S3-as-a-datasource capability.

Change-Id: Ied75211044b8f51623d87d36aa0588632645eea3
2018-06-25 15:36:28 -04:00
Doug Hellmann 9940c4e78f fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: Id456fc88170c4466a434100934e150f403e7ecd9
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-06 15:27:01 -04:00
Zhuang Changkun c118e53a8e Add support to build hadoop-3.0.1 related package
Change-Id: I2ba39c3d84bdd0b07e9fbe58ef0c8bc655c303ac
2018-05-18 19:23:28 +08:00
Zhuang Changkun af6cac8e36 Add Hadoop 2.7.5 native libs to common artifacts
Download the binary format package of hadoop 2.7.5, and create
tarball with hadoop native libs (hadoop-2.7.5/lib/native)

Change-Id: I4eb9906a4cd4e1c41ff01a4b88fdb5a7199ecf5a
2018-04-26 16:00:02 +08:00
Zhuang Changkun ef7019c470 Add support to build hadoop-2.7.5 related package
Change-Id: I07a909431f6dce6f0ad723821f5c11386d363e95
2018-04-16 19:39:20 +08:00
Luigi Toscano 82fb1fd1f4 Migration to Storyboard
Sahara projects migrated to storyboard.openstack.org.
Replace the references to Launchpad.

Change-Id: I5133ff63b928c314672310622c4beb275010e388
2018-03-17 13:28:59 +01:00
Jeremy Freudberg 02ec7b9a62 Add Hadoop 2.8.2 native libs to common artifacts
Without these, users will encounter messages like "Unable to load
native-hadoop library for your platform...", which is no fun at all.

For posterity's sake, you can note that this artifact was built by
following the instructions given in the README of the "hadoop" element
in SIE, with the additional note that newer Linux distros do not
package protobuf 2.5.0 which must be built from source instead.

Change-Id: I51a9343bd5804254e36b4d52334644ac6f3f8cef
2018-02-16 17:55:55 +00:00
Shu Yingya 2eedcec728 Add support to build hadoop-2.8.2 related package
This patch include some modification to build hadoop 2.8.2
required packages.
1. Modify the hadoop-swiftfs/pom.xml to fix dependency error
   while building hadoop-openstack.jar. It is caused by change
   in hadoop upstream [1],[2]
2. Add hadoop version 2.8.2 into sahara-extra/tools/ to build
   related packages
3. make some changes on build-oozie.sh so that we don't need
   to replace hadoop.version in pom.xml. we can specify
   hadoop.version directly in the command.
4. Apply a workaround patch to oozie 4.3.0 to solve WebUI issue [3]
5. Insert a dependency into oozie/sharelib

[1]: fa440a39b7
[2]: 12aa184479
[3]: https://issues.apache.org/jira/browse/OOZIE-2533

Change-Id: I186cf5e01907cfab4a34507be464a0da170ebb5d
2018-02-08 13:20:50 +00:00
Luigi Toscano 558f100a37 Enable also building Oozie for 2.7.1
This should be enough to generated and copy the tarballs
to tarballs.openstack.org.

Change-Id: Ide850fc26d94c250983488105fab1f4dd43597ff
2018-01-17 13:11:31 +01:00
Jenkins dd99413cd3 Merge "Update tag link address in README" 2017-08-03 14:43:40 +00:00
zhangdebo1987 edbd4d6103 Update tag link address in README
Change-Id: Ifbdfe41b95db517803680e6504d6c24a0de9f409
2017-08-03 14:56:25 +08:00
blue55 3d4144b446 Modify a description of the parameters to function.
Change-Id: Ic6d9354df8ec180da91fcf88df15f2e054a9be11
2017-08-03 13:58:34 +08:00
Jeremy Freudberg 54ba24e1fa Add Hadoop native libs to common-artifacts
The Hadoop 2.7.1 native libs found on sahara-files host were actually
the wrong thing; additionally, we are no longer using that host.

Instructions for building these libs are found in the README of the
"hadoop" element in SIE.

Change-Id: I07ed1164266a2d2d2ad63ef12c8364935c9b95cc
Partial-Bug: #1705942
Related-Bug: #1304995
2017-07-23 23:56:33 +00:00
Luigi Toscano 6ec96264a4 Fix building oozie 4.2.0
Remove the reference to repository.codehaus.org, which is still
still references by the pom.xml file for oozie 4.2; see
https://issues.apache.org/jira/browse/OOZIE-2417

Change-Id: I7420cd27bd732932bd113f18060ca89c2a882ff8
2017-07-10 18:21:52 +02:00
OpenStack Proposal Bot 70090c2a94 Updated from global requirements
Change-Id: I90dc72e452cc1a00f22d54fc14d73bd454e01a94
2017-04-12 04:22:48 +00:00
OpenStack Proposal Bot 69f1cb0910 Updated from global requirements
Change-Id: Ibc8c7ce906fd8e4ba80ecfa1c596a81adc5a9ecb
2017-03-02 17:53:51 +00:00
ricolin f10d138814 [Fix gate]Update test requirement
Since pbr already landed and the old version of hacking seems not
work very well with pbr>=2, we should update it to match global
requirement.
Partial-Bug: #1668848

Change-Id: I70ab80335d0cc25ed31e2b9557e4363b6f152148
2017-03-02 19:58:09 +08:00
Tony Breeds 85fd5506fa Add Constraints support
To be co-installable with other OpenStack projects and have consistent
testing it's a good idea (tm) to use constraints when creating tox
environments.

Change-Id: I2996f6fb97f9511ab13e8f5c4460014b286df7a8
2016-12-21 13:37:36 +11:00
Jenkins d95ffe7014 Merge "Adds the lazy seek" 2016-11-28 15:32:29 +00:00
Flavio Percoco ac8b0480df Show team and repo badges on README
This patch adds the team's and repository's badges to the README file.
The motivation behind this is to communicate the project status and
features at first glance.

For more information about this effort, please read this email thread:

http://lists.openstack.org/pipermail/openstack-dev/2016-October/105562.html

To see an example of how this would look like check:

https://gist.github.com/b8a9835f9193c610b76acdf617aecc40

Change-Id: I4202a0e91d951debdf33ef5224d98bab21121aa1
2016-11-25 15:28:26 +01:00
OpenStack Proposal Bot a7af6c1367 Updated from global requirements
Change-Id: Ic1bd81b024d4de61cd02936f1691a9d715a60556
2016-11-09 04:24:59 +00:00
Ray Zhang ef3c1ab4d5 Adds the lazy seek
Can make the query couple of 10 times faster.

In order to debug a Presto query performance issue, I observed the
seeking in Sahara-extra is expensive and sometimes even unnecessary.
The best way to avoid the overhead and unnecessary calls of seeking
is to do it only when the client really needs the data.
After this changes, the same query in Presto able to run 30 times faster.
Both Presto and S3 clients have added the similar changes too.

Change-Id: I8586af0d481fd08d48620e699467280f7b93150a
2016-10-05 13:26:54 -07:00
ChangBo Guo(gcb) b2f6ad9168 Make all components in a list in README
This makes all comments listed in a unified format
and update with official repository links.

Change-Id: If4591c26414557c9eaa7c5ab237d16203b731624
2016-10-04 23:59:10 +08:00
Shu Yingya e7a84c6b82 remove section [files] in setup.cfg
The files section defines the install location of files
in the package using three fundamental keys: "packages",
"namespace_packages", and "data_files".
The "elements/*" in sahra-extra has been removed, we also
delete this section

Change-Id: Ifcc0904844d910eaa75ee5138853c74a480f47fe
2016-10-01 03:11:15 +08:00
Shu Yingya 2e516a87f7 Remove not exists dir in tox.ini
This dir has been cleaned after oslo.incubator deprecated.
But still exists in tox test configure file. Delet it.

I am also thinking that this project doesn't contain any
python file except "setup.py". Why do we still execute pep8
check? Can we remove it to reduce the work load of jenkins?
This result would looks like "openstack-dev/devstack".

Change-Id: Ib6d28fabeeab662c4b447c6a0027afa2b921ec54
2016-09-30 13:58:48 +08:00
Shu Yingya f01fb7b7a7 Modify the description of edp-examples
The edp-examples have been moved to sahara-tests repo.

Change-Id: I9a440a0388d56d881048afd652d7927b575543f5
2016-09-29 15:28:49 +08:00
Vitaly Gridnev 451423f8d7 fix process of copying common artifacts
Change-Id: Ie4790b3b1d126a976c6d6c613d137c67879f6eb4
2016-09-14 15:34:52 +03:00
Jenkins f939988140 Merge "migrate artifacts to sahara-extra" 2016-09-12 14:28:05 +00:00
Vitaly Gridnev c808fe5b4b migrate artifacts to sahara-extra
Change-Id: I00fc23eba9049002d6498b77636450384ae9dc02
2016-09-12 12:27:16 +03:00
Vitaly Gridnev 728a106985 fix broken integration with Swift when Kerberos enabled
related issue in hadoop jira [0]

[0] https://issues.apache.org/jira/browse/HADOOP-10326
Change-Id: Iecdff57454ff91187fba077af73211faf2100d57
2016-09-05 23:34:56 +03:00
Shu Yingya 3b389d6573 Trivial: Update home-page in setup.cfg
It opens the liberty home page. Update it!

Change-Id: Iaf2bffecb150d9f20b59ca7a49e244d635443265
2016-09-01 07:06:23 +08:00
Vitaly Gridnev 2e26674615 update readme for sahara-extra
Change-Id: I7f1a50ff040efa46e330d7f17715b423d0267e69
2016-09-09 13:38:21 +00:00
Jenkins 8cd6d7b0ca Merge "hadoop-swiftfs: Allow configuring container.tenant" 2016-06-23 21:02:49 +00:00