Commit Graph

736 Commits

Author SHA1 Message Date
Zuul 2bfc66f176 Merge "DNM: test build with latest setuptools" 2023-11-03 07:46:17 +00:00
Zuul 3b386cb3eb Merge "setup.cfg: Replace dashes with underscores" 2023-11-02 01:53:47 +00:00
zhouxinyong cf6e9e6ec6 Fixes the several problems/errors caused by tox 3
Change-Id: If56cab52b26e5f00ac5738544bdb213a97af073d
2023-10-11 13:32:20 +08:00
zhouxinyong 528272d787 setup.cfg: Replace dashes with underscores
Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:

  UserWarning: Usage of dash-separated 'description-file' will not be
  supported in future versions. Please use the underscore name
  'description_file' instead

[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb

Change-Id: I5c64c191b98f88d48a6878e960a6b7694a185153
2023-06-12 15:28:22 +08:00
Ghanshyam Mann ace7eda8cd Drop lower-constraints.txt and its testing
As discussed in TC PTG[1] and TC resolution[2], we are
dropping the lower-constraints.txt file and its testing.
We will keep lower bounds in the requirements.txt file but
with a note that these are not tested lower bounds and we
try our best to keep them updated.

[1] https://etherpad.opendev.org/p/tc-zed-ptg#L326
[2] https://governance.openstack.org/tc/resolutions/20220414-drop-lower-constraints.html#proposal

Change-Id: I0fc96a3e9ce4626a9f137c1af02c738330235c43
2022-05-01 13:07:06 -05:00
Elod Illes 008b0d7e83 Disable auto-discovery for setuptools
With setuptools release 61.0.0 sahara-image-elements' package build
command (python3 setup.py sdist bdist_wheel) started to fail:

error: Multiple top-level packages discovered in a flat-layout:
['roles', 'elements', 'playbooks'].

This bug is mentioned in setuptools issue 3197 [1], and the suggested
workaround is to disable auto-discovery by adding 'py_modules=[]' in
setup.py.

To unblock the gate the following needed to be updated as well:
- requirements constraints
- fix lower-constraints job constraints (only the 1st constraints is
  used by pip, which was upper-constraints.txt from install command)

[1] https://github.com/pypa/setuptools/issues/3197

Change-Id: Ib2d3e296e6d405671594dfd630e381ae414a2709
2022-03-28 16:19:31 +02:00
Elod Illes c80594592a DNM: test build with latest setuptools
Change-Id: I0e489c0d1b0b1c641164d64f1bc0120d4b5da21e
2022-03-26 11:32:32 +01:00
Jeremy Freudberg ded80db266 Fix URL of Maven Central Repository
Change-Id: I7accc23d6e91ff4a525c31616033f76c07dec21e
Story: 2007996
Task: 40639
2020-08-10 18:21:16 +00:00
Luigi Toscano 2d0c3a8b98 Fix the buildimages jobs: call the ensure-tox role
... and use its output.

tox is not available by default on the images anymore.

Change-Id: If5ea0ffdfc51860842c19f0a6f238127d42eb292
2020-08-10 18:00:06 +02:00
Jeremy Freudberg 6a316ee9bf Ussuri contributor docs community goal
- Updated CONTRIBUTING.rst per goal guidelines
- The repository does not have docs, so rely on main sahara docs

Change-Id: Iacf22500cc86457c51afe49dd71a451a89085287
2020-04-22 13:05:04 -04:00
Andreas Jaeger 2921d50974 Cleanup py27 support
Make a few cleanups:
- Remove python 2.7 stanza from setup.py
- Add requires on python >= 3.6 to setup.cfg so that pypi and pip
  know about the requirement
- Remove obsolete sections from setup.cfg:
  * Wheel is not needed for python 3 only repo

Change-Id: Ie5bf4b5226a06d828ec24a6134d7ea8a3f45c8db
2020-04-17 01:12:32 +02:00
Luigi Toscano d0b1d24be4 Fix linting (both dib-lint and bashate)
- dib-lint: make sure every element has a README.rst file
- bashate: ignore E006 (line too long).

Change-Id: I53a334b05a30e98f1df19504cee616e3d145e9e3
2020-04-17 01:09:43 +02:00
Ghanshyam Mann 1e3f59084b [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in ussuri cycle.

sahara-image-elements is ready with python 3 and ok to drop the
python 2.7 support.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: I5680fecb667038e3fcededaa89df815c07a913c7
2019-11-17 14:27:07 +00:00
pengyuesheng aaf5ed69e1 Update the constraints url
For more detail, see http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006478.html

Change-Id: Iebe62db0490c522ee085008b4a9672addad54198
2019-09-29 10:34:10 +08:00
jorgefrancoibanez 24a0ea0195 Allow qemu-utils to be used to build image
This patch allows build sahara images in ubuntu/debian
distribution with qemu-utils package.

Change-Id: Ibfa5c2550898a0d36b51057f5cd2b85f434b57f5
Story: 2006505
Task: 36483
2019-09-10 21:03:10 -04:00
Luigi Toscano 36d2edc24d Replace git.openstack.org URLs with opendev.org URLs
Thorough replacement of git.openstack.org URLs with their opendev.org
counterparts.

Change-Id: I97409815d943ade7adce00168066ca6275bfd68a
2019-04-20 13:51:31 +02:00
OpenDev Sysadmins 3b3e33b1c7 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:31:58 +00:00
Luigi Toscano 48491edb72 archive-primary.cloudera.com -> archive.cloudera.com
The former is no more active.

Story: 2004936
Task: 29329
Change-Id: I260bf4cd3840ba2c44b85df87b89b717831b3b5c
2019-02-04 22:07:02 +01:00
Andreas Jaeger e9a16e949b Use template for lower-constraints
Small cleanups:

* Use openstack-lower-constraints-jobs template, remove individual
  jobs.
* Sort list of templates

Change-Id: If67b57ec2517eba2691c3cfcc6540813633ec25d
Needed-By: https://review.openstack.org/623229
2018-12-20 21:39:04 +01:00
Zuul dcf80439b7 Merge "Update the min version of tox to 2.0" 2018-12-12 17:35:50 +00:00
Luigi Toscano 29c817d217 Plain Ubuntu image are still based on Xenial
Even if we are not really testing plain images much, let's make clear
that we didn't test any hadoop stuff on bionic yet.

Change-Id: I684a98338945d4449e37d9f652bddddf827838bf
2018-12-03 11:57:58 +01:00
Luigi Toscano 578cb4824b firstboot: make rc-local start after cloud-init
The change is required because the cloud-init user (centos, ubuntu,
cloud-user, ...) is used in the firstboot code.
All distributions where vanilla can be deployed are based
on systemd.

Story: 2004479
Task: 28194

Change-Id: I9d8a626b84d5d3c2a91348895cded5fd32ded52a
2018-11-30 23:20:01 +01:00
Luigi Toscano 9851391c28 Update devel info: mailing list
The new openstack-discuss mailing list is going to replace few
mailing lists, including openstack-dev.

Change-Id: I4e6abbd29667f84a2ea07089010d1da1266ca7b7
2018-11-21 18:39:49 +01:00
Kim Bao Long aae9ed8be6 Update the min version of tox to 2.0
The verification comands used by tox need at least 2.0 version. This
patch aims to update tox version from 1.6 to 2.0 which should help with
local running of constraints targets.

Change-Id: I84a53543b50184b4e838cd6e597dfc80ec025588
2018-11-02 08:29:19 +00:00
Sean McGinnis 020d1bbcf5 Increment versioning with pbr instruction
With moving away from required milestone releases, the version numbers
calculated by PBR on the master branch will not work for those testing
upgrades from the last stable release. More details can be found in the
mailing list post here:

    http://lists.openstack.org/pipermail/openstack-dev/2018-October/135706.html

This is an empty commit that will cause PBR to increment its calculated
version to get around this.

PBR will see the following which will cause it to increment the version:

Sem-Ver: feature

Please merge this patch as soon as possible to support those testing
upgrades.

Change-Id: I0ec6d6877f3518f267a977effb4e9d32a7300b89
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2018-10-12 13:09:54 -05:00
Doug Hellmann 2f0fbdffb4 import zuul job settings from project-config
This is a mechanically generated patch to complete step 1 of moving
the zuul job settings out of project-config and into each project
repository.

Because there will be a separate patch on each branch, the branch
specifiers for branch-specific jobs have been removed.

Because this patch is generated by a script, there may be some
cosmetic changes to the layout of the YAML file(s) as the contents are
normalized.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: Ia20404b70cb5589b8ffacff1d7de39b14566987c
Story: #2002586
Task: #24332
2018-08-20 22:11:07 +00:00
Luigi Toscano 5eeec72859 Use the correct "or" operator to check the spark version
It clearly slipped through the review...

Story: 2003281
Task: 24222
Change-Id: I094045a665a4441029fa4930e278e94d0cff50f8
2018-08-02 09:58:34 +02:00
Telles Nobrega 34e8f4a27a Updating Storm version list
We are adding Storm 1.2.0 and 1.2.1 and removing 0.9.2

Change-Id: I7e82464dda5d7d9f03e634eef814c63b661bb0c4
2018-07-12 15:24:42 -03:00
Zuul 93f57b7e3c Merge "Adding Spark 2.3" 2018-07-11 23:22:29 +00:00
Nguyen Hung Phuong 2d86783ad2 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I204e381701c91e59558eee57efac7ae7566acf76
2018-07-11 00:15:08 +00:00
Jeremy Freudberg 7910521a7e Totally rewrite s3_hadoop
Remedying of patchings, version conflicts, classpath issues, etc.

ALSO: Switch the Hadoop libraries used on the Spark standalone plugin to
Hadoop 2.7.3. The version was previously 2.6.5, to match Cloudera's
so-called "Hadoop 2.6.0", but in fact this concordance is not at all
necessary...

Change-Id: Iafafb64fd60a1ae585375a68173c84fbb82c7e1f
2018-07-02 11:51:28 -04:00
Telles Nobrega 0421dfb8e5 Adding Spark 2.3
Adding ability to create Spark 2.3

Change-Id: I5fcfd050121b563960a48a3c8dbad9dda19b2a3e
2018-06-26 09:14:44 -03:00
Zhuang Changkun 83224a6c5e Add support to build 3.0.1 image
Change-Id: I1d90099b22ed9a0ede08df45059e6d7848c237b8
2018-06-08 14:11:37 +08:00
Luigi Toscano 5d05a3db2f Remove CentOS6/RHEL6 support
Deprecated in Pike, and removed from diskimage-builder for a while.

Story: 2001996
Task: 19626
Change-Id: I9b4eab56fcbfcb2e97805fba805eafeeafa5d540
2018-05-15 18:22:50 +02:00
Zuul aa3ff84f2b Merge "Add support to build 2.7.5 image" 2018-05-10 14:41:00 +00:00
Luigi Toscano 30717f2516 Fix (again) the detection of scala version (now https)
The www.scala-lang.org switched to https and the curl call did not
account for this scenario, so it failed.
Switch the address to https and as extra measure add also the -L
argument, which forces curl to follow the redirects.

Switch also the download URL to https.

Story: 2001964
Task: 15089
Change-Id: I82d3dd0bd000284fb5404483bf3ee34a6fbb147b
2018-05-04 17:55:50 +02:00
Zhuang Changkun 5707f5cc5a Add support to build 2.7.5 image
Change-Id: Ia16692632cbe6380d4021dd031eed09362873905
2018-04-17 19:26:08 +08:00
Luigi Toscano ba498abc7d Zuul roles: allow to build only a specific image
Useful to generate a single image on-the-fly
and use it for scenario tests
(or simply useful for a refactor of the rest of the
build-images script).

Change-Id: Ica615e9e7d93b38fd12189a9f97d26ca62d84a3e
2018-03-30 15:33:35 +02:00
Doug Hellmann af81b26545 add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: I00b41e7e3bcd86677084243917a205c703097674
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-03-22 18:14:51 -04:00
Luigi Toscano d987717768 Migration to Storyboard
Sahara projects migrated to storyboard.openstack.org.
Replace the references to Launchpad, including the bugs now
available as stories.
Fix a reference to github.
Change-Id: Iadba69efc1e310b6a19463d3398bf5c6549acd73
2018-03-17 14:40:54 +01:00
Zuul ce56817439 Merge "Add support to build 2.8.2 image" 2018-03-15 14:40:31 +00:00
Elise Gafford 859baa5b0a Preload soci-mysql and soci on RHEL7 images
Change-Id: I00d78b45dffd785e956058ca7d76fb9e32bfac60
Closes-bug: #1754313
2018-03-09 20:49:35 -03:00
Shu Yingya 12efcf84e2 Add support to build 2.8.2 image
Also tweak Hive a bit and refer to artifacts in a new (but not totally
ideal) location.

Co-Authored-By: Jeremy Freudberg <jeremyfreudberg@gmail.com>
Change-Id: I3a25ee8c282849911089adf6c3593b1bb50fd067
2018-03-01 09:51:01 +08:00
Luigi Toscano 8cdff5d71f Update the links to artifacts (removing sahara-files too)
The canonical location for the artifacts going on is
tarballs.openstack.org/sahara-extra/, so fix the link to use that
and also use https.

Moreover, since the last tarball required for building images
is available on tarballs.openstack.org, remove the last references
to sahara-files for artifacts and documentation
(the new location was used already for a while in few places).
there are still few references to sahara-files,
but they are all about CentOS6 which is no more supported
by diskimage-builder, and should be removed separately.

Change-Id: Iab5a4d50a0abc6ab278837b6a9efd5e30f31c44a
2018-02-27 12:14:09 +01:00
Telles Nobrega 6229ee0de9 Disables CA checking for Ambari on Centos/RHEL
The default SSL certificate, which is generated when the Ambari server is
installed, is invalid.

We are disabling check for now.

Change-Id: Ifcbc931e2ca23cb1fe221d509f57750e7e060aea
Partial-bug: #1748507
2018-02-13 10:43:11 +01:00
Zuul 8a9185373f Merge "Revise s3_hadoop" 2018-02-08 18:47:14 +00:00
Jeremy Freudberg 3ee20cbc09 Revise s3_hadoop
* Handle Hadoop classpath better
* Include proper support for Spark classpath
* Formally limit element's use to Vanilla and Spark

Change-Id: I65abd7e375dba11599a4ab943d24f878235cd71d
Closes-Bug: #1727757
Closes-Bug: #1728061
2018-02-07 05:58:44 +00:00
Zuul a9ac718020 Merge "Adding fuse package to Ambari" 2018-02-02 16:56:10 +00:00
Zuul 3be2c3273f Merge "Zuul: Remove project name" 2018-02-02 01:58:49 +00:00
James E. Blair 3c472542ae Zuul: Remove project name
Zuul no longer requires the project-name for in-repo configuration.
Omitting it makes forking or renaming projects easier.

Change-Id: I2604716833cb0b1ac64facf3ae09236ad7d56fb9
2018-02-01 06:18:25 -08:00