Commit Graph

66 Commits

Author SHA1 Message Date
Ian Wienand bf7bcbcacd
Remove lower-constraints.txt
Per [1] lower-constraints aren't necessary any more, let's remove it.

[1] https://governance.openstack.org/tc/resolutions/20220414-drop-lower-constraints.html

Change-Id: I2688313c62c5fc3ceac1631c7aae3a7f5bb3ef7a
2023-03-23 09:12:15 +11:00
Clark Boylan 8bb9a37a0d Clean up tox.ini for tox v4
skipsdist now basically means don't install the project at all
(regardless of the usedevelop setting) which creates problems for dib's
entrypoints. Remove skipsdist so that entrypoints can be found. Also, we
remove basepython because this confuses tox v4 on whether or not the
python it wants is present.

Change-Id: I16388a8ad50483228d0b71745f11563f891249c0
2023-01-09 08:18:01 -08:00
Ian Wienand 2a25db9ee6
Start running dib-lint again
A recent change that didn't fail with hard-tabs made me realise we're
not running tox -e pep8 ... which means we're not running dib-lint
which should find this (and other things).

I couldn't pinpoint when this happened; maybe job config was never in
this repo.

Anyway, move the pylint and dib-lint/flake8 testing to the now
standard "linters" and update the linting job to
openstack-tox-linters.

It looks like pylint is very lightly used (came in with
I7e24d8348db3aef79e1395d12692199a1f80161a and we've never expanded any
testing).  Leave this alone for now, but probably it is not important
any more.

This revealed some issues; updated flake8
(Iaa19c36f8cab8482a01f764c588375db8e7d8be3) found some spacing issues
with keywords and an update to elrepo to match our standard bash
flags.

Change-Id: I45bf108c467f7c8190ca252e6c48450c2622aaf8
2022-09-21 07:56:05 +10:00
Dr. Jens Harbott 4667e6b361 Stop using openstack upper constraints
We want to be able to support older Python versions than those that are
possible with the current OpenStack constraints. Run unit tests without
constraints instead.

Change-Id: I4237a56ff9733b5e3c9521193cc5e9758c28095e
2022-06-10 13:33:29 +02:00
HeroicHitesh 687a3127d3 Migrate from testr to stestr
* Replace .testr.conf by .stestr.conf for migration and update
  .gitignore, test-requirements.txt and lower-constraints.txt
  files accordingly
* Use py3 as the default runtime and 3.18.0 as the minversion
  for tox
* Add group_regex to run all tests

Signed-off-by: HeroicHitesh <email.hiteshkumar@gmail.com>
Change-Id: I704356082e2c25d21aff3c5433efa077259b0b1d
2021-06-30 18:58:35 +05:30
zhangboye d7621706fe Remove install unnecessary package
The docs  requirements migrated to doc/requirements.txt
we need not install things from requirements.txt.

Change-Id: Iddb46304a6c7611f3e8e993561537fe2e5d02bcf
2020-09-23 14:17:08 +08:00
Daniel Bengtsson b4e520d125 Update the tox minversion parameter.
Update the minversion parameter to use the python -m pip to install
python packages:

https://tox.readthedocs.io/en/latest/changelog.html#id185

It's recommend to use this.

Change-Id: Ifd00b2f45533402d0f69ea53b6b3a7d5922691a7
2020-08-04 12:37:05 +02:00
Andreas Jaeger e7885499dd Cleanup py27 and docs support
This repo is now testing only with Python 3, so let's make
a few cleanups:
- Remove python 2.7 stanza from setup.py
- Remove obsolete sections from setup.cfg
- Update classifiers
- Update requirements, no need for python_version anymore
- Cleanup doc/source/conf.py to remove now obsolete content,
  use sphinxcontrib.apidoc to build module index again (this
  seems to have been broken when switching to sphinx-build).
- Remove install_command from tox.ini, the default is fine
- Use TOX_CONSTRAINTS_FILE instead of obsolete UPPER_CONSTRAINTS_FILE.
- Update tox.ini for python3 only support.

Change-Id: Id8738ecfb0f578d2a7953c63ffe10779f835bcaf
2020-06-05 12:04:30 +02:00
Monty Taylor 3477dfad00 Drop support for python2
python2 is EOL. OpenStack and Zuul both are python3 only.

Drop the Python-2 CentOS 7 build host testing, including the tripleo
centos-7 train job.

Drop the old openstack-python-jobs template and move to specific
version templates for what we support.

Drop upper-constraints from the py35 environment, as master of
requirements is now using python 3.6-only libraries.

Drop the Xenial tests for similar reasons (rather than making some
sort of forced install); we are keeping Python 3.5 support just for
Zuul compatability at this point so a tox run is enough.

Depends-On: https://review.opendev.org/729787
Depends-On: https://review.opendev.org/729508

Change-Id: I6d90754cb4b7217b10ae777d414c03466b751761
2020-05-29 09:47:18 +10:00
Ian Wienand de57271ed2 Prepare to drop Python 2 support
Pin func test requirements to stable/train; the last stable release
with Python 2 support.  Switch to the python-jobs-no-constraints
template to avoid using master constraints file.

Only build focal on bionic, other debootstraps are too old.

Remove pip-and-virtualenv testing as we are moving to plain images.

The tripleo-buildimage-jobs are unstable; see linked bug.

Add a note that this is the last Python 2 release.

Change-Id: Ibde7a564dd41cc2d6e80e2dffe5a95a57bbf8ada
2020-05-29 09:44:54 +10:00
Ian Wienand 28ebd24844 Uncap hacking
This causes problems for other projects incorporating dib; we don't
have a specific need for a cap.

Fix a few issues, mostly spacing or regex matches.  No functional
changes.

W503 and W504 relate to leaving artithmetic operators at the start or
end of lines, and are mutually exclusive and, due to "ignore"
overriding the defaults both get enabled.  It seems everyone gets this
wrong (https://gitlab.com/pycqa/flake8/issues/466).  Don't take a
position on this and ignore both.

Use double # around comments including YAML snippets using "# type: "
which now gets detected as PEP484/mypy type hints.

Change-Id: I8b7ce6dee02dcce31c82427a2441c931d136ef57
2020-02-24 10:34:46 +11:00
Andreas Jaeger f0dd0bd63a Only install doc requirements if needed
We only need doc requirements for the doc jobs, so only install them
there.

Also, update docs job to not publish dirtrees.

Change-Id: I928f4e24970a680bf340ce15a08a4e72432fb0e4
2019-09-13 10:09:59 +02:00
jacky06 9e3400fa9c Move doc related modules to doc/requirements.txt
Change-Id: I5106dc365856932f00a2fbdf4e7a4d58e48cca8a
2019-09-09 14:13:36 +08:00
Sorin Sbarnea b9e322f2d3 Fix broken requirements url
Fixes tox jobs which were broken since opendev migration.

Change-Id: Iebf6c9cb956e7ab37c590e456f59f460839b4e8c
2019-04-30 09:31:44 +01:00
caoyuan 0329a6de5e Replace git.openstack.org URLs with opendev.org URLs
Change-Id: Iac5a9da62db84365a769ea07146281866215a9c5
2019-04-29 20:15:25 +10:00
Natal Ngétal 07657a73e9 [Configuration] Add missing py37 and corrected default envlist.
Change order of default tox envlist in order of likeliness to fail on a
new change and add py37 and py36.

Change-Id: I9d21561991c1d956a62de72c906cf91305ce5328
2019-01-09 11:41:22 +01:00
Ian Wienand c48a40d479 Move several packages to bindep.txt
Move several parts of the "install_test_deps.sh" script into the more
standard bindep.txt.  This list is intentionally restricted as a first
step.

Developer documentation is updated to use bindep and clarified
slightly.

Change-Id: I7520902dc324d920a0c7c44a2d35fe49f9b05614
2018-10-29 09:52:13 +11:00
Nguyen Hung Phuong 7834e2ea6b fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I64d454d251712281911a1aa6e4ecd6139ad1bd8b
2018-07-03 10:08:53 +07:00
Doug Hellmann 28287942bb add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: I911c66b2e9971a3e134c482a4b4ffa529d358a76
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-04-06 01:23:10 -04:00
Andreas Jaeger 3c8e424150 Avoid tox_install.sh for constraints support
We do not need tox_install.sh, pip can handle constraints itself
and install the project correctly. Thus update tox.ini and remove
the now obsolete tools/tox_install.sh file.

This follows https://review.openstack.org/#/c/508061 to remove
tools/tox_install.sh.

Change-Id: Ieec1e7c7a4ae9a89fb413e4aa5288396690ed538
2017-12-02 17:07:13 +00:00
Tony Breeds ab1ed1e59a Add Constraints support
Adding constraints support to libraries is slightly more complex than
services as the libraries themselves are listed in upper-constraints.txt
which leads to errors that you can't install a specific version and a
constrained version.

This change adds constraints support by also adding a helper script to
edit the constraints to remove diskimage-builder.

Change-Id: I795800589ea1491efd0e489a38b60e6adf33e7f3
2017-09-22 14:54:42 +10:00
Ian Wienand d5c3863b87 Add env var to dump config graph
Make this a bit easier during debugging.  Add env var and some
developer instructions.

Change-Id: I34978ddb47d6642dfa22cae0f4c0543c0ba5475f
2017-06-08 05:04:58 +00:00
Ian Wienand 7101c52620 Add missing test requirements, fixup pylint env
Add some missing test requirements.  I noticed these because pylint
was unhappy about the imports if you look closely.

Also, pylint shouldn't be in deps as it comes from the parent's
"test-requirements.txt" install.  We don't need the VIRTUAL_ENV
setting either.

Change-Id: Ie082a058a9d3d51164448410a00d0719b0b37c4a
2017-06-05 12:22:52 +10:00
Andreas Florath f314df12c3 Refactor: use lazy logging
As described in pep282 [1], the variable part of a log message
should be passed in via parameter.  In this case the parameters
are evaluated only when they need to be.

This patch fixes (unifies) this for DIB.

A check using pylint was added that this kind of passing parameters to
the logging subsystem is enforced in future.  As a blueprint a similar
(stripped-down) approach from cinder [2] was used.

[1] https://www.python.org/dev/peps/pep-0282/
[2] https://github.com/openstack/cinder/blob/master/tox.ini

Change-Id: I2d7bcc863e4e9583d82d204438b3c781ac99824e
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-05-30 14:39:58 +10:00
Ian Wienand 543dc1baa6 Add pylint with indent check
This is an initial creation of pylint with a basic indent checker.
Small issues corrected.  Job added to gate with
Ib554a284e92583cc1d6a5c2219b3922852ca4c73

Change-Id: I7e24d8348db3aef79e1395d12692199a1f80161a
Co-Authored-By: Andreas Florath <andreas@florath.net>
2017-05-29 16:12:35 +10:00
Andreas Florath fdd2c4b236 Semi-automatic doc generation of element dependency
Currently there is no description of dependencies in the generated
documentation of the elements: therefore a user of an element does not
know which other elements are automatically included and e.g. which
configuration options are available. In addition there are some
copy&pastes of parts of the README.rst scattered thought different
Ubuntu and Debian specific elements.

This patch adds a semi-automatic generation of dependency information
of all elements.  Nevertheless these are not automatically included.
The author of the element's README.rst can decide if and where the
dependency information should appear and can use the descriptor

.. element_deps::

for this.

This patch adds the dependency information for some Debian and
Ubuntu patches - and creates the base for later removing the
duplicated parts.

A call is added to element_dependencies._find_all_elements() to
populate reverse dependencies for Element objects.

(This is a reworking of I31d2b6050b6c46fefe37378698e9a330025db430 for
the feature/v2 branch)

Change-Id: Iebb83916fed71565071246baa550849eef40560b
2017-02-09 09:50:30 +11:00
Ian Wienand 3f8800832a Release 1.27.0
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJYk8+9AAoJEBty/58O8cX8LdIH+wU/VrEVs0XYohiL6DUgabzs
 112U3UUihH5xMc/ca9Tarx+XwEvfMZkwYN2Qr0JoRJjmSt2AL6AezUhGSV+98vaY
 iQEccaFDFYlyDHm4V2r7N1xwS0B3mx87FPqVQQSUKlc3JsQxCy4o9RtD9aM8Gvqy
 +gAxMxL3p3O131K0Rvb0U5lC1FLgft9SuljCV8i5nU4/HdoryD6hedz2/ss8a9KG
 KKEdBKvPBKn73+nb8peQD/VXpej9C31r87q5VEjUsZkJ7gduY/qYLlGGgoBQqAXN
 WQ/ef1RkQKW5ba2jsjnk7fdOrA0+wYENxorR2WecuZbe2ieXw6fP3lYiD6VeWsM=
 =IUuh
 -----END PGP SIGNATURE-----

Merge tag '1.27.0' into merge-branch

Release 1.27.0

Change-Id: I9f6948636cae6d375d1d8315976504021f5a3bbb
2017-02-03 11:49:45 +11:00
Jenkins 03d1af7380 Merge "Don't run unit tests from run_functests.sh" into feature/v2 2017-02-02 22:51:04 +00:00
Ian Wienand adf39c52cf Release 1.21.1
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJYW2GoAAoJEBty/58O8cX8uSAH/15dJsglP6Zie7jSSJcR6k+e
 PJembHn9qrqrCjmJ5EwakojySaaLhwEJKvlP54OU9v7pmUXL9gJtK2OzW54LQ41g
 xBHIu0Pg4z7juyHm9+1P2Sr7Mzs1pVSEbsIYpDYUU19eghI1EAeIj3I1woKgajN7
 JlI61j3r67G6EAVtPOnmD1jvXS8CrtjiJ9wtWTH20pWfmksovg/GuXUCZrLkAAhO
 NcK35CdMii1Hkr7XOH424La/Ar+3qfUX18ZkbJY6yHzkq/ityTzzKOFjAaDl2Jg9
 WNc+SLCVYpPhPwgt7miTywamUNj3ZviA5/Hd8fuLXmtHSLQ23WOtBiaQMLtwXHs=
 =8dIl
 -----END PGP SIGNATURE-----

Merge tag '1.26.1' into merge-branch

Release 1.21.1

Change-Id: Ib9eb3dd1d384fc5b6a9846608216e056c57a173a
2017-02-02 20:36:23 +11:00
Ian Wienand f1af76f228 Don't run unit tests from run_functests.sh
Change ec7f56c1b2 add added unit tests
in diskimage_builder/tests/functional (this is probably misnamed).

These are found and run by testr just as part of the normal "setup.py
test" run.  Don't run them as part of the functional tests (this
breaks "-h"/"-l" because it installs a virtualenv and runs tests, and
also is incorrect in the gate where it's creating a nested virtualenv
underneath the testing virtualenv).

Change-Id: I9908e080042d3026a198ba89eb653c6eff376d22
2017-02-02 15:20:24 +11:00
Ian Wienand bfca36c772 Release 1.25.2
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJYV1yqAAoJEBty/58O8cX8hLwIAKP66w6MdPN8PDgUOteui/Sx
 N0UFKJ9yR4GQOAP0NffPLjch5/g0iJLs3eFKOhtGC1LjbDjpVgjX8vW18ib8wBZK
 GemOZPF3uxg8FROrZF1vpoDy/cHgL1YV10hCnwdjN/r9rb8zOuSabqjW+Dennj2n
 fZ0SJfa8Owfudn3YxGuOymVb/wMtEloDmVGBEI1Y+h7osELCCDi3OXmwsA8qMsdl
 cTwbeugBs4PlOVbZUK/JKGuwIHKgPnDYzYu5KpXw77/MdjGT0fo5Tlq5AOBDI2sC
 9JOFEBDli4Ro05VwvI58ADMpvvOax+9EvOhLbB1dRPdZl21Iyb6gOdy2PUbFO0c=
 =aKxq
 -----END PGP SIGNATURE-----

Merge tag '1.25.2' into merge-branch

Release 1.25.2

Change-Id: I698bcf2e82117bd81649cd065a7af5cac85990c7
2017-02-02 11:20:00 +11:00
Andreas Florath ec7f56c1b2 Refactor: block-device handling (partitioning)
During the creation of a disk image (e.g. for a VM), there is the need
to create, setup, configure and afterwards detach some kind of storage
where the newly installed OS can be copied to or directly installed
in.

This patch implements partitioning handling.

Change-Id: I0ca6a4ae3a2684d473b44e5f332ee4225ee30f8c
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-01-24 19:59:10 +00:00
Ian Wienand baf3ced0c3 Fix coverage report
There's a few things going on here

Firstly, we need to install coverage tool in test-requirements

Secondly, .testr.conf has to use PYTHON because the coverage report
works by resetting PYTHON='coverage run ...'

Thirdly, because we call ourselves diskimage-builder but the python
module is diskimage_builder that seems to confuse things.  We need to
use "setup.py test" (note, that is different to "setup.py testr"!) to
use the PBR testr wrapper.  That exposes a --coverage-package-name
argument that calls the coverage tool with the right argument.

With this I got a coverage report for our unit tests

Change-Id: I9012e18eb7d01bee035140e70afa76c47c27eb01
2017-01-18 16:14:01 +11:00
xhzhf f6ecfb3b24 H803 hacking have been deprecated
H803 hacking have been removed.
https://github.com/openstack-dev/hacking/blob/master/setup.cfg
Closes-Bug: #1650741

Change-Id: Ic88b431d9260c6e31717b001ff657d10d76b9213
2016-12-21 22:19:59 +08:00
Gregory Haynes 54dd54d81d Switch py34 tests to py35
Py35 is now commonly available on things like -infra test nodes, lets
use that instead.

Change-Id: I554ec84d709a9937df5ec3c2fa7eab8f158cf64e
2016-12-19 23:22:24 +00:00
Gregory Haynes 5da7574aee Add get_elements; preserve backwards compat for expand_dependencies
expand_dependencies() was a public interface so we should try and
preserve backwards compat. However, since the interface is really
broken, add a new exported function "get_elements" that instack can
switch to.  This returns the canonical list of elements without
duplicates, and gives the path to each element too.

This highlighted that the unit tests were really a bit wrong.  They're
testing inner functions when we have an "API" in the get_elements()
function.  Convert all unit-tests to use this function instead.  Since
this is a library call, convert the sys.exit() calls to raised
exceptions.

Refactor the variable output into a separate function so we can do a
sanity check on it.

The added flake8 ignores are for the "over-indented for ... indent"
which happens a lot with these new longer lines.  Most other projects
ignore them.

This is an alternative proposal to
I15609389c18adf3017220fc94552514d195b323a

Change-Id: If97bcd21e45de1b5ed91023fdc441a4617051a6b
2016-12-20 08:18:59 +11:00
gengchc2 b2c3a3afdd Delete deprecated Hacking in tox.ini
Some hacking have been removed,so we can delete them.
More details:
    https://github.com/openstack-dev/hacking/blob/master/setup.cfg

Change-Id: I2ba6b00aa2a968ffb53d1061f895a593ea57a063
2016-12-04 13:04:50 +08:00
Ian Wienand 7d5afecfd9 Merge remote-tracking branch 'origin/master' into merge-branch
Change-Id: Ibab1bb95521292ae818bd91f7073c3749a2cc0cb
2016-11-18 13:53:56 +11:00
nizam 3b855faf5f Don't include openstack/common in flake8 exclude list
The directory openstack/common was used to keep codes from
oslo-incubator, we have retired oslo-incubator, so don't use
this directory any more

Change-Id: Icd13d32674b117626fbecdfdec2881463a87ad5d
2016-10-28 19:52:25 +05:30
Andreas Florath 3d48a528c1 Refactor: block-device handling (local loop)
Block device handling can be somewhat complex - especially
when taking things like md, lvm or encryption into account.

This patch factors out the creation and deletion of the local
loop image device handling into a python library.

The main propose of this patch is to implement the needed
infrastructure.  Based on this, more advanced functions can be added.
Example: (advanced) partitioning, LVM, handling different boot
scenarios (BIOS, UEFI, ...), possibility of handling multiple images
(local loop image, iSCSI, physical hard disk, ...), handling of
different filesystems for different partitions / LVs.

Change-Id: Ib626b36a00f8a5dc3dbde8df3e2619a2438eaaf1
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-09-08 04:31:01 +00:00
Ian Wienand 634391185c Add releasenotes
Use reno to start at keeping release notes.  Add an initial log.

Change-Id: Iba3ebd3b01c15030ac2585dda82e43657e511310
2016-04-21 13:19:53 +10:00
janonymous d2bc111d6a py26 is no longer supported by Infra's CI
Python 2.6 support would be dropped by
Infra team from mitaka,CI would no longer be testing it,
so projects should drop it too.

Change-Id: I916150ec5508a095fd850c9cb85dbeb464cdb826
2015-12-26 13:44:33 +05:30
Ondřej Nový b35f3a6c4b Deprecated tox -downloadcache option removed
Caching is enabled by default from pip version 6.0

More info:
https://testrun.org/tox/latest/config.html#confval-downloadcache=path
https://pip.pypa.io/en/stable/reference/pip_install/#caching

Change-Id: If624cea7246e0a16355923af122b2f8586718d5e
2015-12-11 22:58:50 +01:00
Dmitry Tantsur 0e122e8e35 Add a tox target to run functional tests locally
Now 'tox -efunc' can be invoked to run all functional tests in
the 'venv' tox environment. Also `tox -efunc element-name` can be
used to run function tests for one element (e.g. ironic-agent).

Change-Id: Ia685d1b2a7deef2f8b98876ac09792134dd30f2f
2015-10-23 12:05:23 +02:00
THOMAS J. COCOZZELLO d8a8f9b732 Activate pep8 check that _ is imported
Remove the specification in tox.ini that _ is a builtin so that
it will no longer assume that _ does not need to be imported.
This helps ensure that the _ from i18n is used.

Activating this check did not flag any violations.

Change-Id: Iac73937b60a6f8e7520123fbf59e748d296d4c7d
2015-09-03 15:41:31 -05:00
Ben Nemec 7f974d4b85 Reorder tox environments
If py34 is run after py2x, it results in a "db type could not be
determined" error.  The only current way to avoid that is to run
py34 first.

Change-Id: Id81e127e71ecd04a2ed16ab899d6fbf0d15bfee3
2015-03-30 20:57:34 +00:00
Gregory Haynes 4fbad49972 Add py34 to tox
We gate on py34, so we shoudl add it to our tox config.

Change-Id: I0d21cdd47841af8d05df7b36b2302f13bda3367b
2015-03-29 04:28:11 +00:00
James Polley 20b7c0b7b3 Add `tox -edocs` capability
This is much easier to remember than the full commandline - and keeps
this project in line with others

Change-Id: I2f631c25d8839272941b7c8d4bb571a853422083
2015-02-10 14:34:46 -08:00
Gregory Haynes c4bbb6f3bc Create docs site containing element READMEs
We currently do not have the ability to create a docs site which
outlines all the elements.

Change-Id: I77ccf61e0c4b1509b3e7ce9b8f15ea5ccfd50d9b
2015-02-10 11:45:35 -08:00
Gregory Haynes e5b8656141 Add new package-installs system
We currently support package-installs definitions which has some
limitations and oddities. This new format requires only one definition
which does not reside in our run-parts directories and follows a
consistent naming scheme (package-installs.yaml).

Change-Id: Ie51a7c4fdc15634ae8e069728e5e07cc1dc36095
2014-12-01 21:29:47 -08:00