Commit Graph

52 Commits

Author SHA1 Message Date
Zuul bed20ed9d3 Merge "pre-commit: Integrate bandit" 2024-03-29 02:56:38 +00:00
Takashi Kajinami 8c89839e37 Use upper-constraint in doc generation
Currently doc jobs are broken because the latest sphinx 7.2.0 is not
compatible with the latest release of openstackdocstheme.

This updates the docs/releasenotes target to use the global upper
constraints to avoid that issue.

Change-Id: I7eef81e2cfc2156dd8aa076e09de4904dda6eb9a
2024-03-14 00:06:39 +00:00
Takashi Kajinami 3d23bfdc81 tox: Drop envdir
tox now always recreates an env although the env is shared using envdir
options.
~~~
$ tox -e genpolicy
genpolicy: recreate env because env type changed from
{'name': 'genconfig', 'type': 'VirtualEnvRunner'} to
{'name': 'genpolicy', 'type': 'VirtualEnvRunner'}
~~~

According to the maintainer of tox, this functionality is not intended
to be supported.
https://github.com/tox-dev/tox/issues/425#issuecomment-1011944293

Change-Id: I5db68dc1ca25626d7f50c2fa9319896721c08d3d
2024-02-02 23:43:25 +09:00
Takashi Kajinami 1a7af1289b pre-commit: Integrate bandit
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
Change-Id: I8ce1724bb1f7eab88204d350616bf59d910ea57d
2024-02-02 10:19:06 +09:00
Grzegorz Grasza fb6b880eaf Fix tox4 error
tox.ini started failing with Tox4 which had some
incompatible changes. One of them is due to the
skipsdist = True.

This also adds setup-vault-env.sh to allowlist_externals, since it
fails otherwise.

Change-Id: If0ace2c2ef0915d9a4ffcba99d664ffe97621b12
2023-01-12 14:01:22 +01:00
Ade Lee 316db6cb53 Make tests more consistent
Tests periodically fail right now because the cleanup which is
running in a different thread sometimes removes the managed objects
before we have completed testing with them.  The change to use
concurrency=1 will slow down the tests, but also make them more
consistent.

Also, when things are not cleaned up, you can get false positive
results if multiple objects contain the same content.  This will
fix this problem by making the contents unique.

Change-Id: Ic2b9e6afe9371dbe135e90fa6df36a8e91921556
2022-09-21 16:27:54 +02:00
wu.shiming 2073f450be Changed minversion in tox to 3.18.0
The patch bumps min version of tox to 3.18.0 in order to
replace tox's whitelist_externals by allowlist_externals option:
https://github.com/tox-dev/tox/blob/master/docs/changelog.rst#v3180-2020-07-23

Change-Id: Id9b9f5c3a99f6b3717a69a65e1aadc25aa4d48b4
2021-06-10 09:30:57 +08:00
wu.shiming e7b8bf91ec Remove lower-constraints remnants
These were missed in change: I1d4de15d017a306e64df8447cf1bd64c70cf361a

Change-Id: I906c1e5f24aa93d7cc65f149b9164f85a7c57e7c
2021-03-11 12:59:24 +00:00
Hervé Beraud 47b1733198 Use py3 as the default runtime for tox
Moving on py3 as the default runtime for tox to avoid to update this at
each new cycle.

Wallaby support officially the following runtimes [1]:
- Python 3.6
- Python 3.8

During Victoria Python 3.7 was used as the default runtime [2] however this
version isn't longer officially supported.

[1] https://governance.openstack.org/tc/reference/runtimes/wallaby.html#python-runtimes-for-wallaby
[2] https://governance.openstack.org/tc/reference/runtimes/victoria.html#python-runtimes-for-victoria

Change-Id: I4f955502694a26db13a36ff4b1fea1c0a3b3b03e
2020-11-04 10:00:36 +01:00
Hervé Beraud b719a5b56d Adding pre-commit
Introduced changes:
- pre-commit config and rules
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.

Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.

pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker (check-byte-order-marker);
- Checks that non-binary executables have a proper
  shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings (check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
  calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)

For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks

Change-Id: I35e092c472e5d564ebc9bb6c2a4f6d40b54ff120
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
2020-09-29 13:24:21 +00:00
Andreas Jaeger a39ed69ea0 Some Cleanups
This repo is now Python 3, update:
- Remove python 2.7 stanza from setup.py
- Remove install_command from tox.ini, the default is fine
- Remove Babel from requirements, it's not needed for running.
- Use TOX_CONSTRAINTS_FILE instead of obsolete UPPER_CONSTRAINTS_FILE.
- Use py36 for fault test

Change-Id: Ia23841a4899361178994e349e1323de857b4ee23
2020-06-11 12:26:35 +02:00
Sean McGinnis e9dc8e51eb
Bump default tox env from py37 to py38
Python 3.8 is now our highest level supported python runtime.
This updates the default tox target environments to swap out
py37 for py38 to make sure local development testing is
covering this version.

This does not impact zuul jobs in any way, nor prevent local
tests against py37. It just changes the default if none is
explicitly provided.

Change-Id: I0bd1180719e4e093a24bc53424052354ad1cd900
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-24 10:25:34 -05:00
Moisés Guimarães de Medeiros 1d06f31f97 Fix coverage tests
Coverage tests seems to be broken on a fresh clone of castellan.
Copying the coverage config from Oslo.Config repo.

Change-Id: I9a01313d3d1d95726372d5d5f2ac8ab5c3cd13dc
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
2020-02-07 15:58:12 +01:00
Andreas Jaeger 141e7e4209 [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in ussuri cycle.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Cleanup setup.cfg as well.

Increase requests version in lower-constraints:
requests>=2.18.0 is needed by oslo.config, which leads to some other
updates...

Change-Id: I09f0da9a59952739dd1575200aa47cc12026264f
2020-02-03 19:51:46 +01:00
Alexandra Settle 6a90cdc520 PDF Documentation Build tox target
This patch adds a `pdf-docs` tox target that will build
PDF versions of our docs. As per the Train community goal:

  https://governance.openstack.org/tc/goals/selected/train/pdf-doc-generation.html

Add sphinxcontrib-svg2pdfconverter to doc/requirements.txt
to convert our SVGs.

This PR also introduces the docs testenv and a doc
requirements.txt file.

Change-Id: If782737efe7114422be9cc3fc586500a2065f07e
Story: 2006072
2019-10-01 13:53:06 +00:00
Zuul 54af64d0d3 Merge "Add Python 3 Train unit tests" 2019-08-14 15:26:11 +00:00
Ben Nemec 2547c8c9f8 Add bindep.txt
The Vault functional tests are currently failing in the gate, and it
appears to be because the unzip binary is not present. We need to
add it to bindep so it will be installed before running tests.

Change-Id: I4b8075429218a494f75f5e3611f48927d18f1700
Closes-Bug: 1840061
2019-08-13 16:57:44 +00:00
Corey Bryant 6bbbfa2cc4 Add Python 3 Train unit tests
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.

See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html

Change-Id: I613d3cb6dbe74ebf173a72c7f5a650c8e4494efc
Story: #2005924
Task: #34234
2019-07-05 14:38:32 -04:00
Le Hou c9ce2c6cc1 Use opendev repository
Change-Id: I6e19a18dac9d1f9393f202b491e890f19a02395c
2019-04-23 16:23:55 +08:00
Moisés Guimarães de Medeiros 9a34dc9107 Set py3 tests according to Stein runtimes.
According to answers on the mailing list, python 3.6 is a must while
python 3.5 and 3.7 are not. So this patch drops 3.5 and sets 3.6 on
tox.ini and zuul jobs.

https://governance.openstack.org/tc/reference/runtimes/stein.html#python-runtime-for-stein
http://lists.openstack.org/pipermail/openstack-discuss/2019-February/003205.html

Change-Id: If708bbec3a4a59261b2ce2f94580d3cdcae55c3a
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
2019-02-27 13:44:46 +01:00
zhangzs 0450c73819 Update min tox version to 2.0
Change-Id: I06aabf17c0783d354d471a33e7c79147d450fe64
2018-11-09 15:59:22 +08:00
Nguyen Van Trung 27c6f63bcd Don't quote {posargs} in tox.ini
Quotes around {posargs} cause the entire string to be combined into one
arg that gets passed to stestr. This prevents passing multiple args
(e.g. '--concurrency=16 some-regex')

Change-Id: I7e3f0bf2f5999cc0f4b54f68da33debccf077f43
2018-10-10 09:24:34 +07:00
Vu Cong Tuan 22a5b8b1ec Switch to stestr
According to Openstack summit session [1],
stestr is maintained project to which all Openstack projects should migrate.
Let's switch to stestr as other projects have already moved to it.

[1] https://etherpad.openstack.org/p/YVR-python-pti

Change-Id: I644c28acb04451113849691cd64c209258cef7d7
2018-07-11 08:56:06 +07:00
Doug Hellmann b9f6436776 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I0980a08f35965292a705ad7299cfdc9a433dfc87
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-06 14:53:48 -04:00
Doug Hellmann e9ea252bbc fix list of default virtualenvs
Update the list of default virtualenvs that are run when a developer
just runs "tox" without specifying an environment.

Change-Id: I38caffbc10c2c8c559212e371baee9d3272f8864
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-04-13 15:51:05 -04:00
Doug Hellmann 837fc4c5d8 set default python to python3
Set the default python to python3 except for the py27 environment. We
have to set that explicitly to override the new default.

Change-Id: I64799bb18c6c5848d23378e163481ad0fe1eea2b
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-04-13 15:50:21 -04:00
Doug Hellmann 18068e1779 add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: Iee4a31b43f1f60410892e25d4e33ce63a620a95b
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-04-09 11:00:20 -04:00
Andreas Jaeger c67357173e Avoid tox_install.sh for constraints support
We do not need tox_install.sh, pip can handle constraints itself
and install the project correctly. Thus update tox.ini and remove
the now obsolete tools/tox_install.sh file.

This follows https://review.openstack.org/#/c/508061 to remove
tools/tox_install.sh.

Change-Id: I11127d4cc7263ced57309952b71df1c7813f57d6
2017-12-02 17:04:19 +00:00
Davanum Srinivas a972da32a9 Vault based key manager
* Uses https://www.vaultproject.io/ to store/fetch secrets
* All we need is the URL and a Token to talk to the vault server
* tox target "functional-vault" sets up a server in development mode
  and runs functional tests
* Supports both http:// and https:// url(s)
* the https support was tested by setting up a vault server by hand
  (https://gist.github.com/dims/47674cf2c3b0a953df69246c2ea1ff78)
* create_key_pair is the only API that is not implemented

Change-Id: I6436e5841c8e77a7262b4d5aa39201b40a985255
2017-11-13 20:56:34 -05:00
Jeremy Liu 5d946f0855 Add releasenotes for castellan
Change-Id: I668815f3b353e15b72b88a62ea098fef66a94ae0
2017-09-05 13:40:42 +08:00
lioplhp 3945e3e960 Enable some off-by-default checks
Some of the available checks are diskabled by default, like:
[H106] Don't put vim configuration in source files;
[H203] Use assertIs(Not)None to check for None.

Change-Id: Ibe4e8bb6373def06452ae6fc9ca16a99a8a05f29
2017-06-23 15:52:17 +08:00
Kiran_totad cad413f2eb removed the older version of python
Change-Id: I8329000b54f289166dc3c366b13230a1a5e58456
2017-03-07 10:39:40 +05:30
Jenkins c83eb195e3 Merge "Enable coverage report in console output" 2017-01-11 15:31:01 +00:00
Jeremy Liu 2ba9b07004 Enable coverage report in console output
Change-Id: Ied4cac092fba5d4b5027e72e1f18a65745480927
2016-12-30 16:39:38 +08:00
xhzhf c70a4795fc H803 hacking has been removed
H803 hacking have been removed since hacking 0.10.
https://github.com/openstack-dev/hacking/blob/master/setup.cfg.
castellan have remove ignore statement, but don't remove comment
Closes-Bug: #1651358

Change-Id: Ic5d0dbbf975a7ffd58b3fba06d6f0971711365bd
2016-12-20 00:56:59 +08:00
Steve Martinelli fccb2fef4b remove obsolete oslo incubator code
as part of the openstack wide community goals, oslo
incubator code should be removed from all projects [1]

[1] https://governance.openstack.org/goals/ocata/remove-incubated-oslo-code.html

Change-Id: Ifa3564df125ed002dc1710d7a7c0e9346c34c9f1
2016-11-01 14:16:50 +00:00
Jenkins 3612ca730f Merge "Add Python 3.5 classifier and venv for castellan" 2016-10-17 12:15:04 +00:00
Jenkins 3f0688dab4 Merge "Update flake8 ignore list" 2016-10-17 11:37:47 +00:00
Jeremy Liu c6875035ea Support upper-constraints in tox.ini
Since the castellan itself is in upper-constraints.txt now,
in CI job, we should remove it from the constraints file before applying it,
otherwise pip will fail due to castellan version conflict.

Change-Id: I5d58303b7f76e0e92083e14d3cff009c02c9fc14
Closes-bug: #1614361
2016-10-10 03:19:27 +00:00
gengchc2 298d1c9c9b Update flake8 ignore list
In hacking > 0.10.0, there is no E123 and E125, so this commit to
delete them.

Change-Id: I85f9f52b78fddb9f10586d0dc7e381bb191b84de
2016-09-28 11:31:57 +08:00
gengchc2 7ba1214f08 Add Python 3.5 classifier and venv for castellan
Now that there is a passing gate job, we can claim support for
Python 3.5 in the classifier. This patch also adds the
convenience py35 venv.

Change-Id: I667af65814350664bb5cd75f79ad3ac1b56130a1
2016-09-05 09:56:34 +08:00
Robert Clark dd6ad90e01 Add a bandit environment to tox
This change is being proposed as part of the OpenStack Security Project
working session at the Austin 2016 summit. It adds support for running
the bandit[1] security linting tool against the Castellan codebase.
This change adds a targetted environment for bandit and also adds
bandit as part of the pep8 job.

The bandit configuration has been tailored to exclude tests that are
currently producing warning against the codebase. These issues will be
followed up with bug reports and patches.

At the time of submission, Castellan passes all Bandit tests configured
in tox.

[1]: https://wiki.openstack.org/wiki/Security/Projects/Bandit

Change-Id: I19368d3440ad5dc862e7d91f7890f9b1901fced3
2016-04-28 17:06:16 -05:00
ting.wang c18e510ff0 Clean up removed hacking rule from [flake8] ignore lists
We bump hacking>=0.10.0, and hacking removed some rules,
for the full list of rules please see [1]. So don't need them any more.

[1]https://github.com/openstack-dev/hacking/blob/master/setup.cfg#L30

Change-Id: I61b57fc7c958a869bc97d1cb73d447413f60cebd
2016-01-21 10:54:24 +08:00
sonu.kumar 25902e24e2 Remove py26 support from castellan
As of mitaka, the infra team won't have the resources available to
reasonably test py26, also the oslo team is dropping py26 support
from their libraries. sine we rely on oslo for a lot of our work,
and depend on infra for our CI, we should drop py26 support too.

Change-Id: I4db3fb61038377bc96e47ac160a2da1652901a1b
Related-Bug: #1519510
2015-12-16 14:54:55 +05:30
“Fernando 2f47c5f39b Add the debug flag to tox.ini
Allows us to set a breakpoint in code, using pdb.set_trace().
Documentation on how to debug has also been added.
Change-Id: I34bde581ce331b2eb63e4dab08e9f9c597598bf1
2015-11-17 18:16:15 -06:00
Jenkins b7d77bba52 Merge "Activate pep8 check that _ is imported" 2015-08-18 03:32:43 +00:00
Michael McCune d768fbc97d refactoring castellan configuration
This change adds a module for listing configuration options and setting
their defaults. It also changes the key manager base class to
incorporate a configuration during creation. By default, the key manager
will continue to use the global CONF object from the oslo.config
package.

For the most part, this change will be backwards compatible. The one
exception is the creation of sample configuration files. Previously,
importing castellan was sufficient to add these options to the global
configuration object. Now, these options will need to be applied by
using the castellan.options.list_opts function, or adding them through
other means, to create sample configuration files. Similar applies for
setting configuration before instantiating a key manager.

changes
* adding castellan.options with list_opts and set_defaults functions
* changing KeyManager abc to include a configuration option to __init__
* changing barbican and not_implemented key managers to accept
  configuration parameters
* adding tests for set_defaults function
* fixing barbican tests to accomodate new configuration parameter
* adding documentation about configuration usage
* adding castellan configs to oslo entry point in setup.cfg
* adding a genconfig target to tox for producing a sample castellan
  configuration file
* adding the sample configuration file to the git ignore
* renaming barbican option api_version to barbican_api_version

Change-Id: I86d6d7d49a893beaae6f311060ec593e0482d889
Implements: blueprint improved-configuration-options
2015-08-12 12:03:36 -04:00
Tom Cocozzello 01df1e7a22 Activate pep8 check that _ is imported
Remove the specification in tox.ini that _ is a builtin so that
it will no longer assume that _ does not need to be imported.
This helps ensure that the _ from i18n is used.

Activating this check did not flag any violations.

Change-Id: Ibb45131025bb64feb26d3c2fd596fe2b07c0b889
2015-08-04 10:42:19 -05:00
Kaitlin Farr 573235913f Add functional tests for Barbican key manager wrapper
Creates the initial functional testing infrastructure for Castellan.
Functional tests are run using:
    tox -e functional
The Barbican wrapper tests require running local Barbican and Keystone
instances.
The functional testing config parameters can be overwritten by creating a
config file at /etc/castellan/castellan-functional.conf. A sample config
can be generated using:
    tox -e genconfig
The generated sample config can be found at etc/castellan/castellan-functional.conf

Implements: blueprint add-barbican-key-manager
Change-Id: Ideb9b1f01e51d85ff56575d8ab6ac970053a9604
2015-07-23 14:27:20 +00:00
Kaitlin Farr a6c9df4849 Move unit tests to unit test folder
Change-Id: Ife49e3b046a841a6fd8ac698b9d19c66850eedfd
2015-07-22 14:29:06 -04:00