Commit Graph

70 Commits

Author SHA1 Message Date
Dmitriy Rabotyagov e77c311442 Update tox.ini to work with 4.0
With tox release of 4.0, some parameters were deprecated and are ignored now
which causes tox failures. One of the most spread issues we have is using
`whitelist_externals` isntead of `allowlist_externals`


Change-Id: I7807b7d29f4504404253f5c42b624639c8b19c97
2022-12-27 17:53:11 +01:00
Jonathan Rosser 7b7e25b0a8 Cleanup setup.py config
Change-Id: Id743a4119b35789eb8522f9e95e7fc442a8e6011
2022-04-04 10:50:36 +01:00
Andreas Jaeger 1fa67fd90d Enable syncing of docs
The docs job is failing in https://review.opendev.org/671840 and thus
nothing is synced in from openstack-ansible-tests. The failure is due to
the removal of entries from doc/requirements.txt. Add those
to test-requirements.txt instead.

Change-Id: I21bcbde8acc8d4fd83b28026bcec33f388e69912
2020-06-03 22:04:16 +02:00
Andreas Jaeger 6e23deb6af Cleanup py27 support
This repo is now testing only with Python 3, so let's make
a few cleanups:
- Remove python 2.7 stanza from setup.py
- Remove obsolete sections from setup.cfg
- Update requirements, no need for python_version anymore
- Use newer openstackdocstheme and Sphinx versions
- Cleanup */source/conf.py to remove now obsolete content.
- Remove install_command from tox.ini, the default is fine

Change-Id: Ic96b71596d4523e55fa4b451c99a8521dd581e4d
2020-04-25 15:07:26 +02:00
Ghanshyam Mann 83ac8bfd6d [ussuri][goal] Updates for python 2.7 drop
OpenStack is dropping the py2.7 support in ussuri cycle.

openstack-ansible repos only need updates on requirements
and tox file.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: Idf700e627b5c88059762690aec6dc3e3a345a39f
2020-04-03 21:18:52 +03:00
Dmitriy Rabotyagov 9cfc60f307 PDF Documentation Build tox target
This patch adds a `pdf-docs` tox target that will build
PDF versions of our docs. As per the Train community goal:

  https://governance.openstack.org/tc/goals/selected/train/pdf-doc-generation.html

Add sphinxcontrib-svg2pdfconverter to doc/requirements.txt
to convert our SVGs.

Change-Id: I04319a1195873d63bfc45ffb0f5c7c89fb797652
Story: 2006105
2019-10-17 17:37:23 +00:00
98k 543e885ffc fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: If229d848b92ed10ea4b788598c575c5b9f693c90
Closes-Bug:  #1801657
2018-11-06 05:22:32 +00:00
Doug Hellmann 7184d5d40e fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I75820c3160ffa9cbe048650ba30aa44281a7c9a1
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-10-03 23:55:21 +00:00
Heba Naser ee9ac0881c Use tests repo for common role test requirements
Using tox for requirements management requires in-repo
requirements files for all our repositories. Rather than
do that, we make use of the tests repo to capture our
common requirements and use this to install them.

This reduces our review requirement rate and simplifies
maintenance for us for the tox config. It also makes it
usable with 'Depends-On', which is marvellous!

The tox requirements definitions for docs/releasenotes
builds are left in-place as those are standard entries
across the community. If that changes at some point, we
can re-assess those entries too.

Depends-On: https://review.openstack.org/579208
Change-Id: Ibaf4aeeb60b0ceca8352cef87bf7c38529ca0fd2
2018-07-01 15:51:35 -04:00
Jesse Pretorius 649e6ce02a Remove tests-repo-clone.sh
Now that run_tests.sh handles the tests repo clone, we can
remove the use of the older tests-repo-clone.sh script.

Change-Id: I839a959565585af033188ef13087d52dc320bc1f
2018-03-28 10:10:35 +01:00
Jesse Pretorius cf86f3e55a tox.ini: Expose USER environment variable to execution environment
In order to allow the use of the environment variable which informs
Ansible which user executed the playbook, we pass the USER env var
into the environment that tox builds.

Change-Id: Ia4e760454d216b8d2f54fc6fc11d7455f148427c
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
2018-03-15 17:55:14 +00:00
melissaml 540408f597 Follow the new PTI for document build
For compliance with the Project Testing Interface as described in:
https://governance.openstack.org/tc/reference/project-testing-interface.html

For more detials information, please refer to:
http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html

Change-Id: I56c7a5247235f550ee3ed4344f79c8fa9e86cc67
2018-03-12 12:48:34 +08:00
Major Hayden 0c0767b3f1
Queens doc updates + removal of RHEL 6 STIG
This patch begins the teardown of the RHEL 6 STIG content from the
ansible-hardening repository. It will still be maintained in
Pike and earlier branches.

This patch also updates the ansible-hardening documentation for the
Queens release and notes that Pike is the latest stable version.

Closes-Bug: 1715745
Change-Id: Iaae52c97a35d82dd807ef78a1a6593ce3aa33540
2017-09-12 08:19:54 -06:00
Jenkins 051fe3195f Merge "Ensure that role tests pin pip/setuptools/wheel" 2017-06-13 06:55:30 +00:00
Jesse Pretorius 2b14fca934 Ensure that role tests pin pip/setuptools/wheel
Currently the role tests use whatever versions of pip,
setuptools and wheel are already installed on the host.

When a version of these tools changes it often causes
problems for our testing.

This will ensure that we use a known good set of pins
which is maintained in the general SHA bumping process.

Change-Id: Ib19dcb75e8f924ae294299242a7f51efe6df6126
2017-06-13 06:35:00 +00:00
Major Hayden 875f635ab4 [Docs] Overhaul STIG by tag docs
This patch gets rid of the old "special notes" section that was a
dead-end in the documentation and replaces it with a brief header
followed by a dynamically-generated list of tag-specific
documentation. All of this sits underneath the "Hardening Domains"
section.

It also splits the "Deviations" documentation into its own section
because it's quite important for a deployer to review.

The patch also includes a link to video/slides from the Boston
Summit, which provided the latest updates for the project and some
background on how everything fits together.

Change-Id: I1a5e78733c301335fe1bcfcee36cc146d690b841
2017-06-13 06:33:16 +00:00
Major Hayden 68ecd213b8 Fix ansible-hardening references in tox/playbook
This patch fixes the role name for ansible-hardening in tox.ini as
well as the test playbook.

Change-Id: Id26a17c484da51b67f2aa7921bb92d752d67a024
2017-06-12 18:25:40 +00:00
Jesse Pretorius 38255a83c2 Use zuul-cloner for tests repo in OpenStack-CI
When executing the tests repo clone in OpenStack-CI,
use zuul-cloner instead of git to enable cross-repo
testing. This ensures that if a dependent patch from
the tests repo is noted using 'Depends-On: <change-id>'
in the commit message, that patch will be included.

Depends-On: Idce7abebf32f24c356a27e099fbca954d917402b
Depends-On: I5da7802d61d2ab6b03908138e3a3ed2db22e3d29
Change-Id: I4da173e3c41e70ff48b3c88c430a6a65eded295a
2017-05-16 15:37:26 +00:00
Major Hayden 5ef94bf0ca
Fix security role gate
This commit removes the verbose options from the gate job and disables
clamav installation in the CI jobs. The clamav package is only available
in the EPEL repository, but the EPEL repo has been removed from
the CentOS images in the OpenStack gate. This will need to be handled
carefully in a later patch.

It also removes an apostrophe from `tasks/main.yml` that breaks syntax
highlighting in vim.

Change-Id: Ifbfc56ed5fe92887cf5beb6b2703fdc3e1c8bb05
2017-05-16 10:24:25 -05:00
Major Hayden a54773938a
Cleanup tox.ini
This patch removes the `func_rhel7` environment and brings over the
verbose options from the tests role.

Change-Id: I44c2e089ff6175b3004ef7f6713622ac615bf6db
2017-04-28 13:30:40 -05:00
Major Hayden 5a4efe7cde Maintain default ansible parameters
This patch ensures that the default ansible parameters are still used.

Change-Id: I6bef1045d2e150508d86dbc21887c87af3179c61
2017-04-21 14:28:56 -05:00
ji-xuepeng 215fb08eb5 Use https instead of http for git.openstack.org
Trivialfix

Change-Id: I5b595713eeed1ea59756485866acb43d94aa7c19
2017-02-06 21:27:54 +08:00
Major Hayden 1025238b01 Restore RHEL 6 STIG content gating
When the role was switched to use RHEL 7 STIG content as the default,
the RHEL 6 STIG content was no longer being gated. This patch adds a
new test environment, `func_rhel6` and replaces the `func_rhel7`
environment with testing of the RHEL 6 STIG content. This is only
temporary and the `func_rhel7` environment should be removed as soon
as upstream gating is updated to use the new `func_rhel6` environment.

Change-Id: I9f91dac8ecadd3b791954f18e0607403ff147876
2017-01-30 20:37:23 +00:00
Major Hayden 6f6c08f4c3 Enable RHEL 7 STIG tasks as default [+Docs]
This patch enables the RHEL 7 STIG content tasks as the default.
Documentation has also been updated to reflect the change and provide
more concise information about what is available with each release.

The OpenStack-Ansible repo is still set to use the RHEL 6 STIG until
some issues with individual roles are resolved.

Implements: blueprint security-rhel7-stig
Change-Id: Ic72d97b87c0fb16646e5a31030404e1a9ad6a469
2017-01-13 19:06:07 +00:00
Major Hayden 14fa6e5060 Enable chrony [+Docs]
This patch enables chrony and performs basic configuration to meet the
STIG requirements.

These tasks can't be enabled in OpenStack CI due to conflicts with existing
NTP daemons in the CI image.

Documentation is included.

Implements: blueprint security-rhel7-stig
Change-Id: If6736c0f4a16de1ba41a4cfa00f5f72f8baf0054
2016-12-09 10:32:24 +00:00
gecong1973 113947b265 Delete deprecated Hacking in tox.ini
Some hacking have been removed,so we can delete them.
More details:
     https://github.com/openstack-dev/hacking/blob/master/setup.cfg

Change-Id: I32ec78a9d08cef076856b5841f8aaa9a2d6de8a8
2016-12-04 10:09:39 +08:00
Major Hayden 784a38ec4c Speed up package install/removal
Instead of breaking up package installations and removals into separate
tasks, this patch moves them all under one task that does two execution
steps.

In addition, the security_enable_chrony variable was added to control the
installation and configuration of chrony. The tox tests for the role were
configured to skip chrony in the gate using a skipped tag, but this caused
the package install/removal task to get skipped.

Docs/release notes are included for the chrony change.

Change-Id: I1def033953b50be3911cd932fd17b10dd2c658b7
2016-11-03 13:30:56 -05:00
Jesse Pretorius eed96b4bf0 Use upper constraints for all tox targets
With the implementation of https://review.openstack.org/388087 all
tox targets may now use upper constraints.

Change-Id: I08cb68bf8eb8ec734894dc85314ed5fbda6a78e3
2016-10-19 07:46:37 +01:00
Major Hayden 13e3fd4208 Security: Remove quotes from extra vars
The func_rhel7 gate is not running the RHEL7 STIG tests due
to an issue in the tests repository (see Depends-On below) and due
to some errant quotes in the tox.ini.

Depends-On: I0ec6eb0692e67ebdfdf81b3cbfa89e8c51d88d10
Change-Id: Ife5591ea98fae9e502a9f230ab13a73c11fe78a3
2016-10-12 10:03:59 -05:00
Major Hayden 90c363031e Use centralised Ansible test scripts
This patch consumes the centralised Ansible test scripts
implemented in https://review.openstack.org/381853

Depends-On: I5c1f2f0949d6b7ad7bfc4151257b081728ba956f
Depends-On: Ie379de765c6ebba958ce8e7f9dc27b7a3af74ff8
Change-Id: Ib7fe11b666322b11b1e30dea775304fd5d236f2f
2016-10-10 08:56:20 -05:00
Jesse Pretorius 687dcdc3ea Remove install_test_packages variable
With https://review.openstack.org/381479 merged, setting this
variable in tox.ini via an extra var is unnecessary.

Change-Id: I48fbeb9cdaa2c70269be2f07008ee4ffd2f04396
2016-10-04 11:45:08 +00:00
Major Hayden d001b9dda5 Initial scaffolding for RHEL 7 STIG
This patch adds the initial scaffolding for the RHEL 7 STIG content
and provides a pathway for adding gate jobs that test the tasks for
the new content.

Implements: blueprint security-rhel7-stig
Change-Id: I4cc9468977fc6c14f4ca792a8964fa7a60a4e831
2016-10-03 16:37:46 +00:00
Major Hayden 401ccd7d97 Skip V-38620 (chrony) in gate
The OpenStack CI runs ntpd in the gate images and this causes chrony
to fail on startup. This patch skips V-38620 so that chrony won't
cause gate failures.

Closes-Bug: 1629936
Change-Id: I0c67241c0725621715877e728a6c6c17d771a596
2016-10-03 11:35:51 -05:00
Jesse Pretorius ec1b42a2f9 Use centralised test scripts
This patch consumes the test scripts implemented by
https://review.openstack.org/375061 to ensure that
the tests and test preparation is consistent and
more maintainable.

Change-Id: I2c26eb12711128082a7136ab962f8239b59124b4
2016-09-28 12:16:50 +01:00
Kevin Carter e5a346f4f5 Update testing bits for consistency
Change-Id: I9d1951c5b594fb20a543d90a22fac510973d8a0d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-09-19 08:39:55 +00:00
Major Hayden e57593dfd4 Automate the STIG documentation
With the upcoming changes to rebase onto the RHEL 7 STIG controls,
there needs to be a new solution for documentation that is easier
to manage and filter. This patch automates the generation of the STIG
control documentation in the following way:

* A Sphinx extension runs early in the doc build process that writes
  all of the individual STIG control docs as well as ToC pages.
* ToC pages are now sorted by severity, tag, and implementation status.
* A giant listing of controls is easier to navigate now.
* Docs are generated from metadata in the /doc/metadata directory. New
  documentation only needs to be added there. (Will explain this in
  the developer notes in a subsequent patch.)

Implements: blueprint security-rhel7-stig
Change-Id: I455af1121049f52193e98e2c9cb1ba5d4c292386
2016-09-09 14:43:30 +00:00
Jesse Pretorius 28c73b4bce Use command to avoid alias execution for log compression in CI
To avoid executing an alias and therefore get the default behavior
from gzip, executing gzip with command is better than using which.

Change-Id: I376af163a0b7c7aec3ba5d323d3f9c4128b55735
2016-09-08 13:21:59 +01:00
Jesse Pretorius 6d67b6afc6 Rename collected logs for easier CI viewing
OpenStack-CI facilitates the ability to view compressed
files on the log server if they have the suffix .txt.gz.

This patch ensures that all collected log files are renamed
to have a .txt suffix before compressing them.

The following changes are also made:
- The bindep file is also cleaned up a little to reduce
  unnecessary duplication.
- PYTHONUNBUFFERED is set to ensure that the console log
  from the CI jobs are in the exact order of execution.

Change-Id: I89f5734275dc2789f44b5bd9c0b45dc34c4a7a50
2016-09-07 17:55:18 +01:00
Jesse Pretorius 1889953f48 Collect compressed logs after functional test execution
This change enables log collection within the gate so that further
analysis on gate tasks can be performed post build. This is very
useful when debugging problems and also for investigating the
consequences of patches once they've been tested.

Related-Bug: #1620849
Change-Id: I2bb923ebcd73114c1199b14f9b769435596091eb
2016-09-07 13:28:10 +01:00
Kevin Carter 427cd00acd Enable log collection after functional testing
This change enables log collection within the gate so that further analysis
on gate tasks can be performed post build. This is very useful when
debugging problems.

Change-Id: I41e70d0f6a0e5fed78e0a5462ee4d1730c94ec21
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-08-29 21:38:38 -05:00
Major Hayden fb33be7e68 Update to Ansible 2.1.1
This patch bumps the openstack-ansible-security role to use Ansible
2.1 and adds the python-apt package which is now required for
Ansible's check mode on Ubuntu.

Change-Id: I4899e426a7bb5623837704b49920847c1308af53
2016-08-22 21:29:21 +00:00
Jesse Pretorius 862b713fd6 Make all linting tests use upper-constraints
As per [1] all linting tests can now use upper-constraints. This patch
removes all instances of the install_command override relating to lint
testing which were needed to negate the use of upper-constraints.

[1] http://lists.openstack.org/pipermail/openstack-dev/2016-August/101474.html

Change-Id: I016b1c1f77a7cc9b2020d8455208a977cb80ffc6
2016-08-20 16:08:18 +01:00
Major Hayden f85e9e4b72 Skip SNMPv1/2 (V-38660) checks in gate
The CI gate job images have SNMPv1/2 configurations applied and
they are in the process of being removed. This patch should get
the openstack-ansible-security gate jobs unblocked by skipping
V-38660 temporarily.

When https://review.openstack.org/#/c/354819/ merges and new
images are deployed, this task won't need to be skipped.

Change-Id: I2bad2ce8bb5ba73356d224ce1093bc4f19fe75b9
2016-08-16 18:22:27 +00:00
Major Hayden 675c9e84f6 Show idempotency check output
This patch displays the idempotency check output so that issues
can be found more easily when the check fails.

Change-Id: I302d6ba581da6fc454d0fa27002e2b7d74881c82
2016-08-12 15:47:23 -05:00
Jean-Philippe Evrard c458db68fa Include ansible commands for ansible linting
The plugin repo needs to be cloned for ansible-lint to
understand ansible plugins. The commands currently
reside in tox.ini under the ansible section and are not
currently included. This commit fixes that error.

Change-Id: I499f5807afcf3c8e94d571eb7975d7f198b72538
2016-08-11 18:12:17 +01:00
Jenkins 66ba1f3c3d Merge "Ensure that doc linting is included in the linters test" 2016-07-27 18:20:53 +00:00
Major Hayden 088884c731 Ensure that doc linting is included in the linters test
The 'docs' tox target executes the doc8 lint test which may result in
failures when testing documentation builds, but OpenStack-CI does not
execute that tox target.

In order to ensure that we catch all standard documentation syntax
errors and prevent them from merging, this patch includes the docs
target in the 'linters' chain of tests.

Fixes for any failures which result from executing this test are also
included in the patch.

Change-Id: I80c2ce387e59a30c34bf2252a54037c00b420380
2016-07-25 10:45:11 -05:00
Major Hayden fa11dd430b
Add idempotency check
This patch adds idempotency checking for the security role. It
ensures that no changes are made when the security role runs
multiple times against the same system.

Change-Id: Ia5df45ddc64b1af5149df64f3483f472b06d73f7
2016-07-22 10:52:49 -05:00
Major Hayden 7f7098c25e
Restore check/audit test in tox.ini
The check/audit mode test in tox.ini was removed in
If42e739002e36669044a9396e233dbd382add4c8 with the tox.ini cleanup.
This patch adds it back into tox.ini so that it runs prior to the
functional tests.

Change-Id: I1842c8170ec532baf713ab789ffe389369c5e48e
2016-07-20 07:19:13 -05:00
Jesse Pretorius cc01563a1c Use plugins repo version of the human_log callback plugin
With the implementation of https://review.openstack.org/321331 the
human_log callback plugin is now part of the plugins repo.

This patch removes the retrofitted version in tox in favor of using
the version from the plugins repo instead.

Change-Id: Ic8c204a8bb61a041bd361dff5ec3a24bc376685a
2016-07-15 14:05:20 +01:00