Commit Graph

59 Commits

Author SHA1 Message Date
Ghanshyam Mann 0a53c4a961 Retire openstack-chef: remove repo content
OpenStack-chef project is retiring
- https://review.opendev.org/c/openstack/governance/+/905279

this commit remove the content of this project repo

Depends-On: https://review.opendev.org/c/openstack/project-config/+/909134
Change-Id: Idb3769e91814eb7703a3a4787f485fab1caa97f8
2024-02-17 20:47:50 -08:00
Lance Albertson 7d2ecf253c CentOS 8 support
This adds support for testing CentOS 8.

- Update documentation
  - Add references to CentOS 8
  - Update genealogy to include latest releases
  - Update minimum Chef requirement
  - Add Bare Metal to list of cookbooks when creating a stable branch
- Include memcached in identity role
- Update InSpec tests for CentOS 8
- Ensure we install the yum package on rpm distros (required by Chef to
  run properly and is missing on the default image)
- Remove yum-epel attributes in integration since it breaks on EL8

Change-Id: Ie648432021325150fdd2fd7ab7f1af10bc69b655
Signed-off-by: Lance Albertson <lance@osuosl.org>
2021-10-22 16:29:33 -07:00
Lance Albertson e84e1be514 Disable SELinux for integration testing on RHEL only
We currently don't fully support SELinux so we need to disable it for now until
we get it fixed properly. Make this a variable so that we can change dynamically
later if we need to for testing purposes.

Temporarily disable voting until we get things working properly again.

Change-Id: Ib6dec3f6ea1cbc13d9c82d372b4f65f689adc0bc
Signed-off-by: Lance Albertson <lance@osuosl.org>
2021-07-14 10:59:55 -07:00
Zuul a9e48de68e Merge "Add testing with the community built client" 2020-06-11 20:54:10 +00:00
Jens Harbott 967ea2bd5e Add testing with the community built client
Create a role for installing a chef-client implementation instead of
having it hardcoded within the pre playbook.

Create additional jobs running with cinc-client.

Change-Id: Ib6c592f684072f60962c1be5fccff035fd4c3459
2020-06-10 09:52:46 +02:00
Lance Albertson 89f4b5c1e7 Start Train development
Changed:
- Temporarily disable voting during development
- Update git URI for bind cookbook to use upstream master
- Renamed .kitchen.yml -> kitchen.yml
- Genealogy updated

Fixed:
- Idempotency with use of apt_update resource
- InSpec tests to work with Train
- Cookstyle

Added:
- Data bag for placement service
- Set deprecations_as_errors to true in kitchen.yml

Removed:
- Unused statsd cookbook in Berksfile
- Unused .rubocop.yml

Deprecated:
- LBaaS recipes and tests. This has been deprecated upstream.

Change-Id: I53a28de6aad000a246ff8f610b64dd5201c03b27
2020-06-08 10:59:07 -07:00
Dr. Jens Harbott 5111603bb5 Start Stein development
- Update references from Rocky to Stein.
- Switch to using chef-workstation
- Exclude integration group from Berksfile. This is to ensure ChefSpec tests
  should always work regardless if we're working on integration related
  cookbooks that don't interact with those tests.
- Temporarily disable voting for integration tests due to cookbook version
  conflicts
- Add tests for openstack-common::etcd and openstack-common::completions
- Update tests for Stein
- Update OVS port to check
- Add tests for lbaas but only for kitchen, not for CI pipeline
- Add suite and tests for integration-test

Change-Id: Id451b08e361acc709879a65d0d92b35640a5dba9
2020-03-23 14:22:28 -07:00
Lance Albertson cbc1b9bcf4 Refactor integration tests
This changes the logic so that we use the Berksfile in the cookbook
we're doing changes in instead of what's included in openstack-chef.
This should allow us to properly deal with cookbook dependencies in a
saner fashion with Gerrit without getting into depends-on hell.

This also required refactoring the Rakefile and roles used in Gerrit and
test-kitchen. Each cookbook now has their own role and test role which
includes specific Tempest configuration settings. This will require
updates in most cookbooks' Berksfile to include missing cookbook
dependencies such as ops-database, ops-messaging and integration-test.

Other major changes includes migrating to using InSpec for integration
testing instead of relying on simple commands included in the Rakefile.
I have replaced most of what was included in _run_basic_queries as
InSpec tests which also improves our coverage overall. Currently the
InSpec coverage is fairly basic and should probably include additional
tests, but in general it covers the following:

I've also added minimal non-voting jobs for both platforms so we can
catch any issue we might have missed downstream.

- Services are enabled and running
- Basic commands return sane output
- Openstack users, services and endpoints are correct

The all suites exclude running Tempest due to the fact that InSpec
doesn't have the ability to change the command timeout beyond 600
seconds [1] which is an issue I've run into. It's recommended to run
tempest manually when using test-kitchen.

NOTE: I haven't made any updates to the multi-node test-kitchen
environment in this patch and will plan on doing that later. I don't
believe this will break anything there.

Other various minor changes include:

- Replaced bare-metal cookbook with block-storage in minimal role
- Print out the command that run_command is going to run to help with
  debugging.
- Use different public_ip so that it doesn't interfere with people using
  10.0.0.0/24 locally (like me!).
- Add forwarded port to access Horizon
- Switch to using Chef 15 with test-kitchen
- Include log files from additional services that we were missing
- Add missing databag for aodh and telemetry_metric
- Include a real rndc key in the databag to assist with testing
- Use OSUOSL for EPEL mirror to work around public mirror issues

[1] https://github.com/inspec/inspec/issues/1675

Depends-On: https://review.opendev.org/703882

Change-Id: Idebc87ebb6acad7d0af222fb9025fefcb9bf5a1e
2020-01-30 10:10:45 -08:00
Lance Albertson 52970bdd45 Switch to using Chef 15
Change-Id: I3c981bfc19f39d869c5890e159431c50d5c97a5d
2020-01-03 13:49:05 +00:00
Lance Albertson d3a307d917 Add CentOS 7 for integration check and gate jobs
I'd really like to re-enable integration tests for CentOS since we use
it quite a bit. This should also hopefully catch any issues that might
be missed.

Change-Id: I2be43a81c5c8d4e36253b740affe7223f7a54acd
Signed-off-by: Lance Albertson <lance@osuosl.org>
2019-07-07 22:14:03 -07:00
Lance Albertson 9a1672a072 Install and use Chef Client instead of ChefDK on integration tests
Using the chef-client baked into ChefDK is causing issues with the
fog-openstack gem since it is locked in ChefDK. To work around this and
better mimic how this is used in production, switch the integration
tests to installing chef-client instead of ChefDK.

- Split out pre-run playbooks between delivery and integration (keep
  ChefDK for delivery)
- Install Chef 14 by default
- Install Berkshelf in Chef
- Install cookstyle which is needed in the Rakefile
- Update command paths so that berkshelf is available with Chef and
  ChefDK
- Update ChefDK to 3.2.30 but not higher due to issues with chefspec and
  apache2 [1]
- Upgrade fog-openstack gem to work around extraneous warnings to match
  cookbook-openstackclient
- Remove unneeded pip downgrade

[1] https://github.com/sous-chefs/apache2/issues/588

Change-Id: I0b0c2eb80dc0d831685cc7f69a1186a6148e6258
Signed-off-by: Lance Albertson <lance@osuosl.org>
2019-07-04 11:12:41 +00:00
Samuel Cassiba 39f982d42a Update cookbook references for berkshelf
- cookstyle fixes to pass rake

Change-Id: Ice10e100468e3ad96e5b929350dae0cba73489b3
2018-06-06 09:08:04 -07:00
Samuel Cassiba a317a117c8 Updated integration attributes and methods
- Implemented changes consistent with cookstyle
- DRY the environment files
- Added tempest recipes in minimal role; drop build-essential
- Corrected url for chefdk with current scheme
- Removed nova smoke test since this is handled by tempest and tends to fire
  before nova is available in integration

Implements blueprint modern-chef

Change-Id: I5c5371eed12b282a527576822b270cce8b0e76a7
2017-12-23 21:54:20 -08:00
Jens Harbott 577adaea35 Implement native zuul v3 tests
Change-Id: I5160ce88aefa2c5b70e4a4cb085867198e60b45a
2017-10-27 13:34:05 +00:00
Samuel Cassiba f6ce85b324 Initial repo Pike updates
- Switched default linter to cookstyle
- Renamed rake tasks to conform with Chef conventions
- Added repo, memcached, libvirtd overrides for testing
- Added Python wrapper recipe

Change-Id: I91d380006d8ca8b81a05c629aff444321ada13e7
2017-09-27 09:04:37 +00:00
Samuel Cassiba 2873826b76 Refactored provisioning cookbook for Ocata, ChefDK updates
- changed to a more alive fork of statsd, as att-cloud/statsd has been stale
  for years
- renamed multi-node to multinode to be consistent with OpenStack
  naming conventions
- introduced network node in the multinode scenario
- bumped mysql2 to 0.4.5 to align with upstream
- removed nova cell setup helper since it is no longer needed

Change-Id: If4d4d45981c8895ca57cba9c34fa1eea4b8871e3
Depends-On: I950605f384405b55737af77bc4224f1e18fee8e1
Depends-On: I4c26aea78220eb20fc4e5e964af93414855df5f6
Depends-On: I3112615cec3dd80c63779d164893b4171773856c
Depends-On: I97b453fc419bfbf01679dadf39a256b1f0f99859
Depends-On: Iba9ba7110dfde7635edc6706ecbbafbdbb161002
Depends-On: Ie1d5f27a9cf8803044568a31e4dae7654b02c9a1
Depends-On: I7c2950270f88cc9d54de593460009c6741913796
Depends-On: I16d6f892f325a80eb3eabd10110177246b63663f
Depends-On: Ie61ae3ca685738fe1de98fc9223d3e39535ce000
Depends-On: I63e6680cec8b66e2ece2d2627c0b413f5d401317
2017-08-02 16:58:56 -04:00
Jens Rosenboom 887f82636d Only run a minimal integration test setup
Currently integration tests running on Infra instances are failing
because they are hitting the 8GB memory limit. So we remove all but
the most basic services from being deployed, creating effectively a
minimal-in-one setup.

Change-Id: Ica74ed9a79e7aa6800a9b5e874013df99f6654aa
Depends-On: I9e7b24ba40c9f8303eae1325ced1f5a787466888
2017-05-30 06:08:21 +00:00
Jens Rosenboom c2537fdf36 Revert workaround for broken OSC client
With the depending patch, we upgrade OSC client to 3.11.0, which should
have the fix for https://launchpad.net/bugs/1677236, so we no longer
need to work around that bug.

At the same time simplify getting the UUID of the volume that we want to
attach.

Change-Id: I1c610ab362ccddaf61cec7b6d1e97535f81e0821
Depends-On: Id5606aa7e731bcab0b18aa0ff336c342449c1701
2017-05-26 07:50:40 +00:00
Jens Rosenboom 020c81d106 Update things for Ocata release
- Remove old workaround for oslo-privsep
- Set up nova cells
- Add databags for keystone fernet keys
- Drop ceilometer-{api,collector}

Change-Id: I256b9d4294282ede5235d6d86cf14d48667e9430
Depends-On: I8bc2eb5622b14457ef80cb70bf7a1d60c9fe9793
Depends-On: Ib87e9b5142b835c4be5e07b9d53e39b48b6f605e
Depends-On: I28c27caacc09a3e46eca135a6c4f5a841f4715f9
Depends-On: I77ac294fd8e1cd4e6bc39667ddfdea21c4daed8a
2017-03-30 14:28:36 +00:00
Jens Rosenboom a3b1f57fa5 Fix tempest invocation
The old run_tempest.sh script has been removed from tempest, so we call
our own wrapper that is created in the openstack-integration-test
cookbook instead.

Change-Id: I4a9d437b8227bda7fbdd0f3d41b4f7ca4c695400
Depends-On: If64fd079a8955d417daa76d431959f0335280467
2017-03-01 15:29:02 +00:00
Jens Rosenboom 5c9be13219 Fix memcached_servers setting for integration test
When running with chef-local, the search that is used to determine the
list of memcached_servers fails, leaving this setting empty for
integration tests. As a workaround, set the corresponding variable
explicitly in the allinone environmnet.

Also fix the Rakefile in order to allow the integration test to be
run more than once without issues.

Change-Id: I526de86e674477075c8ea51b31473d7a376723d7
2017-02-10 10:49:06 +01:00
mbuege 6162b549cc fix the integration test for ubuntu
- adapt parameters to python-openstackclient
- adapt options of libvirtd to work with Xenial
- create subnet
- add temprary fix for oslo.privsep caused by outdated ubuntu sources
- added temporary workover for ossdk bug
- grab the log output of mysql-default folder
- add empty bindep.txt to avoid installing unneeded fallback packages

Depends-On: I0431f09c5898bc2a42b17d27b60592ed5d2a4353
Depends-On: I29fd680ca4b16160a015fbf809cdd81860df9eeb

Change-Id: Ie867e9438bf24accfe466c117a8e02ffc9b1352c
2017-01-18 09:40:23 +00:00
Samuel Cassiba f9ffdfa11f Enables Ubuntu Xenial release (16.04)
- replaces ubuntu14 with ubuntu16
- style fixes to make rubocop happy
- converts smoke tests to python-openstackclient
- tweaks memory limits for multi_node
- corrects ethernet interface naming
- prunes misleading pre-provisioning environment files
- uses atlas for vagrant images
- makes the destroy_all recipe work again

Change-Id: I79aa5f793614f5430c613dc0b2fc241434fde1f9
Implements: blueprint newton-xenial
2016-12-12 14:13:44 +01:00
Jenkins 4b51993150 Merge "Convert provisioning recipes to cookbook" 2016-06-30 06:56:09 +00:00
Samuel Cassiba 606507409d Convert provisioning recipes to cookbook
Allows for newer ChefDK releases.

Change-Id: I6619a4852dfeb4ad0dda610b14fe097ad84db9e0
Closes-Bug: #1589764
2016-06-27 21:33:08 -07:00
Jenkins 1c53e790ee Merge "add telemetry to allinone setup, data bag items" 2016-06-20 20:29:16 +00:00
Jens Rosenboom 440a2c342c Follow symlinks when copying logs
There are symlinks in /etc/openvswitch, these cause failing log uploads
because the target will be unreadable.

Change-Id: I0c07b41161aa47e518d343196242e339fdf2281d
2016-06-20 15:02:04 +02:00
Jan Klare e4870d3dab add telemetry to allinone setup, data bag items
- adds telemetry to allinone role
- add telemetry-related data bag items
- use database as default store until gnocchi packages work on trusty
- add some basic ceilometer api querys to Rakefile

Change-Id: I87ff3df0dc92b9e9f5c7abc06727b53be521d8c7
Depends-On: If513cb2715d8266845bd6541d12005edef70f11c
Depends-On: I432555c558adfd9db07b27acd15365e333a13dc2
2016-06-14 09:41:40 +02:00
Jenkins a25d83fad1 Merge "multi_node example added and role cleanup" 2016-05-09 16:50:54 +00:00
Jan Klare 49bc780dd4 multi_node example added and role cleanup
* remove unused roles (may be refactored and added back later)
* added multi_node scenario to deploy controller + 2 compute nodes on vagrant

Depends-On: Idc605fed9066abbfbc1d6cd67805bf2a1aa1c31c
Change-Id: Ie32b743c09cdd3f805a374e1cb9e15f07978166a
2016-05-09 17:36:06 +02:00
Mark Vanderwiel bd4534fc81 Enable tempest tests
Most of the changes are in the integration cookbook, they consist of
- Setup a nano flavor to avoid Not Host Available issues
- Run just smoke and in serial to avoid out of memory issues
- Setup a floating ip range
- Disable volume snapshot
- dont install python virtualenv
Share the local-net to allow tempest to use it as well.
Create volumes per pass to avoid collisions.

Depends-On: I42216a174930b17a4771e78c639da54e170a288c
Change-Id: Ia988fd4a8aec84d3e4dd8c45a4f49eb401875504
2016-05-04 16:34:42 +00:00
Jenkins 1a3edc5155 Merge "Give cinder time to create volume" 2016-04-19 09:38:34 +00:00
Mark Vanderwiel 6c965940e8 Give cinder time to create volume
Avoid error trying to use volume before create is done:
volume 'xxx' status must be 'available'. Currently in 'in-use'

Change-Id: I30231c6bfb30de07685ad2136fce18595302eac2
2016-04-15 17:59:31 +00:00
Samuel Cassiba a1014325e5 Capture mariadb logs for centos
- Adds mariadb to list of projects for log shipping

Change-Id: Ibf4603131390594746b823c43ce59e377b356e39
2016-04-13 14:32:52 -07:00
Jenkins 4825218378 Merge "Make conditionals more consistent" 2016-04-11 07:43:36 +00:00
Samuel Cassiba c8a9c6604f Make conditionals more consistent
Other OpenStack projects use the existence of apt-get/yum to determine
platform, and not redhat-release/lsb-release. This change corrects that
discrepancy.

Change-Id: I8d9013a3bbad9254b2158ec7f0f89cc98b8b7e2f
2016-04-09 09:40:40 -07:00
Mark Vanderwiel daadb4516e Only run setup tempest once
No need to run it with each pass

Change-Id: Ie208be823147aad971825a30a29c5ce6eba204f6
2016-04-08 12:14:59 -05:00
Jenkins 745f01e2cf Merge "Add a few more projects to the logs" 2016-04-08 07:54:17 +00:00
Samuel Cassiba 66045d888d Differentiate between Ubuntu and CentOS for integration
Change-Id: I438fe85a2883029973fa2f5f8b263b7689334c8f
2016-04-06 10:18:00 -07:00
Mark Vanderwiel 017858a47e Add a few more projects to the logs
Added a few more core projects to the logs.

Change-Id: If8b784baadb761f1e92c9014c595361096384add
2016-03-30 12:47:53 -05:00
Jens Rosenboom 4a76bf5a82 Place log directory in Jenkins workspace
When the integration task is run from some cookbook, our CWD is not the
Jenkins workspace, but one level below, so we need to use the proper
destination for our logs to be found by Jenkins after the run.

Change-Id: I5f9e9bb3c642634c65902e417f3e635008331761
2016-03-30 02:00:38 +02:00
Jens Rosenboom 960b72804a Use new devstack-logs publisher
Instead of pasting all the logs onto the console, we can now have them
saved seperately by the devstack-logs publisher.

Add "|| true" rescue commands to the chef-client invocations, so that
rake will continue to run through to the save_logs even if the chef run
fails.

Change-Id: I475509d770e90da38ba76d4fe126d6610ef280e2
2016-03-11 13:05:08 +01:00
Jenkins 960c05c54b Merge "add heat back to allinone role after refactoring" 2016-03-09 17:48:01 +00:00
Jan Klare 7fc714104b add heat back to allinone role after refactoring
* added orchestration recipes to integration test (allinone role)
* added heat queries back to basic_queries in Rakefile (were removed previously
  until refactore)

Change-Id: Ic1a1863343be0d0d21b66a804af18c67ee625ee0
Implements: blueprint cookbook-refactoring
Depends-On: I16a29e28068d106f0edcbe04cb529aabbbed1ac5
2016-03-08 10:55:54 +01:00
Jens Rosenboom 130e3655ef Use "--no-color" option for chef-client run
In order to make the logs more readable, tell chef-client not to output
lots of color codes.

Depends-On: I75088a00dc17b8a4907a1314d82aff500c5dccfe
Change-Id: I2ae9eb2daa7674cf4a8d922f1337d8c4c47b8475
2016-03-04 14:15:16 +01:00
Jens Rosenboom 1974405090 No longer purge libvirt0
The package is no longer installed on recent infra nodes. Also the bug
that triggered the removal in the first place seems to have been
resolved some time ago [1].

[1] https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1499199

Change-Id: I2a69abab35c154b5d20c228cc3941c8e69fba7d9
2016-03-03 15:28:31 +01:00
Mark Vanderwiel c1965ae7a7 fix ubuntu-cloud-keyring gate issue
The images are "pre-cached" with apt repo data and this can become
stale and in our case we hit missing packages like:
ubuntu-cloud-keyring.  I added a apt cookbook option to force
the repo cache updates to happen each time. This seems to fix the
issue.

Change-Id: Icfd075e9d2eb4ea2f9768b09f5075b3b1ed3d383
2016-02-16 12:25:12 -06:00
Jan Klare 481af2f96a adaptions for refactored cookbooks in mitaka cycle
* added new test scenarion allione (and all the needed files for it)
* removed old test scenarios aio-nova and aio-neutron as well as multi-neutron
  and multi-node and the related files since they will not work with the
  refactored cookbooks
* removed databag items for currently untested service cookbooks
* modified Rakefile to use new test scenarios for local and jenkins integration
  testing
* adapted README and docs to fit new test setup and explain the new
  attributes/template logic (with TODO)
* adapted Berksfile to use the master apache2 cookbook to comply with the
  dependencies in identity and dashboard
* removed image_upload recipe since this is already done in the
  integration-test::setup one
* moved recipe integration-test to a different chef_run to allow all services to
  be restarted in delayed actions of first run before using them
* removed heat from the testing stack until refactored
* removed tempest unit tests 'run_tests.sh' from integration testing scenarion,
  since there is no need to run tempests unit tests in our deployment

Depends-On: I0547182085eed91d05384fdd7734408a839a9a2c
Depends-On: I3262b2e6f792f37c32a446e6567790b82bdd4613
Depends-On: I80ed8a7892b59f7fd582505445a31b0df8d68752
Depends-On: Ia7fd927f42f4a1e484e8f2d34c52b1dc6d0ce6dd
Depends-On: Ifd11f86c576d7dbb709bdd302d41e39e900e1b74
Depends-On: Ida408a025f1a3e6a632108a9a32877026e286116
Depends-On: Idadc97bd7380d6c4f9f6f33d9c6b1215a5f24772
Depends-On: I9ac9eeb29ab27f31394830e4b6f999d5870cc0e4
Depends-On: Ifa5a7f4e1df47a3961976e64f654224864c3dcb4
Depends-On: I9cc1b5cc069987ac83e064322c2291772505ff5f
Implements: blueprint cookbook-refactoring

Change-Id: Icf0921229e9473a842d7ce5706027d56229e11d4
2016-02-10 09:25:55 +01:00
Mark Vanderwiel f890bfeb0d Fix interface selection to be automatic
Added a list of possible interfaces for which the
vagrant provisioner will choose the first one that
is available.  No more editing the test recipes.

Cleaned up the os selection to hide the
version specific stuff to be consistent with the
environment logic.

Fixed up some file modes.

Forced the chef client config directory as it can search
in bad places in some scenarios.

Change-Id: I38a7a86b58893b912af99fabf04ca9052f914346
2015-11-06 11:04:32 -06:00
Mark Vanderwiel fb1b9d8a1f Add heat into single controller roles
Keep flushing out our stack by adding heat into the default mix.
I look into adding a simple heat stack integration test with
another patch.

Fix length of heat auth_encryption_key to 32 chars

Depends-On: Ie8d62b449ec272f5cb7a06acdd54b3437aa57f98

Change-Id: I683aafcb060cdcb80ff7ccd025a5bd228c9b41c7
2015-10-02 14:56:13 -05:00