Commit Graph

55 Commits

Author SHA1 Message Date
Ghanshyam Mann 4c9c059883 Fix tempest venv constraints for target branch tempest run
We fixed the tempest venv constraints setting for tempest test run
on base branch but forgot to do the same for target branch test run.
It cause issue when base and target branch have different Tempest
and constraints to use (for example stable/wallaby use old tmepest
and stable/wallaby constraints but stable/xena use tempest master
and master constraints), in such cases we need to set proper constraints
defined in devstack and then run tempest.

We do not see this issue as none of the job run tempest on target except
heat (https://review.opendev.org/c/openstack/heat/+/872055/5).

I reproduced the issue in grenade stable/xena by enaling the tempest run
on target:
- https://review.opendev.org/c/openstack/grenade/+/878247/1
- https://zuul.opendev.org/t/openstack/build/1b503d359717459c9c77010608068e27/log/controller/logs/grenade.sh_log.txt#17184

Closes-Bug: 2003993
Change-Id: I05fcdb5045a33997bd1a4c340a16458d88399f5f
2023-03-30 20:58:04 -05:00
yatinkarel adcb563b18 Dump Console log if ping fails
Console log will be useful in debugging
ping check fails.

Related-Bug: #2007357
Change-Id: I0f1603154eac85353aeb3a0c42da83c268174b05
2023-02-21 19:43:10 +05:30
Sławek Kapłoński 94814b7425 Remove lib/neutron-legacy leftovers
In [1] we finally got rid of the unfinished lib/neutron module and kept
only lib/neutron-legacy. It's renamed to lib/neutron now and it's the
only neutron related module in Devstack.
So this patch removes leftovers related to the old lib/neutron-legacy.

[1] https://review.opendev.org/c/openstack/devstack/+/865014

Change-Id: I2a856b15eda992f0e78ee8eff65f39646e24c936
2022-12-20 11:15:47 +01:00
Matt Riedemann 682a8736a3 Only call save_mysql_dbs if using mysql
This adds some conditional logic around when mysql
is used based on the configured database type in
the devstack rc file. TODOs are left in place for
supporting postgresql which can be fleshed out when
we have a grenade-postgresql CI job to verify the
changes.

Change-Id: I30e7ee0c8de1add3c51824e30ca7ad103bb4a876
2018-09-18 15:29:48 -04:00
Sean M. Collins 350b0e9094 Increase verbosity of SSH when attempting to connect to an instance
Change-Id: I1f53977cd9bc5256ccc88ae3ad6553f7e785013b
2016-01-06 22:40:27 +00:00
Sean Dague 0d5947e064 force smaller ssh connect timeout
The neutron experimental multinode job is experiencing some network
issues, which causes the ssh connections to take 5 minutes to
fail. This means we hit the job timeout and don't capture our dump
information properly. Force a smaller connect timeout on all ssh
connections to avoid this.

Change-Id: I8bc2c46b8fa91f35c285ac0dcf1d669c38156f5d
2016-01-05 14:40:12 -05:00
Ian Wienand d7e0a24bcc Namespace XTRACE calls
As a follow-on to Iba7739eada5711d9c269cb4127fa712e9f961695 in
devstack, where we rename the XTRACE storage calls to not conflict, do
the same in grenade

Also some minor turning-down of tracing when importing stackrc; it
floods the logs with irrelevant stuff.  And now we're tracing
properly, cleanup "echo_summary" output in the logs.

Change-Id: Ibb67a6454e4465a8d0493461e32dbae96c6a03e7
2015-10-28 11:04:25 +11:00
Jenkins 7230183997 Merge "Remove ceilometer in favor of plugin" 2015-08-26 04:52:00 +00:00
Jenkins 2eaf7fb2f8 Merge "Make ssh commands stateless" 2015-08-17 19:40:42 +00:00
Chris Dent fd3b640011 Remove ceilometer in favor of plugin
Ceilometer is moving to using in-tree devstack and grenade plugins. This
removes ceilometer from core grenade to make room for the plugin.

Change-Id: I09f47a8c24ac59e5db840bcbc7970bfb5d259591
2015-08-13 12:23:38 +00:00
Dan Smith eb1cd4c5e6 Make ssh commands stateless
This makes our ssh-into-test-instances commands completely stateless. That
means they accept any key automatically (which we already did) and not
write them out to the known_hosts file of the user running grenade. Without
this, multiple runs of grenade (or existing host keys for the IPs we use)
will conflict and cause us to fail when validating things over ssh because
of the key mismatches.

Change-Id: I0be425764cd640ada4f00300167df77fd870214e
2015-08-06 11:23:38 -07:00
Sean Dague baaa0393a9 tag worlddump files with names
This will make it easier to understand the failures by not having to
time corolate these files, but know which are the good ones.

Depends-On: I4e40eff6d0b1ef194e43b151a83206fbd50deb66

Change-Id: I7080b1d7d3f71d858ad6063c923376e49339b411
2015-08-06 14:03:47 +00:00
Sean Dague b425bb3041 make worlddump a function
This makes it simpler to call it all over the place.

Change-Id: Ib2a6296c590db7a05577abeef5c01fc455bb7b17
2015-05-12 12:10:43 -04:00
Sean Dague 8ee1af8f3b clean up upgrade scripts
The upgrade scripts have a *ton* of historical cruft in them. This is
a bulk clean up patch that does the following things:

 - ensures we source grenaderc before functions, we need this for
   devstack dir definitions.

 - moves xtraces until after sources to trim down irrelevant output

 - removes as many variables and library sources as possible to ensure
   that we include the minimum possible code in these scripts (for
   sanity and readability).

This skips the neutron upgrade script, much like in devstack, the
neutron code has so many extra variables it's not clear what's really
needed.

Change-Id: Idac9190fd33478a479f7c7dd5cb1be894bb03444
2015-05-07 17:51:44 +00:00
Sean Dague f84db387fd disable errexit when inside EXIT trap
We need to turn off errexit in the EXIT trap otherwise we break out if
it early. Rookie mistake, and prevents the rest of the debug from
happening.

Change-Id: Ida7e277dd1693a6c594b2f721ddf15c9a169f07b
2015-05-07 06:49:24 -04:00
Sean Dague e3c883f1ba don't trace function definitions
When including functions file turn off xtrace before we include other
files, to keep the output a bit cleaner.

Change-Id: I4becdba925ae6472a80b2ab279705d34af7a4c44
2015-05-06 14:21:27 -04:00
Sean Dague 1f7021338b add source_quiet function
sourcing a file under xtrace causes a huge amount of gorp in the
console, and it's rarely useful. Create a source_quiet to source
things with tracing turned off.

Change-Id: Ic0569d78314b85a8a7dab028e5895db8f37bc30e
2015-04-24 11:58:57 -04:00
Sean Dague a88dd83270 Comment function files
This provides global variable comments for various function files, as
well as license info and the #!/bin/bash lines so that gerrit will
syntax highlight correctly.

Change-Id: Ice55aff1e39084a55e1a3da6d6418a103df4d04f
2015-04-16 16:17:44 -04:00
Sean Dague e1314ad726 add nova modular resource checks
This adds nova modular service checks, including the use of the
verify_noapi phase to ensure the server doesn't go away when
nova-compute is down (which is a substantial enhancement of our
existing validation).

There is slight regression in checking multiple servers, however that
did not seem to be a bug that we actually ran into.

add our own local copy of ping_check to grenade. The version in
devstack gets diverted down a neutron path even though we actually
want to use it in a public context. We need to change that code to
work on public networks regardless of backend. Until then, we'll keep
our own copy.

Change-Id: I985ca38ef670b03802f9ad67ce19cf94cfc802ae
2015-04-16 06:40:31 -04:00
Sean Dague 7546f56cce modularize mysql db saving
let projects declare that they will have a database to be saved, and
we'll do it for them, in the mysql case. Eventually other saving
issues should be left to per project save-state.sh scripts.

services which aren't always enabled in grenade jobs have their
settings files guarded with conditionals. Eventually we'll do
conditional loading of these settings files, however for now this lets
the rest of the infrastructure around running actions based on
PLUGIN_DIRS and UPGRADE_SERVICES be *non* conditional, which
simplifies them.

Change-Id: I06cad95ca319e4403772e5fb394c0f8965e0bf9d
2015-04-10 08:15:57 -04:00
Sean Dague 176ab63307 always install the exit trap in scripts
This moves the exit trap into grenade/functions, which means all
grenade scripts will get the exit trap registered on them. This also
adds stack tracing into the scripts so that we can determine where we
were when we failed.

Change-Id: I4100628180498ff8723a1af6c0a4c10b4d8244c8
2015-04-10 06:06:24 -04:00
Sean Dague b8c64e0de2 restructure upgrades into per project directories
This moves all the project specific upgrade code into project specific
directories. This forshadows how we might break out this code into
external plugins. It happens all at once to keep a working grenade
environment on both sides of this change.

Change-Id: If47395c796bdb0ae2b8fe65bfe4b4556a6487c7a
2015-04-09 08:27:40 -04:00
Sean Dague 6bb23b443f refactor to remove functions-common
This refactor's the grenade code to remove the copy/pasted
functions-common from devstack and instead use the functions from
TARGET_DEVSTACK. In order to accomplish this we need to refactor when
the devstack trees are pulled, so that we always pull both the base
and target devstack trees *extremely* early.

We also need to include a small number of functions (mostly related to
git) in grenade to be able to bootstrap these trees. This is done in
the inc/bootstrap tree.

There are some slightly unrelated function moves into inc/upgrade to
consolidate all the upgrade functions which were scattered through the
codebase for clarity. inc/* will be the contract library interfaces
out of grenade (mirroring devstack).

To do this grenaderc has to be split between variables which can be
set without devstack functions, and variables we compute later. These
later variables will be moved into project specific directories based
on the plugin plan.

Change-Id: I1a2d1a1fed7858c4f3b51a19be2d12cc2d260a24
2015-04-08 07:24:39 -04:00
Adam Gandelman 0b60d599fd Add partial ironic upgrade support
Moves checks of DO_NOT_UPGRADE_SERVICES to a function, and
uses it to also setup for partial Ironic upgrades, where we
upgrade the entire cloud except for ironic-api and ironic-condcutor.
This also ends up adding ironic to services that are acted upon
in check-sanity and stop-base.

Change-Id: I392b41d956bc1b0c0eb8383f0eaad9b0c5461760
2015-01-26 12:35:38 -08:00
Adam Gandelman 313014ef84 Move shared devstack functions to functions-common
Syncing devstack/functions-common -> grenade/functions is confusing,
and as a result grenade/functions has been updated with grenade-specific
stuff.  This moves existing functions to 'functions-common' and isolates
grenade specific things to 'functions'.  'functions' sources 'functions-common'
so this should not require any other changes and will make it easier to keep
code synced with devstack.

Change-Id: I3b6324fc3067833075fc5487684d31593627b779
2014-10-15 11:59:21 -07:00
Adam Gandelman dff87873e6 Add Ironic sideways migration testing
This adds the ability for grenade to be used to run a 'sideways'
upgrade within the Juno release to migrate nodes from Nova Baremetal
to Ironic.  The process is as follows:

  * Deploy base Nova with VIRT_DRIVER=fake, stop base.
  * Deploy target with VIRT_DRIVER=ironic
  * Create Nova baremetal nodes and interfaces in db that map to
    VMs
  * Run Ironic migration scripts to migrate Nova baremetal nodes and
    interfaces to Ironic
  * Update baremetal flavor to reference Ironic deployment ramdisk
    and kernel
  * Run smoke tests against target

Change-Id: Icca6ed4d304c5177a8e41db14c85354d73f10edf
2014-09-05 16:14:11 -07:00
Chris Dent 819518d464 Extract database dumps to function
There was duplication between base and target dumps that is resolved
by use of arguments.

By extracting to function it becomes easier to make changes to how
database dumps are performed. The current setup is a bit hairy,
aggregating in a function makes it easier to improve later.

Change-Id: Ic5d4f13eaa7a4d842c9af0073de17e1cffad0a88
2014-08-29 14:09:43 +01:00
Adam Gandelman 156339e4e8 Add within-juno dir and within-master symlink
This adds logic to upgrade_project() to call project upgrade
scripts from the within-$release directory if performing a
sideways upgrade within the same release.  A symlink named
within-master is added and linked to the current development
release to allow sideways migration testing of trunk.  When Juno
tags are cut and K opens, a from-kilo directoy should be created
and this symlink updated.

Change-Id: I770075442d05f38515e5dadf24f2c642d14d7845
2014-08-26 15:17:41 -07:00
Sean Dague e3b58e67dc add bashate (nee bash8) test support
style checks can now be run with 'tox'. This does not yet integrate
this to the gate, but it does clean up all the existing code to
work with the currently released bash8 pip.

Done as an easy example to create a pattern for other shell programs.

Change-Id: I5ecf28cfa2551f0cf3502263a379242c8803252c
2014-06-19 17:14:47 -04:00
Dean Troyer 96a930be7d Fix error message in upgrade_project()
It's not just for Cinder...

Change-Id: I285279f5e8f4b414d75be80776a039fa98dbe5c8
2014-04-18 10:43:02 -05:00
Sean Dague 28989efa14 support proposed/* branches
provide support for named proposed branches for selection logic

Change-Id: Ia27fc54f41404a113065258e950adb2851f1ece8
2014-04-11 16:41:13 -04:00
Jenkins d8ae6d5da2 Merge "Adding the prep_cache_dir function" 2014-03-03 16:50:20 +00:00
Joe Gordon da7fb5d7c5 Remove stop nova processes from upgrade-nova
stop-base already stop services, so don't need to do it again in
upgrade-nova. This is needed So we can keep the old nova-compute up for
rolling upgrade testing. Instead of killing the entire screen process
kill all the services inside of it.

stop-base is the preferred place to manage the stop logic, because it
has access to the base devstack's stop-* commands.

Change-Id: Ifa2b6305f1f6bbe41407ca12a0f3cbc99b09f301
2014-02-24 13:31:22 -08:00
Attila Fazekas e4848f2c01 Adding the prep_cache_dir function
Adding the prep_cache_dir function from the
https://review.openstack.org/#/c/74297.

Change-Id: I9b725608fb76406192481e258b00ae822531d16e
2014-02-20 13:04:55 +01:00
Jenkins ed1bc8a928 Merge "Add clean configuration files to upgrade from H to I" 2013-12-12 12:34:25 +00:00
Sean Dague 79077660ba ignore failures to apt-get update calls
systems may have a ton of apt repos on them, many of which aren't
really needed. We'd rather take the updates we can, and not fail
on secondary repositories, especially as secondary repository fails
will stop us dead in our tracks.

Closes-Bug: #1259911

Change-Id: I08b19cd4d48fba3b4884fa4bc011cde451136594
2013-12-11 08:42:15 -05:00
Mauro S. M. Rodrigues 4c7726ea43 Add clean configuration files to upgrade from H to I
Any new configurations for Icehouse should go in one of the upgrade files in
from-havana directory.
Also moves {BASE,TARGET}_RELEASE vars forward to Havana->Icehouse pair.

Change-Id: Iac520a8de4c3b9f14e2289cfb9b6a50c4ab0ce32
2013-12-10 22:20:18 +00:00
Mauro S. M. Rodrigues ac53d2c3f9 Modify upgrade-* scripts to allow skip configuration if not needed
This allow grenade to move and test from a brand new release to master without
any change.

This way it allows to enable d-g grenade testing new-release->master where for
a just released Openstack means that we don't need any new configuration yet.

Change-Id: I3e685deb61b95116420eae9cacb6b38dc405ffe9
2013-12-10 11:46:02 +00:00
Mauro S. M. Rodrigues d0654b98c0 Refactor new options into a new config file.
From now on all releases configurations will be available, which will make
easier to test more than two versions upgrade (meaning test upgrades like
g->h->i).
Notes:
 * from-folsom files will not be necessary since we don't test
folsom->grizzly.
 * from-havana files will be added in the next patch(with the change-id follow
change-id Iac520a8de4c3b9f14e2289cfb9b6a50c4ab0ce32).

Change-Id: I3312732b8c344ce8844a82282a032e383e07e07c
2013-12-02 11:36:03 +00:00
Dean Troyer 86d0d0a80f Update functions from DevStack
Change-Id: I5abfa76e42ee3ca35c8b8fc77619d8ccfbd916cd
2013-08-30 15:09:56 -05:00
Avishay Traeger 23b8346046 Update Cinder's policy.json file for migration
Add necessary policies to support Cinder's volume migration, using a
new policy_add function which can be re-used.

Fixes: bug 1214569

Change-Id: I9baad5f933e14c5d3647786478992462d190f53e
2013-08-23 08:50:33 +03:00
Sean Dague 2a55ea070c add oslo and infra from git to the upgrade
to make grenade work like havana master we need to bring in infra
and oslo repositories from upstream git and apply global
requirements to all the repositories.

Update functions file from devstack.

Attempt to uninstall oslo.config first, to deal with weird pip
issues where 1.1.1 is still installed, but 1.2.x from git is in
the easy repo files, and nova-manage explodes.

Change-Id: I310d40a0adc9355eea91e79598bf867ad6818b79
2013-08-05 14:14:23 -04:00
Monty Taylor 8dc625eafb Use pip -e in setup_develop
We have some complex logic in here to try to do the right things with
the requirements before doing the install of the package which still
winds up being wrong in some cases. Since having written this code,
we've learned that the logic we're trying to achieve is actually what
pip install -e does. So just use that. We have to follow up with a chown
of the resulting egg-info directory, because the sudo command will cause
it to be written by root, which prevents subsequent commands from
operating without privilege in the directory.

Change-Id: I7454f0053c36b15051b616363a5fc04c0045b57c
2013-08-02 18:46:34 -04:00
Dean Troyer deb0699eac Copy current functions from devstack
Change-Id: Ib26d3dae2ab84484aca081a29a9318a4914284f7
2013-04-05 15:09:25 -05:00
Sean Dague 6cf35f0cca sanity checkpoints
After we supposedly shut down Folsom, actually check that it's all
the way down.

After we start grizzly, before we do anything, make sure that the
services look up, and the final vm that should have survived, did.

Change-Id: I13159c84444c3b3e9744acac7110b5cbd5ff9233
2013-03-29 15:59:29 -04:00
Sean Dague 96fdbef13d make the emacs mode declaration correct
Change-Id: I4f8fd64dbfecd8db6b7f36c1bfb0f82e6ae58521
2013-03-29 10:27:55 -04:00
Dean Troyer 41ab79c35b Update functions file from DevStack
Change-Id: Iac97e09db1f40e76d61164971147b307fc26b3ad
2013-03-15 15:57:03 -05:00
Dean Troyer 6c0c3b40dc Update functions from devstack 2013-02-05 17:34:11 -06:00
Dean Troyer 966de3cb5c Fix setup_develop 2012-09-29 17:44:45 -05:00
Dean Troyer 1406174c2f Fix screen_rc 2012-09-18 16:28:47 -05:00