Commit Graph

92 Commits

Author SHA1 Message Date
Zuul df9e1ba80e Merge "[codespell] Fixing Spelling Mistakes" 2024-03-14 17:13:05 +00:00
Dmitry Tantsur 4b31cc1c48
Switch to qemu-img functions from ironic-lib 6.0.0
Change-Id: I0010d8710db4f2859dabe8ccfa5bf478d4f039f6
2024-03-11 17:16:12 +01:00
Sharpz7 949387bd80 [codespell] Fixing Spelling Mistakes
This is the first in a series of commits to add support for codespell. This is continuning the process completed in ironic-python-agent.

Future Commits will add a Tox Target, CI support and potentially a git-blame-ignore-revs file if their are lots of spelling mistakes that could clutter git blame.

Change-Id: Id328ff64c352e85b58181e9d9e35973a8706ab7a
2024-02-12 19:58:56 +00:00
Zuul 2dec169587 Merge "Fix missing oslo.versionedobjects library option" 2023-08-08 12:57:29 +00:00
Takashi Kajinami 0bda652262 Fix missing oslo.versionedobjects library option
This ensures the options for oslo.versionedobjects library are
included in the file generated by oslo-config-generator.

Change-Id: Ib63c4dd1c14905ec200e67a8fe9ba5f20b160b08
2023-08-08 15:05:22 +09:00
Julia Kreger c4e3100d5c Add hold steps
* Updates API version to 1.85 to permit an ``unhold`` verb
* Adds the ``deploy hold`` and ``clean hold`` provision states
  to the internal state machine.
* Adds on documentation on steps to help provide greater clarity
  to Ironic's users on how to utilize steps. It should be noted
  this documentation also includes the power state reserved step
  names from the DPU functionality patch.
* Fixes the state machine diagram. Changes type to PNG as SVG
  rendering is broken due to python libraries utilized for SVG
  generation which do not work on more recent Python versions.

Change-Id: I34f58f4e77e7757b89247fd64f5fcde26f679453
2023-06-30 14:34:26 -07:00
Dmitry Tantsur 4bcef0eee5 Do not log into the template1 database during test-setup
Template databases are not designed to run random SQL code. They should
only be accessed to modify the template itself. Use postgres instead.

Change-Id: Id7d38895d8d04964557447ecbc6ca29f39f626c9
2023-03-31 12:36:54 +02:00
Julia Kreger b201e52f89 [CI] Fix port list benchmark
Change-Id: Ifdcf69f605e38f4c56661c8fd3c6b8aac087fc78
2023-03-22 11:52:18 -07:00
Harald Jensås d691ee05e5 Add ports statistics to tools/benchmark scripts
Update the scripts in tools/benchmark to enable benchmarking
ports as well.

Change-Id: Ida42e8b7c483d905dbc4226d695f35aca77a8722
2022-11-18 15:55:25 +01:00
Takashi Kajinami f0935c182e Add audit middleware options to ironic.conf
Ironic supports enabling the audit middleware. This change adds
the option for the middleware to the ironic.conf file generated by
oslo-config-generator.

Depends-on: https://review.opendev.org/804316
Change-Id: Ic7adb755f47ea65fe975dfbc7cca905a376d728e
2022-07-07 15:51:07 +00:00
Dmitry Tantsur 1f509c5bde Fix the benchmark job
This is a price to pay for using private API and not having unit tests :)

Change-Id: Ie400a73eb2036678ed9446e0b499301cc2b690dd
2022-02-09 10:11:49 +01:00
Dmitry Tantsur 3f990beb97 Refactor common configuration bits from service commands
The prepare_service call from ironic.common.service is changed to also
configure guru meditation and profiler. A new call prepare_command is
provided for the cases it's not required.

Change-Id: I5b9b7b7bc827c8bcda06e9a967deae8577ad87f4
2021-12-01 12:19:33 +01:00
Steve Baker 6af0eb374e Set postgresql password encryption for FIPS compliance
This is part of the work to add jobs which confirm ironic works with
FIPS enabled, but this change is also appropriate non-FIPS jobs.

Change-Id: I4af4e811104088d28d7be6df53c26e72db039e08
2021-08-05 11:47:11 +12:00
Ruby Loo 7507dec09f Use env to find python3
Our container doesn't have python3 at /usr/bin/python3. This
changes the script so that /usr/bin/env is used to find python3.

For discussion/opinions about how (and the better way) to locate
python3:

- https://developpaper.com/the-difference-between-usr-bin-python-and-usr-bin-env-python-in-python/
- https://stackoverflow.com/questions/13879569/usr-bin-python-and-usr-bin-env-python-which-support

Change-Id: I88cab53244913b5c9bfce6729f38f93a57ddbe80
2021-06-14 17:02:54 +00:00
Julia Kreger ffff76a682 Add basic tools for benchmarking
Adds a horribly written, just hacked together little tool to help
provide sizing insight into an ironic deployment's state and underlying
performance.

Key data:
* Queries the list of node from a pure python interface level with the
  database and reports timeing for the list of nodes to be returned.
  This information helps convey how long a periodic hits the database
  just for the query.
* Requests *all* nodes using the query pattern/structure of the nova
  resource tracker, and uses the marker to make any additional requsts.
  The data is parsed, and collected, and counts identified vendors,
  if any.
* Collects basic data on conductors in terms of running, conductor groups
  as well as currently loaded drivers in the deployment.

All of this information provides operational insight into *what*
conditions exist within the deployment allowing developers to try
and identify solutions based on the unique circumstances of larger
deployments.

Also adds a utility to generate and semi-randomize data to allow us to
create a benchmark job in CI.

Change-Id: Iae660aea82db8f1c4567ee2982595ccfdf434fe3
2021-05-24 16:36:02 +00:00
Dmitry Tantsur f2e8c031cc Switch to JSON RPC from ironic-lib
Change-Id: I8b438861780c85faae7ff18646960723a1fd9876
2021-03-10 16:48:07 +01:00
Steve Baker 235115099a Generate policy.yaml.sample
Generating a yaml file called policy.json.sample is a bit misleading
because the file is not valid json, this change fixes that, with the
intention that the user can copy policy.yaml.sample to
/etc/ironic/policy.yaml and make customizations.

Change-Id: Ie6d5c8c38d785005d2bf2dc8f9f7ac42c2e8f7fb
2021-02-11 11:52:51 +13:00
Kaifeng Wang 3b06feb91e Update checking reno script to use python3
Spotted in focal container while running pep8, updates the shebang
to use python3 explicitly, also removes the unused -tt argument.

Change-Id: Icb16ac63a83379cafe9a7dac380f2b87b881adc9
2020-10-11 22:13:21 +08:00
Julia Kreger eac7226ce2 CI: update bindep for centos-8 py36 job changes
Opendev infra changed py36 defaults to run on centos8 and
of course, there is no python-devel mapping on centos.

Removed the entry and adjusted the test-setup.sh script
so the databases are started locally as they are not auto
started upon installation.

Change-Id: I826757ad73b0b14f119f9205475379b85f111383
2020-10-03 18:22:36 -07:00
Dmitry Tantsur 8aecabcfe4 Stop using md5 for __repr__ of objects
The md5 function is not available on systems in FIPS mode, just
use the callable name (which is closer to how repr usually looks).

Change-Id: I4319ce2f42f35251595306b9a77ae1f8bc55595c
2020-07-07 15:28:03 +02:00
Sean McGinnis 58693c7f4d
Make test-setup.sh compatible with mysql8
Starting from mysql version 8 it's not possible to create a user
implictly when using GRANT.

This patch makes the behavior compatible with that.

Change-Id: I2b73a532deb8782c2c0bb05070b727e9edc6e9d8
Co-written-by: Riccardo Pittau <elfosardo@gmail.com>
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-06-22 14:18:45 -05:00
Kaifeng Wang dcc049768b Fixes unusable Guru meditation report
GMR requires configuring a path for storing the report via [oslo_reports]log_dir,
this is empty by default, and currently there is no means to specify this option,
thus doesn't respond with SIGUSR2.

Though gmr is optional, test requirement is still required so that we can
generate proper configuration sample for it.

Story: 2007570
Task: 39467

Change-Id: I0f03e9d01a3c4e42a4fc8bf2d8ceb2d1a587056c
2020-04-20 09:00:03 +08:00
Riccardo Pittau 80e49538c2 Remove future usage
No need to import print_function from future anymore

Change-Id: I49164db5bd283c42602c75539c0c5fffe6ec2630
2020-03-30 10:52:29 +02:00
Pavlo Shchelokovskyy 5e8c966a40 Use HTTPProxyToWSGI middleware from oslo
currently it is impossible to use ironic-api for both internal and
public api at the same time when both of those are using (ssl
terminating) proxies as there's only one config option to override the
resource url's in responses ([api]public_endpoint).

This patch adds the http_proxy_to_wsgi middleware from oslo.middleware
to the ironic API service, which, with properly configured proxies,
makes the choice of correct URL automatic, and thus makes such scenario
possible.

As this middleware may potentially not properly handle some
endpoint URL schemas, leave the api.public_endpoint option as a backup,
but it will be ignored when proxy headers parsing is enabled.

Change-Id: I3ce6b0726b479c2835f8777957b2cb12d8098aec
Story: #2006303
Task: #36019
2019-08-02 12:29:37 +00:00
Kaifeng Wang bffb075d20 Adds bandit template and exclude some of tests
Adds bandit configuration template and exclude some of
tests that we don't want to fix for the moment.

Keeping job unvoted so that we can keep an eye on possible
issues while not breaking gate.

Change-Id: Ic577cad3b61421c04023ea887236992acb19f77c
Story: 2005791
Task: 33518
2019-06-20 15:08:43 +08:00
Riccardo Pittau f5dbf8ba0c Switch to use exception from ironic-lib
The exception modules in ironic and ironic-lib contain the same
almost identical class IronicException.
With this patch we directly use the one in ironic-lib.

Updating requirements and lower-constraints to use compatible
version of ironic-lib.

Also deprecating duplicated fatal_exception_format_errors
option.

Change-Id: I1ce0d12d912020346425fd658d3b1807607455a4
Story: 1626578
Task: 10515
2019-06-11 12:03:44 +02:00
Dmitry Tantsur c36a01a439 Publish baremetal endpoint via mdns
This change adds an option to publish the endpoint via mDNS on start
up and clean it up on tear down.

Story: #2005393
Task: #30383
Change-Id: I55d2e7718a23cde111eaac4e431588184cb16bda
2019-05-23 17:11:50 +02:00
Kaifeng Wang 7b021309ee Remove tox checkconfig
As we are not maintaining sample config file in the repo now,
the checkconfig can be removed since it's always generated from
the code.

Change-Id: I5bfd7c5cbb048a71864fac4fcfebb940529f87f1
2018-09-29 15:09:12 +08:00
Nguyen Van Trung edd0ff7f46 Update docs to portgroup with creating windows images
This change will add how to create windows images article
in portgroup docs. these images will support to create port
bounding through ironic services.

This feature has tested on Fujitsu servers successfully.

Change-Id: I1ce941a16f080fce1699d8629a7e12a2c2d83ade
2018-09-24 04:11:14 +00:00
Julia Kreger 471aafc047 Ignore bashate E044
Bashate introduces a new error, E044, which attempts to
identify any questionable conditionals that would be
incorrectly using bracketing. That being said, it is
modeled on keeping the design simple, and errors on more
condensed lines where conditionals are included.

Since we don't need this, and the case where it is erroring
is invalid as the test is overly simple, lets ignore it.

Change-Id: Ie46dcd6cd6100f3c7e35c2c817828e45bd6b7921
2018-07-21 17:19:17 -04:00
Zuul 677d9f6e5f Merge "Add optional healthcheck middleware" 2018-03-16 20:46:35 +00:00
John L. Villalovos fabcf1a402 Move execution of 'tools/check-releasenotes.py' to pep8
The 'releasenotes' gate job does not run our tox section. This allowed a
releasenote to land that had a filename in an incorrect format because
it was not created with 'reno new'.

Move the running of 'tools/check-releasenotes.py' to the pep8 section
and whitelist the releasenote that got merged.

Change-Id: Id5bd8e0cbc9186c89f682224088a8f23998d59c3
2018-03-02 12:20:27 +00:00
Jim Rollenhagen 6f439414bd Add optional healthcheck middleware
This adds the healthcheck middleware from oslo, configurable via the
[healthcheck]/enabled option. This middleware adds a status check at
`/healthcheck`.

This is useful for load balancers to determine if a service is up (and
add or remove it from rotation), or for monitoring tools to see the
health of the server.

This endpoint is unauthenticated, as not all load balancers or
monitoring tools support authenticating with a health check endpoint.

Change-Id: I7929d5f4502c3f84b1cf30526c94a458081a6a29
Closes-Bug: #1748515
2018-02-12 14:48:05 -05:00
John L. Villalovos 7e1287f243 Remove ironic_tempest_plugin/ directory
We now use the project openstack/ironic-tempest-plugin to store our
tempest plugin. All content from the ironic_tempest_plugin/ directory
has been ported to that project.

We no longer want to have the plugin content stored here so we delete
it.

Remove check in tools/flake8wrap.sh that prevented changes to the
ironic_tempest_plugin/ directory.

Change-Id: I700bd7b71472fa91f6bc02aebc055584df08e0ef
2018-01-05 15:12:13 -08:00
John L. Villalovos 435328e54f Prevent changes to the ironic_tempest_plugin/ directory
Add a check to the PEP8 test to prevent changes to the
ironic_tempest_plugin/ directory.

This will be removed when we remove the ironic_tempest_plugin/
directory.

Change-Id: I3cdbb39ebc661652ff9ceba5f3115ac26667953b
2017-12-19 09:25:44 -08:00
Jenkins 598aa07f3c Merge "[doc-migration] Add configuration folder for documentation" 2017-08-02 22:26:46 +00:00
Madhuri Kumari adcbd324af [doc-migration] Add configuration folder for documentation
This patch does the following:
* Adds a configuration folder to contain automated generated config
files for Ironic.
* Adds `oslo_config.sphinxconfiggen` to the extensions list.
* Adds `oslo_policy.sphinxpolicygen` to the extensions list.
* Adds ironic-policy-generator.conf

This is important for Ironic to be included in the list of services
on https://docs.openstack.org/pike/configuration/

Change-Id: I51a7204ce00be2588e427c1951e8be7dc7a22647
Closes-bug: #1706176
2017-08-02 13:14:23 +00:00
Ramamani Yeleswarapu 3773f17403 Enable OSProfiler support in Ironic
This patch does the following:
* Adds osprofiler wsgi middleware
  This middleware is used for 2 things:
  - It checks that person who wants to trace is trusted and knows
    secret HMAC key.
  - It starts tracing in case of proper trace headers
    and adds first wsgi trace point, with info about HTTP request.

* Adds initialization of osprofiler at start of service
  - Initialize and set an oslo.messaging based notifier instance
    to osprofiler, which will be used to send notifications to Ceilometer.

* Traces HTTP/RPC/DB API calls and SQL requests

NOTE to test this patch:
1) Make the following changes in localrc to configure DevStack to enable
   OSProfiler:

   enable_plugin panko https://git.openstack.org/openstack/panko
   enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer
   enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler

   # Enable the following services
   CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler
   ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral
   ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector
   ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier
   ENABLED_SERVICES+=,ceilometer-api

NOTE: the order of enabling plugins matters.

2) Run stack.sh. Once DevStack environment is setup, enable profiler options
   in ironic.conf and restart ironic services:
   [profiler]
   enabled = true
   hmac_keys = SECRET_KEY
   trace_sqlalchemy = true

3) Use openstackclient and run baremetal command with
   --os-profile SECRET_KEY
   [--profile can be used, but it is deprecated.]
   For example, the following will cause the <trace-id> to be printed
   after node list:

   $ openstack --os-profile SECRET_KEY baremetal node list
   .....
   .....
   Trace ID: <trace-id>
   Display trace with command:
   osprofiler trace show --html <trace-id>

4) The trace results can be saved using this command:

   $ osprofiler trace show --html <trace-id> --out trace.html

OSprofiler spec: https://review.openstack.org/#/c/103825/

Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com>
Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com>

Partial-Bug: #1560704

Change-Id: Icd3d7c62cf7442de8a77fc67f119ae9b03725f02
2017-07-07 15:58:28 -07:00
John L. Villalovos 0616c7485c Enforce releasenotes file naming
Ensure that releasenotes file naming matches how 'reno new' creates
them.

Change-Id: I796f928995fce27ca871e26d8694c655a21d0861
2017-04-05 16:41:52 -07:00
Andreas Jaeger dfc272c43d Prepare for using standard python tests
Add simple script to setup mysql and postgresql databases, this script
can be run by users during testing and will be run by CI systems for
specific setup before running unit tests. This is exactly what is
currently done by OpenStack CI in project-config.

This allows to change in project-config the python-db jobs to
python-jobs since python-jobs will call this script initially.

See also
http://lists.openstack.org/pipermail/openstack-dev/2016-November/107784.html

Update devref for this.

Needed-By: I58c1d37d620ca058a4c2127e329d8d21b391965f
Change-Id: Id21d3481f7b4929c40c7351e6321d9ba5ad078b8
2017-02-03 14:30:41 -05:00
John L. Villalovos b1b86b6417 Devstack: Create a "no ansi" logfile for the baremetal console logs
In addition to the normal bare-metal console logs that devstack
generates, create a "no ansi" version of the log that will be easier to
view/parse when viewing the logfiles.

Change-Id: Ic321db38f694d82362a6b1be91f891a06fb18c11
2017-01-26 13:15:33 -08:00
John L. Villalovos 3cedd6f924 Have bashate run for entire project
Before bashate was only running on the devstack/ directory. Have it
now run on the entire project for *.sh files.

Fix issues detected by bashate.

Also check the files:
    devstack/files/hooks/qemu
    devstack/lib/ironic

These files were located by doing:
    $ find openstack/ironic/ -not \( -type d -name .?\* -prune \) \
      -type f -not -name '*.sh' | xargs file | grep -i "shell script"

Change-Id: I238b7ab650781143fe8231e01250ab56120ff94f
2016-09-27 11:19:33 -07:00
Jay Faulkner 15c366580a Metric chassis, driver, node, and port API calls
This change adds initial metrics for Ironic based on new support in
ironic-lib. Emits timing metrics for basic Ironic API calls.

Bumps ironic-lib to 2.0.0 in requirements to add metrics support, as
well as adding ironic_lib.metrics and ironic_lib.metrics_statds to
ironic-config-generator.conf to get them in the sample config, which
is also regenerated.

Change-Id: Ic35802e4cd11763ebbedb8ddc28f7e8dc535cc2f
Partial-bug: #1526219
2016-07-15 18:15:11 +00:00
Lin Tan 773aa20a4a Make use of oslo-config-generator
oslo_config provide a utility for generating sample config files,
which provide more detail about opts, like Minimum/Maximum value
and Allowed values.
So drop Ironic's "generate_sample.sh" which was copied from
oslo-incubator long time ago.

Add a new entry point "ironic" under oslo.config.opts namespace to
explore config options to oslo-config-generator.

After this patch, new config options of Ironic code should register
with ironic/conf/opts.py. New external libraries should
register with tools/config/ironic-config-generator.conf

There is a bug #1554657 with oslo-config about deprecated_group.
This bug have impact of some configs from keystonemiddleware
and oslo.messaging in ironic.conf.sample
So currently, deprecated option should always add the deprecated_group
even it didn't alter the group, otherwise the deprecated group value will
be 'DEFAULT'.

Update etc/ironic/ironic.conf.sample via running 'tox -egenconfig'.

Closes-Bug: #1564195
Change-Id: If7721e98e69b6f54f1ee04a07477396b86583371
2016-05-25 11:59:13 +08:00
Jenkins c9f96d6d79 Merge "Show transitions initiated by API requests" 2016-02-09 10:45:14 +00:00
Ruby Loo 4f8b97dff6 Show transitions initiated by API requests
This updates the state machine diagram and documentation to indicate
which transitions are initiated by API requests versus the ones that
are internally done by the conductor. Stable states are highlighted
a bit.

Change-Id: I1a2de81b14696286f1da47c06374ad235962c849
Closes-Bug: #1527316
2016-02-01 14:39:12 +00:00
Zhenguo Niu 29b37065d7 tox: make it possible to run pep8 on current patch only
This makes tox use a simple wrapper around flake8 like
Nova, which can be told to restrict the check to only
files changed in the current command. This can be invoked
in a simple manner with 'tox -epep8 -- -HEAD'. Since most
commits only touch a handful of files, this will usually
be far faster than checking all source files.

To check an entire branch for bisectability it can be
automated via

  git rebase -i master -x 'tox -epep8 -- -HEAD'

Change-Id: I75b1cfe3198b1217f8b25877714a4af47ae7069f
2016-01-05 09:55:48 +08:00
Lucas Alvares Gomes b51f7d4d8c Run flake8 against the python scripts under tools/ and devstack/tools
This patch allows flake8 to run against the scripts under tools/ and
devstack/tools directories.

It also fix the current pep8 errors where it's needed.

Change-Id: I3f447564dd528c6efdee10a4a13f1ff1d7e1c3b1
2015-12-24 01:56:39 +00:00
Lucas Alvares Gomes cd9975dc0a Add extensions to the scripts at devstack/tools/ironic/scripts
In order to easily determine the file type of the scripts in
devstack/tools/ironic/scripts directory, this patch is adding the
correspoding extension to each script.

Change-Id: Ifb53b768b7a3b38561031c92c279fa520e289b3d
2015-12-21 11:55:56 +00:00
Lucas Alvares Gomes d7c41ebf9b Fix "No closing quotation" error when building with tox
Apparently # is treated as a comment inside the string and as
consequence the rest of the command gets stripped. This patch drops
those comments from the bashate command.

This patch also moves the bashate command to its own script under
tools/.

Change-Id: Ic44bb64419ca1465f69c91fd579e8c84d69f84b8
Closes-Bug: #1527562
2015-12-21 11:53:41 +00:00