Commit Graph

88 Commits

Author SHA1 Message Date
Sharpz7 2f74128822 [codespell] Fixing Spelling Mistakes
This is the first in a series of commits to add support for codespell. This is continuning the process completed in ironic-python-agent.

Future Commits will add a Tox Target, CI support and potentially a git-blame-ignore-revs file if their are lots of spelling mistakes that could clutter git blame.

Change-Id: If39cba190cd4cde60ce13ae2fcde77b69776d4d2
2024-01-26 05:18:16 +00:00
Julia Kreger 8fa7e1f9d8
Handle any error from libvirt operations
A review of the helper code which wraps libvirtmod's compiled
module which implements the libvirt wrapper for Python, which
we refer to and call as libvirt or python-libvirt, reveals that
it appears possible for AttributeError and KeyError to be raised
up to the intermediate library level. While everything else refers
to libvirtError, it still seems possible, albeit remote, that
the generated code and interactions may still raise exceptions
besides libvirtError which previously would result in unexpected
behavior.

What we sometimes see on the ipmitool side is a generic
"Error: Unable to establish IPMI v2 / RMCP+ session" error which
is generated in many cases, including when no response is received.

So, since libvirtError is based on an Exception class, changes
exception handling to just catch and log based upon Exception.

Change-Id: I8159f5d1de2acb0678e7c85306413fab6999e615
2023-12-19 14:53:07 +01:00
Pierre Riteau c1b75e0a25 Remove duplicate space in log message
Change-Id: Ia48f9cf09c298cfb70a0337328df61b1c056f672
2023-05-23 22:07:06 +02:00
Julia Kreger 348e965114 preserve secure information on xml changes
When setting parameters revolving around boot
mode, options, firmware, it is necessary to
extract and edit the XML document.

Some details, however, are not automatically
extracted and libvirt must be told to provide
secure information. This change prevents us
from accidently loosing such configuration
parameters

Story: 2010382
Task: 46651

Change-Id: I0da2f37c5b630d2a180f59b74f23c027ee56e17b
2022-10-25 18:50:18 +00:00
Pete Zaitcev 07094b5cd0 Drop redundant milliseconds from logging
The current code logs the milliseconds twice:

  2020-09-21 00:24:21,706.706 3484 DEBUG VirtualBMC [-] Get power...

The documentation for the logging module in py3 is quite
unambiguous about %(asctime) providing milliseconds.
The documentation in py2 is unclear on what is supposed
to offer, but in practice it works exactly as py3.

Change-Id: Ibb050c58d5fb949316700d7d2db4c77bd62f5e25
2020-09-20 23:31:15 -05:00
Riccardo Pittau 71773f6b8d Do not import print function from __future__
We're Python 3 only.

Change-Id: I28292fa258362f4bea3865e8d96c327156bc62e7
2020-05-04 16:21:49 +02:00
Kaifeng Wang 132fe3ce24 Restore pep8 checking on import order
Currently not checking import orders, this patch restore it.

Change-Id: I101814591eaffb694a090aa640ca9ac70db2ecf1
2020-04-27 15:24:17 +02:00
Sean McGinnis 0947b53bdf
Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: I436eab174b823c1e0e028529fed9009cc38776bc
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-18 12:01:29 -05:00
Riccardo Pittau a789257e9e Bump hacking to 3.0.0
The new version enables a lot of standard flake8 checks, so a few
fixes are required. W503 is disabled as it conflicts with W504
and the latter seems to be preferred nowadays.

Change-Id: Ic1efc86ec41daf6fadb4cebb8a73b4dd158f880a
2020-03-30 16:55:25 +02:00
Mark Goddard 3449d74b4e Fix stop command by using default SIGTERM handler in children
The daemon child processes do not respond to a SIGTERM. This means that
they stay running indefinitely. This is because the manager process
installs a SIGTERM handler to propagate the signal to children, and this
handler is inherited by the children.

Return children to the default handler for SIGTERM.

Change-Id: I784e7cd9e0545a591150dc21a8232bd85672c21c
Story: 2003534
Task: 24819
2020-03-20 11:14:27 +00:00
Riccardo Pittau 4f2aa4540b [trivial] change vbmc error message
Changes error message received if client fails to connect to vbmcd.

Change-Id: Ide55ef32b456f6ebb3c8b28001d7cf1557d7179e
2020-03-09 08:53:15 +01:00
Zuul 9742c04208 Merge "Remove `vbmc` feature of starting up `vbmcd`" 2020-03-06 11:03:08 +00:00
Zuul be669452bf Merge "Fix hanging on shell-pipe" 2020-03-06 11:03:04 +00:00
Zuul 0e12336e4d Merge "Add `error` instance status to `vbmc` reporing" 2020-03-06 08:47:03 +00:00
Ilya Etingof bddca37807 Remove `vbmc` feature of starting up `vbmcd`
Removes the backward compatibility feature of `vbmc` to
automatically start up `vbmcd` daemon process if it is not running.
From now on, `vbmcd` should be started by systemd or some other
mechanism.

Change-Id: I931751d7cdd591cae6f9a1bca8d72a4b3935a45f
2020-03-06 09:12:17 +01:00
Ilya Etingof 2651f13f28 Fix hanging on shell-pipe
Properly closes standard I/O streams to prevent shell-piped
processes from hanging infinitely on a dead pipe.

Change-Id: Id9dedac4f778cf37d626f2777d519f73cc4b5f2c
Story: 2007219
Task: 38471
2020-03-06 09:08:56 +01:00
Ilya Etingof f79c9ee89a Add `error` instance status to `vbmc` reporing
Added `error` status to `vbmc list` and `vbmc start`
commands output. If the instance is failing to start, such
instance will be shown as being in `error` satte rather
than being `down`.

Change-Id: I3ad307f97442c6b30c3e60bd8186bfb30903c0b0
Story: 2006457
Task: 36378
2020-03-05 16:46:04 +01:00
Dmitry Tantsur b12197bb0d Fix logging in vbmcd
Currently we log any exceptions as INFO and without much details.
Start logging them as ERROR with tracebacks. Add missing logging.

However, don't treat KeyboardInterrupt as an exception, just exit.
Also log the successful start-up, its absence is confusing.

Change-Id: I7464ce425ed05248152f264d64950e0ab1c0aec0
2020-03-04 11:59:57 +01:00
William Caban 0884014df2 Fix error triggered by reading intergers as string from virtualbmc.conf
Add validation to fix error of "'<' not supported between
instances of 'str' and 'int'" when defining server_response_timeout
value in virtualbmc.conf. The error is trigger because the
config parameters from the configuration file are read as strings
and everything else assume that value is an integer (as the default of 5000).

The patch also include validations for server_spawn_wait and server_port to
check the values are valid integers.

Updates test_config.test_validate for test coverage

Task: 37947
Story: 2007066

Change-Id: Ie4e74585d8e14bd73e5e30b21b4dd7bd7d88f13d
Signed-off-by: William Caban <william.caban@gmail.com>
2020-01-03 12:56:56 -05:00
Riccardo Pittau 3ef73c8d8f Remove version check for encoding
Virtualbmc is now Python 3 only, encoding for ET can be set to
unicode only.
Removing import sys from vbmc.py as it was used only for that
check.

Change-Id: I35ac19308337f9fb603efa5b8a3dc97aea234cbc
2019-12-12 14:41:21 +01:00
Riccardo Pittau 26b7ff6063 Stop using six library
Since we've dropped support for Python 2.7, it's time to look at
the bright future that Python 3.x will bring and stop forcing
compatibility with older versions.
This patch removes the six library from requirements, not
looking back.

Change-Id: Iff319bfedc2c20c139419bb625aa5de83de93538
2019-12-02 17:48:54 +01:00
Ilya Etingof 216b60c961 Respect VIRTUAL_BMC for non-default config path
Adds the ability to override default configuration file location
by exporting the ``VIRTUALBMC_CONFIG`` variable, pointing to the
desired config file, into ``vbmcd`` and ``vbmc`` processes
environment.

Change-Id: I89f7cab09f4c0b2c1d2517c4b6f607ed291f235c
2019-05-09 11:02:08 +02:00
Ilya Etingof bff0e6c73a Ignore "start" command if instance appears to be running
This helps preserving backward-compatible behaviour, as previous
implementation has required the user to explicitly "start" enabled
instances. With current virtualbmc, only the master process needs to
be started.

Change-Id: I3f95cdbd497f7ae3e06d37bb6afac17331f0469b
2019-03-21 11:14:53 +01:00
Zuul abb004765a Merge "Enable debug logging in tests" 2018-08-09 14:45:15 +00:00
Ilya Etingof 8c407df1a3 Enable debug logging in tests
Since log messages interpolation is frequently postponed
till the logging time, occasional bugs in message
templates may go unnoticed.

This patch attempts to enable full logging in unit tests.

Unfortunately, exceptions caused by interpolation failures
get inhibited by the logging module code so they do not
actually fail tests. But at least those exceptions are
noticeable on stdout.

Change-Id: I1b32476c5bb631cef9b1269effd05a88a9fafd2d
2018-08-07 11:27:29 +02:00
Zuul 4e6e901b6c Merge "Fix ZMQError class usage" 2018-08-06 13:03:18 +00:00
Ilya Etingof 28f6d30311 Fix ZMQError class usage
Fix misplaced parameters in ZMQError instantiation

Change-Id: Ia4e04a64a3f52a4c3de62c37d74a72fbec5bc4fe
2018-08-03 11:30:22 +02:00
Ilya Etingof 4974df97d5 Fix crash caused by log message interpolation bug
When running with debug enabled, misnamed variable
in log message templates causes the whole app to
crash miserably. /o\

This patch makes vbmc immortal again.\o/

Change-Id: Ib863aea5699e16649a729f30d09ab4b1501d52ae
2018-08-01 13:07:17 +02:00
Ilya Etingof 9ebc73b63b Fix log message interpolation bugs
Apparently, variable interpolation in the log
messages turned out to be buggy and inconsistent.
This patch hopefully fixes all such issues and unifies
interpolation code across the project.

Change-Id: If49fd2d3c012a1792fe978c77a216a52fc52f6bf
2018-08-01 13:02:46 +02:00
Derek Higgins cad28ace17 Specifiy encoding for xml string
In python 3 we need to specify the encoding as "unicode"
otherwise bytes are returned.

Change-Id: I413ea7ec5db3316f985c38900aa46b8cdfd5d138
2018-07-30 12:06:55 +01:00
Zuul 3ce2501ca1 Merge "Make server spawn and response times configurable" 2018-06-20 11:02:05 +00:00
Zuul f2283541f1 Merge "Minor code style improvements" 2018-06-20 10:59:59 +00:00
Zuul a800f72812 Merge "Domain `start` command accepts multiple domain names" 2018-06-20 10:59:59 +00:00
Zuul 956d3fe026 Merge "Improve PID file creation" 2018-06-20 10:53:21 +00:00
Zuul d8e66cb06a Merge "Fix power status command error reporting" 2018-06-18 19:49:51 +00:00
Ilya Etingof a944852400 Improve PID file creation
This is a followup patch to [1] utilizing built-in tempfile module for
creating PID file securely.

1. https://review.openstack.org/#/c/488874

Change-Id: I419e68302995232dd12639d33821444ced9b1498
2018-06-18 20:05:38 +02:00
Ilya Etingof c70d2aa4df Domain `start` command accepts multiple domain names
This is a followup patch to [1] making the `start` command accepting
multiple domain names in the same way as `stop` command does.

Possible duplicate domain names given on the command line removed
to avoid runtime errors.

1. https://review.openstack.org/#/c/488874

Change-Id: I48987d01187e4b4878e5d501d263e958be4e7693
2018-06-18 13:28:41 +02:00
Ilya Etingof cf52ba4f1f Make server spawn and response times configurable
This is a followup patch to [1] introducing configurable timing
parameters for the client to wait for automatic vbmcd start up and
vbmcd response timeout.

1. https://review.openstack.org/#/c/488874

Change-Id: Id1cdcecd8d407b31dc69166fb0fda555fc4a679f
2018-06-18 12:27:59 +02:00
Ilya Etingof 7149836092 Minor code style improvements
This is a followup patch to [1] improve code style at multiple places.

1. https://review.openstack.org/#/c/488874

Change-Id: I38a87ecf7215cbf3ff4f94cefedc4f2a297e70f9
2018-06-18 12:26:57 +02:00
Ilya Etingof bbfa16cf0f Improve logging
This is a followup patch to [1] addressing minor
fixes and improvements in the area of logging.

1. https://review.openstack.org/#/c/488874

Change-Id: I6c703b35c32644d8769a863fbe7940a002a185f0
2018-06-15 14:36:15 +02:00
Ilya Etingof 7ace4293e9 multiprocess server, ZMQ-based management cli tool
Original design of the VirtualBMC tool was that user manages
config files for individual VirtualBMC (via a cli tool), then
requests the tool to start the instances representing
individual VirtualBMC instances (via the cli tool). Then
the instances become independent processes. The only way
to know their whereabouts is through the pidfiles
they maintain.

There were certain practical inconveniences with the
original design, namely:

* Cumbersome to start/stop/monitor free-standing
  vBMC instances processes
* No two-way communication between the parent process
  and the VirtualBMC instances what makes child state check
  or modification unnecessary difficult

This commit turns server part of the tool into a single
process spawning multiple children processes and herding
them via ZMQ client/server.

The parent process runs server part of the control
interface, maintains persistent VirtualBMC instances
configuration and ensures all its children are alive
and kicking. Each VirtualBMC instance is still a separate
parent fork.

If child dies, parent respawns it right away. If parent
is about to die, it tries its best to kill all the
prospective orphans.

This new implementation tries to stay compatible with
the original one in part of `vbmc` tool CLI interface
and behaviour. Whenever it can't connect to the `vbmcd`
it tries to fork and spawn the daemon behind the scenes.

While the threading design for this tool might look better,
the underlying pyghmi library is apparently rather
complicated to use its concurrency capabilities reliably.
The other minor consideration is that running multiple
processes leverages CPU-based concurrency.

Other changes:

* The `start` command now accepts more than one domains
  to be started

Change-Id: Ie10f4598c7039a7afa9b45d01df3b3c3db252c1d
Story: 1751570
Task:  12057
2018-05-11 17:29:47 +02:00
Ilya Etingof ba4ced4657 Fix power status command error reporting
With pyghmi API, some methods return generic IPMI error
codes, while some return payload values. This patch fixes
the `get_power_state()` method to raise exception instead
of returning IPMI error code because the latter goes against
pyghmi API.

Change-Id: If30c1ee49982a7a1198a39a7afdc1edc2c516078
2018-05-11 11:36:27 +02:00
Ilya Etingof 30e168e792 Revert "Add Serial-over-LAN (SOL) support"
This reverts commit 9f7bfb9f6b.

It seems that the SOL feature introduced by patch [1] led to
the VirtualBMC daemons leaking two file descriptors (ends of
UNIX pipe, perhaps) per each IPMI query it processes. Given the
default ulimit of 1024 open files per process, in circa 500 IPMI
queries vbmc renders itself not quite functional.

The quick research of the problem reveals that the leak comes
to life once the virEventRegisterDefaultImpl() call is made
which prepares libvirt for asynchronous operations over a dedicated
connection to libvirtd. Even though once SOL session is gone and
libvirt is cleaned up (well, dedicated connection is closed),
the libvirt connection, which is open every time when IPMI query
comes in, leaks 2 FDs per query.

This problem has been reported about some other libvirt application
in the past [2] so the problem might go away if we turn the IPMI
part of vbmc to the asynchronous operation mode as well.

Another observation is that libvirt API may not [3] offer a full
clean up upon exiting from the asynchronous operation mode.

To summarize: the real cause of this behaviour is not yet properly
understood.

1. https://review.openstack.org/#/c/482853/
2. https://www.redhat.com/archives/libvir-list/2013-September/msg00019.html
3. https://github.com/libvirt/libvirt/blob/master/src/util/virevent.c#L224

Change-Id: Id599f62e0eaf8c96e4552d2cd263f93155f8afa6
Story: 2001976
Task: 19585
2018-05-09 11:53:57 +02:00
Ilya Etingof 72f873d9f4 Report libvirt failures as IPMI-retryable
It seems that at times of high concurrency, libvirt
occassionally fails on supposedly some race condition
what leads to client failure (e.g. Ironic). This change
is to tell Ironic that it should try some more times rather
than bailing out right away.

Change-Id: I5848d721305c887fb7803ca4b302565aa4b83c88
Story: #2001798
Task: #15076
2018-05-02 18:25:55 +02:00
Akira Yoshiyama 9f7bfb9f6b Add Serial-over-LAN (SOL) support
This patch adds some methods to support Serial-over-LAN capability.
We'll be able to connect serial console of a domain with
"ipmitool sol activate" command.
SOL capability requires a persistent libvirt connection, so this
patchset adds libvirt_open.get_conn() method and use it.

Change-Id: I7f9dfbd2b3eb1320be755bade2f97b9f0621d409
2018-03-20 23:35:37 +09:00
Zuul 3a25646abe Merge "Migrate CLI to cliff" 2018-03-15 14:28:03 +00:00
Ilya Etingof eaf944f09e Migrate CLI to cliff
This patch replaces the use of argparse module
by the cliff package. The rationale is that
cliff infrastructure is easier to use and more
functional. Some other OpenStack projects seem
to migrate to cliff already.

Change-Id: Ib2185dad50e64de354222669d2e65f411ddc96a4
Closes-Bug: #1752911
2018-03-14 18:26:23 +01:00
Zuul 04d9588f68 Merge "Change numeric constants to VARIABLE_NAME" 2018-03-09 20:28:58 +00:00
Jacek Tomasiak 0e74342899 Remove devices/*/boot when setting bootdev
"boot" entries under "devices/*" are mutually exclusive with those
under "os". If such entries are present, vbmc fails when adding "boot"
entry under "os".

Closes-Bug: #1746983
Change-Id: I357d3b8a92ee764e6f47f564fdef54d3a79f0782
2018-02-05 11:31:13 +01:00
John L. Villalovos 739a2688d5 Change numeric constants to VARIABLE_NAME
There was usage of things like: return 0xd5

Change it to: return IPMI_COMMAND_NOT_SUPPORTED

Also give a pointer to the IPMI specification.

Change-Id: I7dbbe593fbf7f49aa5151bbff1678db63b5f172c
2018-01-19 11:57:13 -08:00