Commit Graph

89 Commits

Author SHA1 Message Date
Sven Kieske a2fe33adb8
add new kernel capabilities
CAP_PERFMON, CAP_BPF and CAP_CHECKPOINT_RESTORE
are added.

Change-Id: I5f7758b7284fc16251cccd9114c3ec0e4cc95f93
Signed-off-by: Sven Kieske <kieske@osism.tech>
2023-12-19 16:07:09 +01:00
Zuul e46ad9c44d Merge "setgid should be called before setuid" 2023-07-14 15:22:09 +00:00
OpenStack Proposal Bot 1f9d71260e Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I3f263194f71469ca9ca67f3d99b10dc75c448e20
2023-06-22 04:16:41 +00:00
Max Lamprecht 33fec1971b setgid should be called before setuid
If you setuid to a non-zero value first(meaning you're no longer root),
then call setgroups, the effective uid of the process
is now no longer root, meaning that the internal setgid call fails

This also removes the duplicated if loop

Closes-Bug: #1628360
Change-Id: I5d66fccd9ffb07df0c2e4435ec3da767b3b61117
2023-03-03 08:49:26 +01:00
Rodolfo Alonso Hernandez c2b6df05e0 Setup logging without fixing evenlet logging
Since [1], ``oslo_log.log.setup`` can be called without applying the
fix for eventlet in native threads [2]. This fix clashes with the
oslo.privsep logging handler that replaces the original one. This
handler is implemented to allow the sync between the daemon process
and the process making the privileged call.

Once the oslo.log library version is bumped to 5.0.2, the try clause
can be removed.

[1]https://review.opendev.org/c/openstack/oslo.log/+/864252
[2]https://review.opendev.org/c/openstack/oslo.log/+/852443

Closes-Bug: #1995514
Related-Bug: #1995091
Change-Id: I7a4c55228064cb2dd4f4a359cdd81fd288baaf68
2023-02-08 10:58:08 +01:00
Takashi Kajinami e62d408071 Remove logic for Python < 3.8
... because now this library supports only Python >= 3.8.

Change-Id: I6146d7421f4eaf613419a6887b208549d85a61d1
2022-07-17 01:02:03 +09:00
songwenping fa16f4dc6e Remove unnecessary unicode prefixes
All strings are unicode by default in Python 3. No need to mark them as
such.

Change-Id: I506a7bcd8fb3de2088bf37ebbb117896de9ddc77
2022-04-19 15:11:57 +08:00
Zuul 73610db919 Merge "Add note explaining max_buffer_size value" 2022-04-11 21:59:32 +00:00
Zuul 9bdd2bf457 Merge "Bump max_buffer_size for Deserializer" 2022-03-22 14:10:04 +00:00
Stephen Finucane 394bd3782f Add note explaining max_buffer_size value
Follow-up for change I135917522daff95377d07566317ef0fc0d16e7cb

Change-Id: Ibc63dcf6d130c9d55a6f4c1f38c2da928fe2a4bd
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2022-03-22 12:32:38 +00:00
Zuul 2b26dea61a Merge "Remove six" 2022-02-11 19:02:07 +00:00
Stephen Finucane 7f7b9d921e Remove six
This wasn't actually recorded in our list of dependencies, but we were
using it all the same. In any case, it's no longer necessary so remove
it.

Change-Id: Ia29fdf5058c2b22327cb0ba16c28bef3660e9ceb
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-12-21 11:35:00 +00:00
Mohammed Naser c223dbced7 Bump max_buffer_size for Deserializer
Since msgpack 0.6.0, some limits were introduced for the
deserializer which were put in to avoid any denial of service
attacks using msgpack.  These limits were raised to 100MiB
in the release of msgpack 1.0.0.

The default buffer sizes that were implemented were quite low
and when running certain `privsep` commands, especially for
Neutron when using linux bridge, where there is a large amount
of netdevs, privsep would crash since msgpack would fail to
decode the message since it considers it too big:

  ValueError: 1174941 exceeds max_str_len(1048576)

In this commit, the `max_buffer_size` is bumped to the value
that ships with msgpack==1.0.0 to allow for users who don't
have that to continue to function. Also, since `msgpack` is
only being used by the internal API, we're not worried about
a third party coming in and overwhelming the system by
deserializing calls.

This fix also addresses some weird behaviour where privsep
will die and certain OpenStack agents would start to behave
in a strange way once they hit a certain number of ports (since
any privsep calls would start to fail).

Closes-Bug: #1844822
Closes-Bug: #1896734
Related-Bug: #1928764
Closes-Bug: #1952611
Change-Id: I135917522daff95377d07566317ef0fc0d16e7cb
2021-12-01 13:25:43 +04:00
Corey Bryant 4f1450677f Add except path with exception debug to send_recv
The related bug resulted when an exception occurred within the
future.result() call. This caused the finally block to be executed,
and therefore myid to be deleted from self.outstanding_msgs prior
to _reader_main() checking if the msgid not in self.outstanding_msgs.
This caused _reader_main() to raise an AssertionError because the
msgid was no longer in outstanding_msgs. This is a small step forward
to log a warning when this siutation occurs.

Related-Bug: #1927868
Change-Id: I2eed242e0c796b8a2aa3d1b21bd1da4c497f624d
2021-08-02 13:24:16 -04:00
elajkat f7f3349d6a Add timeout to PrivContext and entrypoint_with_timeout decorator
entrypoint_with_timeout decorator can be used with a timeout parameter,
if the timeout is reached PrivsepTimeout is raised.
The PrivContext has timeout variable, which will be used for all
functions decorated with entrypoint, and PrivsepTimeout is raised if
timeout is reached.

Co-authored-by: Rodolfo Alonso <ralonsoh@redhat.com>
Change-Id: Ie3b1fc255c0c05fd5403b90ef49b954fe397fb77
Related-Bug: #1930401
2021-06-23 09:43:33 +02:00
Gorka Eguileor 71c732c833 Allow finer grained log levels
Currently all privsep calls use the same logging level as they use the
``oslo_privsep.daemon`` logger.

This creates a problem for services like nova where we don't want to
log its own privsep debug messages by default (even in debug mode) but
we want them for os-brick privsep calls.

This patch introduces a new string parameter to ``PrivContext`` called
``logger_name`` to control the name of the logger to use, which defaults
to ``oslo_privsep.daemon``.

Related-Bug: #1922052
Change-Id: Ie7cc25ed53563dd500f7b10ff43a48518d46a2e0
2021-05-10 19:17:41 +02:00
Rodolfo Alonso Hernandez 1dc378c76f Undo the eventlet monkey patch for the privileged daemon
Change-Id: I422125b137a3beadb0a79f5944a19fce62f093d6
Closes-Bug: #1887506
2020-07-15 14:48:08 +00:00
Dirk Mueller a602d03e27 Replace assertItemsEqual with assertCountEqual
assertItemsEqual was removed from Python's unittest.TestCase in
Python 3.3 [1][2]. We have been able to use them since then, because
testtools required unittest2, which still included it. With testtools
removing Python 2.7 support [3][4], we will lose support for
assertItemsEqual, so we should switch to use assertCountEqual.

[1] - https://bugs.python.org/issue17866
[2] - https://hg.python.org/cpython/rev/d9921cb6e3cd
[3] - testing-cabal/testtools#286
[4] - testing-cabal/testtools#277

Change-Id: Ib8a08f1b44712bf2fbcadecdcecd0ee3dd8ab361
2020-06-11 11:51:40 +02:00
Sean McGinnis f142210025
Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: Icac647c8e89cd94cae69e357b009d676b1c744f7
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-03-31 14:36:00 -05:00
Zuul bdb8030582 Merge "Make compatible with msgpack 1.0.0" 2020-03-21 22:24:34 +00:00
Zuul a661079be3 Merge "Add lock around channel creation" 2020-03-21 22:12:37 +00:00
Ben Nemec f19765c683 Make compatible with msgpack 1.0.0
There are a couple of things that changed in msgpack 1.0.0 that were
breaking oslo.privsep:

1) The encoding parameter to Unpacker was removed. This has been
   deprecated for a while in favor of the `raw` parameter[0], so this
   change switches to using raw.

2) The strict_map_key parameter default was changed from False to
   True.[1] I haven't found an explanation of why this was done, but
   we can explicitly set it False to maintain the previous behavior.

Change-Id: Ia97ecf965d807f12524d5b6602446934b5813ce6
Closes-Bug: 1855914
Closes-Bug: 1864811
0: https://msgpack-python.readthedocs.io/en/latest/api.html#msgpack.Unpacker
1: 6e1d12c0a2
2020-03-18 21:02:42 +00:00
Dirk Mueller 52747777df Disable logger validation during unit testing
The unit tests set format to "dummy", which gets rejected by Python 3.8
by default due to newly added validation here:

  18fb1fb943

We can skip this and restore old behavior by disabling validation.

Change-Id: I22ba9c4437787c3f338f17a0b006132b0dd09df5
2020-03-17 08:39:06 +01:00
Darragh O'Reilly bbe24aa6e0 Add lock around channel creation
When a neutron agent starts up, it processes different resources
in separate eventlet threads. These can race creating the channel
which results in redundant privsep-helper processes. This patch
fixes that by adding a lock around channel creation.

Change-Id: I5de22b72059133b05d64be47f4c1d3f566b46a6e
Closes-Bug: #1864664
2020-03-13 17:22:32 +00:00
Ben Nemec dfcd983ee3 Add functional tests
These add some coverage to functionality not covered by the regular
unit tests, but I think they can't be run in the gate because they
use sudo.

Change-Id: Ic62c95b83f68f94328deb00227b7eabf249ce898
2019-12-11 10:26:01 +00:00
Eric Fried e896ed39c4 Self-resetting PrivContext
When nova-compute, an oslo.service, receives SIGHUP, running PrivContext
clients terminate, resulting in the shutdown of their corresponding
ClientChannel threads. Subsequent attempts to execute privileged methods
fail with EPIPE as a result.

The PrivContext._wrap'per (the meat of the entrypoint decorator) already
had a check to lazily start() the ClientChannel if that hasn't already
happened.

This commit makes ClientChannel store state indicating whether it's
still running; and adds logic to PrivContext._wrap to check that state
and reset (stop() and re-start()) the ClientChannel if it was previously
created but terminated.

Change-Id: I8096fc7fd014e6dd299fae8ab073336c7cae362a
Closes-Bug: #1715374
2019-08-28 15:35:00 -05:00
Rodolfo Alonso Hernandez 3574210641 Pass correct arguments to six.reraise
six.reraise input arguments are:
  six.reraise(exc_type, exc_value, exc_traceback=None)

This information is provided by sys.exc_info() and stored
in self.communication_error.

Trivial-Fix

Change-Id: Ibe68d8e36ca424da61a0a9eea20e622440c79464
2019-05-17 16:10:12 +00:00
Rodolfo Alonso Hernandez c9cec8e87b Convert dict keys received in _ClientChannel from byte to str
When executing in debug mode, the arguments passed to [1] are stored
in a dictionary populated with bytes instead of strings. When in [2]
"record" is created and updated [3], the values contained in "msg[1]"
are not updated in "record"

This patch converts all keys in byte format to string (Python 3).

[1] 130d7155c4/oslo_privsep/comm.py (L128)
[2] 130d7155c4/oslo_privsep/daemon.py (L210)
[3] b5409dacc4/Lib/logging/__init__.py (L415)

Change-Id: Idec9ecc9fcc8c5b6779e0194a16cd861c7895d7e
Closes-Bug: #1816804
2019-04-04 11:06:08 +00:00
Eric Harney e7e77096f2 Add sample_default for thread_pool_size Opt
When generating a sample config, this shows
the real default behavior rather than recording
a number at sample config generation time.

Change-Id: I39e5200f795b52e61ab424a5fda4749086e458e2
2019-02-27 14:51:50 -05:00
ZhijunWei b033c2f346 Update hacking version
update the hacking to latest, due to[1]

[1]: https://github.com/openstack/oslo.privsep/blob/master/HACKING.rst

Change-Id: I760d81aa7b19c8df15bf9f187de8ea4945579a9c
Closes-Bug: #1815714
2019-02-13 14:46:48 +08:00
Zuul 225e975abf Merge "Expose privsep options for config-generator" 2019-01-23 19:07:07 +00:00
Zuul 6b5bbbb148 Merge "Set unicode_errors handler to 'surrogateescape' in msgpack" 2019-01-15 18:47:14 +00:00
Ben Nemec c913b972b7 Expose privsep options for config-generator
We were missing the oslo-config-generator entrypoint in this library,
which meant there was no way to include the privsep opts in a
service's sample config file.

One complication is that it is possible to override the group name
for the opts. I assume this was done to allow services to run
multiple privsep daemons with different levels of permission, but
it means that the service will have to document any additional or
different group names that it may use.

Change-Id: I1ef30cb14d365f4bbfa580e75afc3580910d6fcf
2019-01-10 19:56:24 +00:00
Sam Wan 966cace7aa Set unicode_errors handler to 'surrogateescape' in msgpack
External calls may return non-unicode data which cannot be encoded, eg:
'\udc82'. As a result, we need to set a handler for unicode_errors.

Change-Id: I8d913c12cc78b69957a750323cf229bc648c8927
Closes-Bug: 1806842
2018-12-18 21:33:26 -05:00
Zuul 7b31740c09 Merge "Use threads to process target function" 2018-12-12 15:35:40 +00:00
TommyLike f368430f13
Use threads to process target function
This patch takes advantages of threads to make privsep daemon process
concurrently.

Change-Id: Ib20b27d5ea07bd0af61891c7d8c0d352a393aa21
2018-11-02 09:28:34 -05:00
Chuck Short 9391cbf297 Replace assertRaisesRegexp with assertRaisesRegex
This replaces the deprecated (in python 3.2) unittest.TestCase method
assertRaisesRegexp() with assertRaisesRegex().

Change-Id: Ic1aad86680f4b2d6b4e694f8427c6c21c2393590
Signed-off-by: Chuck Short <chucks@redhat.com>
2018-08-27 11:10:06 -04:00
Claudiu Belu b1f50df3b1 Avoids calling ffi.dlopen(None) on Windows
ffi.dlopen(None) cannot work on Windows and it is a
known issue [1]. This patch avoids this call on Windows
hosts.

[1] https://bugs.python.org/issue23606

Change-Id: I44436146cdc402087f33f4b99054176134ec5b81
Closes-Bug: #1788025
2018-08-19 10:42:55 -07:00
OpenStack Proposal Bot 840a390f28 Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I708ab6dd6cce830033f7b2350621fe7178f14fc4
2018-03-01 07:02:35 +00:00
Zuul 2e7308aa9b Merge "add bandit to pep8 job" 2017-12-19 03:53:11 +00:00
Zuul e93b540996 Merge "Expose caps values/names as int enum" 2017-12-15 14:54:30 +00:00
Tony Breeds 42f182828e Expose caps values/names as int enum
This avoids some of the lookup needed via globals()
and such to create a name and value mapping and makes
it a little easier to document the various caps

Based on: I0325fc3229adfe851c4692e07d61433da3961c46

Change-Id: I784e3e6bffbe18bf3712322efdcedcab927b4a78
2017-12-15 12:00:01 +00:00
ChangBo Guo(gcb) 518a811b9f add bandit to pep8 job
Add the bandit security scanner to the pep8 job.
* convert assert statement to raise AssertionError
* Don't hard code temporary file path
* skip B404,B603

Change-Id: If4bdb9569236927449648a8b750ae0fa2da76f53
2017-12-12 14:21:30 +08:00
OpenStack Proposal Bot 79b44227bd Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I7cce5a152bbdcac75e237b923ec6beed890f2cbf
2017-08-23 10:45:06 +00:00
Tony Breeds f53d0837db Update capabilities from current kernel source
These capabilities are from the 4.11.11 kernel tree and a current git
master 235b84fc862ae2637dc0dabada18d97f1bfc18e1

Change-Id: I1e6f7bff66495cdb83428a6c3b62804e5d20fc3b
2017-07-26 21:57:10 +10:00
ChangBo Guo(gcb) 166724787f Update URLs in documents according to document migration
Change-Id: I61a8418023aa3dc6635073812f966bf6b815f733
2017-07-12 23:09:35 +08:00
Doug Hellmann 2574b6c0b7 add sphinx instructions to build API reference docs
Change-Id: I216d6e8ef7afa7c1de47f28a6e33ca0a10975912
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-07 14:51:10 -04:00
xgwang5843 0cc0f2cd99 Using assertIsNone(xxx) instead of assertEqual(None, xxx)
TrivilalFix

Following OpenStack Style Guidelines:
[1] http://docs.openstack.org/developer/hacking/#unit-tests-and-assertraises
[H203] Unit test assertions tend to give better messages for more
specific assertions. As a result, assertIsNone(xxx) is preferred over
assertEqual(None, xxx) and assertIs(xxx, None)

Change-Id: Ieb657a70e94702317d0a7bb254e78d40200e51f6
2017-06-05 09:32:20 +08:00
Hongbin Lu 99e997e90c Add test to verify log record can be formatted
This is a tech-debt for I6948e766d08c3e781a699cca8212b4303aae3273

Change-Id: I4fac5004ec778c8cc8212ef4b109995429841b75
Related-Bug: #1674185
2017-05-17 08:51:15 +00:00
loooosy 1bc87fd868 Remove log translations
Log messages are no longer being translated.
This removes all use of the _LE, _LI, and _LW
translation markers to simplify logging and to
avoid confusion with new contributions.

See:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html

Change-Id: I2e7d8d7643fe6598b3c75c48a801b8d71e5db038
2017-03-26 13:41:14 +08:00