The current shebang requires /usr/bin/python which is not available in
Ubuntu Jammy by default.
This also fixes some unnecessary/missing shebangs.
Change-Id: Ib25a0a7f39f68f43622609391710dd3b1abc2d00
As an operator, we want to set oslo_reports/file_event_handler because
when running behind a wsgi server, GMR cannot register the Signal to be
trigger.
The parameter file_event_handler has been designed for this specific use
case but it was not used correctly by neutron.
Closes-Bug: #2021814
Change-Id: Id13de1a3f9ea2eaaa7521eedf905aa0dd993ff89
Signed-off-by: Arnaud Morin <arnaud.morin@ovhcloud.com>
Currently logger is not properly initialized according to config
options or process arguments when the neutron-rpc-server script or
the api wsgi script is used. Because of this neutron does not generate
the log files as expected when we deploy api using WSGI and use
neutron-rpc-server, instead of the monolithic neutron-server.
This ensures the logger is initialized even when these two scripts
are used.
Closes-Bug: #2030904
Related-Bug: #1693691
Change-Id: I13bcd36c237555e0ca60d5ffc214b8f3bf6e478f
Neutron API server was not using eventlet monkey patch
thus eventlet threads couldn't be properly yielded.
This patch set API neutron server like other monkey patched
neutron services: neutron-server and neutron-rpc-server.
NOTE: this change needs apache service restart. Apache
mod_wsgi auto reload can lead to SSL RecursionError.
Co-Authored-By: Szymon Wroblewski <szymon.wroblewski@ovhcloud.com>
Closes-Bug: 1970216
Change-Id: Ib62c049cc521a548ab7e7e9584b19bdaa67b1c9d
Importing some modules lead to registering config options that may
collide with config options from a project that calls the import. This
patch wraps the side effect that registers config options into a
function that needs to be called in case the caller wants to register
the options.
This solution is also not perfect as it guards the common options to be
registered only once even if the function is called multiple times. This
is to solve problems in unittests, ideally we should always call the
function just once even in our testing suites.
Resolves-Bug: #1968606
Change-Id: Ic1532eb8de887ff1b1085206df11f53e22f7f524
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
If both are run under the same process, and api_workers >= 2, the server
process will instantiate two oslo_service.ProcessLauncher instances
This should be avoided [0], and indeed causes issues on subprocess and
signal handling: killed RPC workers not respawning, SIGHUP on master
process leading to unresponsive server, signal not properly sent to all
child processes, ...
To avoid this, use the wsgi ProcessLauncher instance if it exists
[0] https://docs.openstack.org/oslo.service/latest/user/usage.html#launchers
Change-Id: Ic821f8ca84add9c8137ef712031afb43e491591c
Closes-Bug: #1780139
neutron-api needs to load config files when deployed under
WSGI process. This patch will do this. neutron-rpc-server
also need to be aware of plugin extension.
uWSGI seems not happy with eventlet.GreenPool. So this patch
will switch to futurist.ThreadPoolExcutor for ML2 OVO push
notification.
Partially-implements: blueprint run-in-wsgi-server
Change-Id: Ia2659c9c8f1deeceefc1c75f1149a1e47c91e7a9
Closes-Bug: #1708389
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
This does the same as the logic present but it emits
the registry callback event for resources.PROCESS AFTER_SPAWN
that some plugins may be expecting.
Change-Id: I6f9aeca753a5d3c0052f553a2ac46786ca113e1e
Related-Bug: #1687896
Neutron RPC Server fails with exception
"object has no attribute 'rpc_workers_supported'".
Change-Id: I0997789bb61635ed36e4b95f03fc7d396511e19f
Closes-Bug: #1687896
This can then be loaded into a wsgi compatible web server like
apache2/httpd or nginx.
DocImpact A new neutron-api script is installed, and can be used to
start either a standalone wsgi server (calling the script directly),
or as a wsgi script loaded by a web server (e.g. by httpd/mod_wsgi).
Change-Id: Ie9bb8b33f6e1335219e574d832ac264b043d90e6
Partial-Bug: #1666779
Co-Authored-By: Brandon Logan <brandon.logan@rackspace.com>
Co-Authored-By: Ihar Hrachyshka <ihrachys@redhat.com>
This sets up the factory methods needed to have paste deploy create the
pecan app and return it. It also changes the legacy factory methods to
conditionally use the pecan factory methods if the web_framework config
option is set to 'pecan'. This way, all deployments of neutron will not
need to change their api-paste.ini files to get pecan toggled on. It
should just happen without notice once pecan becomes the default.
Also, by moving this to be loaded by paste deploy, there is a good chunk of
code that has been removed because it is no longer necessary.
Co-Authored-By: Brandon Logan <brandon.logan@rackspace.com>
Change-Id: I8b1bbea8d90fdc62715cd8b6738ad955df53d7cd
This reverts commit b1cdba1696
Original patch was reverted because it broke neutron plugin's
backward compatibility and needed more work.
This patch fixes that problems:
1) original behaviour of add_agent_status_check,
start_periodic_l3_agent_status_check and
start_periodic_dhcp_agent_status_check methods is deprecated but kept
for using in third part plugins for backward compatibility
2) new add_agent_status_check_worker, add_periodic_l3_agent_status_check
and add_periodic_dhcp_agent_status_check method are implemented
instead and are used for implementing plugins in neutron codebase
Closes-Bug: #1569404
Change-Id: I3a32a95489831f0d862930384309eefdc881d8f6
I think this needs a little more thought. This broke OVN at least and
I don't think that's good. We need to figure out how to be compatible
with existing plugins, even out of tree ones.
This reverts commit 1cafff0871.
Change-Id: Ie087fb11213cc85911483c2d32c463fa9c973e54
Forking a process when multiple threads are running is an unsafe
operation and could cause a lot of problems because only current
thread will continue working in child thread. Any locked by other
thread resource will remain locked forever.
We faced with this problem during oslo.messaging development and
added workaround to hide this problem:
https://review.openstack.org/#/c/274255/
I tried to fix this problem in oslo.service:
https://review.openstack.org/#/c/270832/
but oslo folks said that this fix is ugly and it is wrong way to add
workarounds to common libraries because projects use them incorrectly.
I think that is fair.
So this patch fixes incorrect usage of oslo libraries. In this patch
I extended functionality of NeutronWorker and add there
`worker_process_count` parameter which determines how many processes
should be spawned for this worker. If `worker_process_count` = 0 - don't
create process and spawn thread in scope of current process for worker
Then I moved all background tasks to workers and return them by
`get_workers` method. start_plugin_workers collects plugin's workers
using `get_workers` method and starts in ProcessLauncher first workers
with `worker_process_count` > 0 and only after this starts threaded
workers by simple Launcher
Closes-bug: #1569404
Change-Id: I0544f1d47ae53d572adda872847a56fa0b202d2e
The default values needed for neutron's implementation of cors
middleware have been moved from paste.ini into the configuration
hooks provided by oslo.config. Furthermore, these values have been
added to neutron's default configuration parsing. This ensures
that if a value remains unset in neutron.conf, it will be set to use
sane defaults, and that an operator modifying the configuration
file will be presented with a default set of necessary sane headers.
Change-Id: I327d1d7ce50dd4a20d781e6251a5af81c57e6ca1
Closes-Bug: 1551836
Use the same wsgi server as the homegrown method dispatch
so we get support for api_workers and plugin workers when
using pecan.
Partially-Implements: wsgi-pecan-switch
Change-Id: I9fb942651d9b52b48e8cd80b0a6ab77ec37ea1e8
This patch adds support for launching the RPC worker processes
when using the simple wsgi server for pecan.
Change-Id: I7274083c32fd57133f8b33241c5fa95a8bb9bbb3
Partially-Implements: wsgi-pecan-switch
This ensures that the config files passed on the CLI
are parsed before the choice is made based on config options
to launch pecan/homegrown wsgi handler.
Closes-Bug: #1537292
Change-Id: I577befea74d84d611cd198c92c54ae3651e2d921
- This does NOT break other projects that rely on neutron.i18n,
as this change includes a debtcollector shim to maintain those
older entry points, until they can migrate.
- Also updates _i18n.py to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:
http://docs.openstack.org/developer/oslo.i18n/usage.html
Partially-Closes-Bug: #1519493
Change-Id: I1aa3a5fd837d9156da4643a367013c869ed8bf9d
As the Pecan server only server REST requests over HTTP, this
patch introduces a new server implementing the RPC over AMQP
endpoints for agent/server communication.
However, the REST server does not yet have the ability to send
notifications to the RPC server or directly to the agents.
This patch simply adapts the ML2 plugin to run the RPC notifiers
only when initialized in the pecan server, so that notification
to agents can still be sent.
This patch therefore is tantamount to a poor man's
implementation of REST/RPC separation which will be iteratively
improved.
Change-Id: Ie471869d9b2793acdc412f13507038433f6a72c6
This reverts commit 4f1fb05959.
Merge of master went in so this can be re-applied and have the
conflict resovled.
Conflicts:
neutron/server/__init__.py
Change-Id: I917e89b2191b3d4646e66a7a313783a56cf0ace5
There are several cases where plugin initialization should be
handled after neutron-server forks API/RPC workers. For example,
starting a client connection to an SDN controller before forking
copies the fd of the socket to the child process, but then you have
multiple processes trying to read/write the same socket connection.
It is also useful for a plugin to be able to do something in only
one process, regardless of how many workers are forked. One example
would be handling syncing from an external system to the neutron
database.
This patch does 3 things:
1) Treats rpc_workers=0 as = 1. This simplifies the code for
handling notification that forking has completed. In the
existing code, calling the notification in the Worker object's
start() method would happen twice in the case where both api
and rpc workers were 0, despite there being only one process.
An earlier patch already changed the default api_workers to be
the number of processors.
2) Adds notification of forking via the callbacks mechanism.
Plugins can subscribe to resources.PROCESS, event.AFTER_CREATE
and do any post-fork initialization that needs to be done for
every spawned process.
3) Adds core/service plugin calls to get_workers() which defaults
to returning (). Plugins that need additional processes to spawn
should just return an iterable of NeutronWorkers that will be
spawned in their own process.
DocImpact
Closes-Bug: #1463129
Change-Id: Ib99954678c2b4f32f486b537979d446aafbea07b
'newapi' was too generic and not really descriptive of the conversion
to pecan. Since there isn't any ambiguity, we will just keep all of
the pecan components in a directory called 'pecan_wsgi'.
Change-Id: I61f1cfcad4ff00582e0a481400642dd656d5245e
The Pecan WSGI server is supposed to server REST requests only.
As several plugins initialize their RPC interfaces upon initialization,
this results in establishing undesired connections to the AMQP server.
This patch does not alter plugin behavior or initialization process,
but ensures that the setup_rpc operation performed by a plugin has no
effect when executed in the Pecan WSGI server.
This patch also performs some refactoring for server launchers.
Change-Id: I56384f5f964ea90d72babf911aa5639989e9c3d8
Bunch of neutron source code files are marked as executable which is not
appropriate, this patch just 'chmod -x' to all of them.
Closes-Bug: #1468564
Change-Id: Idec4ae16501d68e044e103db24ad3be0e0751e9a
It's mostly a matter of changing imports to a new location.
Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
from its thread local store and not local.store from incubator
* don't store context at local.store now that there is no code that
would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
not exist
* WritableLogger is now located in oslo_log.loggers
Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).
Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.
Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.
Closes-Bug: #1425013
Change-Id: I310e059a815377579de6bb2aa204de168e72571e
Oslo project decided to move away from using oslo.* namespace for all their
libraries [1], so we should migrate to new import path.
This patch applies new paths for:
- oslo.config
- oslo.db
- oslo.i18n
- oslo.messaging
- oslo.middleware
- oslo.rootwrap
- oslo.serialization
- oslo.utils
Added hacking check to enforce new import paths for all oslo libraries.
Updated setup.cfg entry points.
We'll cleanup old imports from oslo-incubator modules on demand or
if/when oslo officially deprecates old namespace in one of the next
cycles.
[1]: https://blueprints.launchpad.net/oslo-incubator/+spec/drop-namespace-packages
Depends-On: https://review.openstack.org/#/c/147248/
Depends-On: https://review.openstack.org/#/c/152292/
Depends-On: https://review.openstack.org/#/c/147240/
Closes-Bug: #1409733
Change-Id: If0dce29a0980206ace9866112be529436194d47e
Mostly trivial import changes.
- oslo.i18n no longer provide install() method to inject _() into
globals(), so removed all calls to it;
- removed Babel from dependencies (it will now be grabbed by oslo.i18n);
- updated tox.ini to ignore import violations for oslo.i18n.
Change-Id: I6623d551f512fb7fe9bf35ee734ed6d4c6cbc287
After oslo.messaging port is done, we'll need to initialize RPC layer
from all RPC agents. We'll put initialization into init() function, so
the first step for migration is to make agents use it.
The function is renamed to be explicit about the fact that we don't just
parse configuration by calling it, but also do other common
initializations, like setting RPC layer.
blueprint oslo-messaging
Change-Id: I71c62f270ac7a1ff1426a3f49a32133b65580e35
In oslo.messaging, local thread store is used to pass RPC request
context [1]. If we try to import oslo.messaging before monkey patching
threading library, it attempts to access unpatched storage and fails
with AttributeError.
[1]: oslo/messaging/localcontext.py#L26
blueprint oslo-messaging
Change-Id: Ied7302fcb1d3e14428540e39e3db704550027890
In the process __init__.py involved have been emptied;
vim modelines and author lines have been dropped from
the license headers affected by the change as well.
Partial-implements blueprint nicira-plugin-renaming
Change-Id: I9c535b59bb2e83f275816472e0e32b41444d3bbe
After enhancements to Oslo Message class, re-enable lazy translation to
enable REST API responses to be translated to the requested user locale.
bp i18n-messages
Change-Id: I80272fb0d063a78882611b3044c920e28cc57d6b
Late in the Havana cycle bug 1225099 was found in the lazy
translation code, and to be safe it was decided to disable lazy
translation for Havana. This change does that.
Change-Id: Ia934a7df9386baf6ae8eb9ff48c24386c47ecd23
Partial-bug: 1225099
Add support for doing language resolution for a request, based on the
Accept-Language HTTP header. Using the lazy gettext functionality, from
oslo gettextutils, it is possible to use the resolved language to
translate exception messages to the user requested language and
return that translation from the API.
Partially implements bp user-locale-api.
Change-Id: I63edc8463836bfff257daa8a2c66ed5d3a444254
The legacy 'quantum-server' script will apply the eventlet
monkey patch whereas the new 'neutron-server' entry point
does not. This prevents the server from properly handling
the REST interface.
Fixes bug #1199383
Change-Id: I498d2b02e6018425fc4538eff6803df8971f8e82
This change renames everything to Neutron while providing backwards
compatible adjustments for Grizzly configuration files.
implements blueprint: remove-use-of-quantum
Change-Id: Ie7d07ba7c89857e13d4ddc8f0e9b68de020a3d19