We spawn a lot of neutron-servers, on all but the smallest systems.
It's often hard to tell which are busy/overloaded or spinning.
Add an option to set the process names to their role.
This has a small chance of breaking existing scripting, depending
how they're parsing ps output.
Sample output:
$ ps xw | grep neutron-server
1126 pts/2 S+ 0:00 grep --color=auto neutron-server
25355 ? Ss 0:26 /usr/bin/python /usr/local/bin/neutron-server \
--config-file /etc/neutron/neutron.conf \
--config-file /etc/neutron/plugins/ml2/ml2_conf.ini
25368 ? S 0:00 neutron-server: api worker
25369 ? S 0:00 neutron-server: api worker
25370 ? S 0:00 neutron-server: api worker
25371 ? S 0:00 neutron-server: api worker
25372 ? S 0:02 neutron-server: rpc worker
25373 ? S 0:02 neutron-server: rpc worker
25374 ? S 0:02 neutron-server: services worker
The "normal" looking ps output is the main parent.
Partial-Bug: #1816485
Depends-On: https://review.openstack.org/637119
Change-Id: I0e664a5f8e792d85b8f5483fb8c6f1cd59a677cd
neutron-lib contains the WorkerBase class [1] that is the equivalent of
neutron's WorkerSupportServiceMixin. This patch switches over to
neutron-lib's version.
Note: IIUC no consumers are using WorkerSupportServiceMixin so this
patch can land without waiting for them to sync-up. I've included
the lib impact tag just in case.
NeutronLibImpact
[1] https://review.openstack.org/#/c/424151/
[2] http://codesearch.openstack.org/?q=WorkerSupportServiceMixin
Change-Id: Iacf1b6dfe318e3e6cfc76e61c65d407cf9ae7b36
neutron-lib 1.6 is out and among other things contains the rehomed
NeutronWorker class; now named BaseWorker in neutron-lib. This patch
updates references to use the lib version.
Since there's only 1 other consumer of these changes [1], this
patch depends on it so that it's free to merge after the single
consumer is updated.
[1] http://codesearch.openstack.org/?q=from%20neutron%20import%20worker
NeutronLibImpact
Change-Id: Id6bd414ae90119bf527bcaca6b88cc33ff79f685
Depends-On: Ifc0ecf6d5d3d4f8c406caad233d3648646724b12
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
Renames the AgentStatusCheckWorker class to PeriodicWorker
and moves it into the worker module since there isn't anything
agent-specific about it and it can be used for other periodic
jobs server side.
TrivialFix
Change-Id: Ic7a55ef534f64e6bfc60ae38bb0e139a0078510b
Once subscribers have been updated, this notification
no longer has reason to exist.
Change-Id: Ie617e0ecd568a1306221b15ddaef102e395f016a
Depends-on: I2b17ee41a9bfc020099c4f2f251afa32bb48b290
Depends-on: Ib225e4fba26265534cfcb2a75042bd193a78e15f
Depends-on: I4513fb64365fe5dcb81ab4ced3dc238e323aa332
Change 877778ee4c clarified the objective of the callback
registry events, and it introduced two new API related events.
This change introduces a new set of events that are meant
to be used for handling process-related operations, in fact
even though AFTER_CREATE was fit for the job, it looks somewhat
misused now.
Change-Id: I9e717e9e4ae4414067477712ac02870fe1041e70
This reverts commit b1cdba1696
Original patch was reverted because it broke neutron plugin's
backward compatibility and needed more work.
This patch fixes that problems:
1) original behaviour of add_agent_status_check,
start_periodic_l3_agent_status_check and
start_periodic_dhcp_agent_status_check methods is deprecated but kept
for using in third part plugins for backward compatibility
2) new add_agent_status_check_worker, add_periodic_l3_agent_status_check
and add_periodic_dhcp_agent_status_check method are implemented
instead and are used for implementing plugins in neutron codebase
Closes-Bug: #1569404
Change-Id: I3a32a95489831f0d862930384309eefdc881d8f6
I think this needs a little more thought. This broke OVN at least and
I don't think that's good. We need to figure out how to be compatible
with existing plugins, even out of tree ones.
This reverts commit 1cafff0871.
Change-Id: Ie087fb11213cc85911483c2d32c463fa9c973e54
Forking a process when multiple threads are running is an unsafe
operation and could cause a lot of problems because only current
thread will continue working in child thread. Any locked by other
thread resource will remain locked forever.
We faced with this problem during oslo.messaging development and
added workaround to hide this problem:
https://review.openstack.org/#/c/274255/
I tried to fix this problem in oslo.service:
https://review.openstack.org/#/c/270832/
but oslo folks said that this fix is ugly and it is wrong way to add
workarounds to common libraries because projects use them incorrectly.
I think that is fair.
So this patch fixes incorrect usage of oslo libraries. In this patch
I extended functionality of NeutronWorker and add there
`worker_process_count` parameter which determines how many processes
should be spawned for this worker. If `worker_process_count` = 0 - don't
create process and spawn thread in scope of current process for worker
Then I moved all background tasks to workers and return them by
`get_workers` method. start_plugin_workers collects plugin's workers
using `get_workers` method and starts in ProcessLauncher first workers
with `worker_process_count` > 0 and only after this starts threaded
workers by simple Launcher
Closes-bug: #1569404
Change-Id: I0544f1d47ae53d572adda872847a56fa0b202d2e
There are several cases where plugin initialization should be
handled after neutron-server forks API/RPC workers. For example,
starting a client connection to an SDN controller before forking
copies the fd of the socket to the child process, but then you have
multiple processes trying to read/write the same socket connection.
It is also useful for a plugin to be able to do something in only
one process, regardless of how many workers are forked. One example
would be handling syncing from an external system to the neutron
database.
This patch does 3 things:
1) Treats rpc_workers=0 as = 1. This simplifies the code for
handling notification that forking has completed. In the
existing code, calling the notification in the Worker object's
start() method would happen twice in the case where both api
and rpc workers were 0, despite there being only one process.
An earlier patch already changed the default api_workers to be
the number of processors.
2) Adds notification of forking via the callbacks mechanism.
Plugins can subscribe to resources.PROCESS, event.AFTER_CREATE
and do any post-fork initialization that needs to be done for
every spawned process.
3) Adds core/service plugin calls to get_workers() which defaults
to returning (). Plugins that need additional processes to spawn
should just return an iterable of NeutronWorkers that will be
spawned in their own process.
DocImpact
Closes-Bug: #1463129
Change-Id: Ib99954678c2b4f32f486b537979d446aafbea07b
Doug Wiegley