Requests handled by the metadata-agents can now be rate-limited by
source-ip. This is done to protect the OpenStack control plane against
VMs querying the metadata endpoint in an overly enthusiastic way.
Co-authored-by: Miguel Lavalle <mlavalle@redhat.com>
Related-Bug: #1989199
Change-Id: I748ccfa8b50496dcbcbe41fd22f84249a4d46b11
Importing some modules lead to registering config options that may
collide with config options from a project that calls the import. This
patch wraps the side effect that registers config options into a
function that needs to be called in case the caller wants to register
the options.
This solution is also not perfect as it guards the common options to be
registered only once even if the function is called multiple times. This
is to solve problems in unittests, ideally we should always call the
function just once even in our testing suites.
Resolves-Bug: #1968606
Change-Id: Ic1532eb8de887ff1b1085206df11f53e22f7f524
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
Now that we use setproctitle for neutron-server workers (and
neutron-keepalived-state-change), this has the side effect of changing
the process name for agents, impacting some monitoring systems. More
details in launchpad bug.
This patch fixes it by setting the name with setproctitle to:
agent name (original process name).
Also use the newly introduced name constants to replace existing
hardcoded uses.
Change-Id: I74c3a4d3e9f833752571a75f196560cd45529385
Closes-Bug: #1881297
Remove this duplicated option and rely only in OVS.integration_bridge.
NOTE: other projects are still using it; first we need to deprecate it
in those projects.
Change-Id: I4e826c8b9fa764b1820adacc3427934dc393c0bc
Related-Bug: #1856152
New releases of oslo.config support a 'mutable' parameter to Opts.
This is only respected when the new method mutate_config_files is
called instead of reload_config_files. Neutron delegates making this
call to oslo.service. This was provided in patchset
Icec3e664f3fe72614e373b2938e8dee53cf8bc5e
Further patches will be needed to make select config options be
marked as mutable. This change enables support for oslo provided
config options to be updated via SIGHUP such as log level.
Task: 6389
Story: 2001545
Change-Id: I9442965607f3248706464643c6d87a04edcae24e
The neutron.common.topics module was rehomed into neutron-lib with
commit Ie88b84949cbd55a4e7ad06341aab77b286cdc485
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.
NeutronLibImpact
Change-Id: Ia4a4604c259ce862597de80c6deeb3d408bf0e95
Refactoring neutron agent linux and ovsdb config opts
to be in neutron/conf/agent so that all the config options
reside in a centralized location. This simplifies the
process of looking up the config opts and provides an easy
way to import.
NeutronLibImpact
Change-Id: Ib1e0e63dec2985c417412d1ecc68e2a74ef87182
Partial-Bug: #1563069
Refactoring Neutron configuration options for agent common config to be
in neutron/conf/agent/common. This will allow centralization of all
configuration options and provide an easy way to import.
Partial-Bug: #1563069
Change-Id: Iebac0cdd3bcfd0135349128921b7ad7a1a939ab8
Needed-By: Ib676003bbe909b5a9013a3178b12dbe291d936af
Due to the high memory footprint of current Python ns-metadata-proxy,
it has to be replaced with a lighter process to avoid OOM conditions in
large environments.
This patch spawns haproxy through a process monitor using a pidfile.
This allows tracking the process and respawn it if necessary as it was
done before. Also, it implements an upgrade path which consists of
detecting any running Python instance of ns-metadata-proxy and
replacing them by haproxy. Therefore, upgrades will take place by
simply restarting neutron-l3-agent and neutron-dhcp-agent.
According to /proc/<pid>/smaps, memory footprint goes down from ~50MB
to ~1.5MB.
Also, haproxy is added to bindep in order to ensure that it's installed.
UpgradeImpact
Depends-On: I36a5531cacc21c0d4bb7f20d4bec6da65d04c262
Depends-On: Ia37368a7ff38ea48c683a7bad76f87697e194b04
Closes-Bug: #1524916
Change-Id: I5a75cc582dca48defafb440207d10e2f7b4f218b
Refactoring neutron agent metadata config opts to be in
neutron/conf/agent/metadata so that all the configurations options
reside in a centralized location. This simplifies the process of looking
up the config opts and provides an easy way to import.
Change-Id: I8bae1facc58a4f9e21196f625478532403651545
Partial-Bug: #1563069
Refactoring l3 ha agent options to be in neutron/conf/agent/l3.
This would allow centralization of all configuration options and
provides an easy way to import.
Partial-Bug: #1563069
Change-Id: I2d6bd6beb0d1658baf88c49b954d2db3136e0c8d
Refactoring neutron configuration options for l3 agent to be in
neutron/conf/agent/l3. This would allow centralization of all
configuration options in neutron/conf and provide an easy way to import.
Change-Id: Ie7533ea55eaa4d0f2c1919131a75f56e027c4d6e
Partial-Bug: #1563069
In order for the l3-agent to see the RA and PD config options,
it needs to register them when it starts. Noticed this when I
went to override something for a test and it wouldn't work.
It now passes the config down to radvd on start so the correct
values are picked-up.
Change-Id: Iec0e0d16eed4f12af77fcd4f0b93b641b1146293
Related-Bug: #1532338
The use_namespaces option has been defined as a workaround to kernels
not properly supporting namespaces. This limitation is behind us, it's
time to remove use_namespaces after its deprecation in Kilo in order to
simplify code and remove a poorly tested case (use_namespaces=False).
This change prepares for removal pullup_route method[1] which was only
used when use_namespaces=False.
[1] neutron.agent.linux.ip_lib
DocImpact
UpgradeImpact
Closes-Bug: #1508188
Related-Bug: #1435382
Depends-On: I303038eec560a6d99421140c2822aed8b518470b
Depends-On: I4feb2a15c7e1e4bfdbed2531b18b8e7d798ab3cc
Change-Id: I2fbf65df1250d9f9f1656b3964ee3b6de1ef1118
This patch adds the availability_zone attribute to agents and
supports availability_zone API.
Availability_zone support for resources (network/router) and
the schedulers are included in subsequent patches.
APIImpact
DocImpact
Co-Authored-By: IWAMOTO Toshihiro <iwamoto@valinux.co.jp>
Change-Id: Id7a62000ab0484412b3970199df8c374568fe70d
Partially-implements: blueprint add-availability-zone
oslo.service has graduated, so neutron should consume it.
Closes-Bug: #1466851
Depends-On: Ie0fd63f969f954029c3c3cf31337fbe38f59331a
Depends-On: I2093b37d411df9a26958fa50ff523c258bbe06ec
Depends-On: I4823d344878fc97e66ddd8fdae25c13a34dede40
Change-Id: I0155b3d8b72f6d031bf6f855488f80acebfc25d4
Currently metadata proxy cannot run with nobody user/group as metadata
proxy requires to connect to metadata_proxy_socket when queried.
This change allows to run metadata proxy with nobody user/group by
allowing to choose the metadata_proxy_socket mode with the new option
metadata_proxy_socket_mode (4 choices) in order to adapt socket
permissions to metadata proxy user/group.
This change refactors also where options are defined to enable
metadata_proxy_user/group options in the metadata agent.
In practice:
* if metadata_proxy_user is agent effective user or root, then:
* metadata proxy is allowed to use rootwrap (unsecure)
* set metadata_proxy_socket_mode = user (0o644)
* else if metadata_proxy_group is agent effective group, then:
* metadata proxy is not allowed to use rootwrap (secure)
* set metadata_proxy_socket_mode = group (0o664)
* set metadata_proxy_log_watch = false
* else:
* metadata proxy has lowest permissions (securest) but metadata proxy
socket can be opened by everyone
* set metadata_proxy_socket_mode = all (0o666)
* set metadata_proxy_log_watch = false
An alternative is to set metadata_proxy_socket_mode = deduce, in such
case metadata agent uses previous rules to choose the correct mode.
DocImpact
Closes-Bug: #1427228
Change-Id: I235a0cc4f0cbd55ae4ec1570daf2ebbb6a72441d
This should remove the last bits of passing root_helper around
for execute(). Some tests store and pass the root_helper argument
outside of this, and have been left alone.
Depends-On: I1e3b64e5a1d6cff2aebc638710487bbdbdba61d4
Depends-On: I5d51fa5edc7f427d84d7ae49d3a4c972dca8ac37
Depends-On: I21c6e7795c8b2e8f87bbd2b00a8eb7cb215ece74
Depends-On: I6566e17a1ce07a3bebf2a1a3653ad7e7f397f0b9
Change-Id: I38c901c65fb0c8598589ec56d5afd50d2f882f4f
Partially-Implements: blueprint rootwrap-daemon-mode
Oslo project decided to move away from using oslo.* namespace for all their
libraries [1], so we should migrate to new import path.
This patch applies new paths for:
- oslo.config
- oslo.db
- oslo.i18n
- oslo.messaging
- oslo.middleware
- oslo.rootwrap
- oslo.serialization
- oslo.utils
Added hacking check to enforce new import paths for all oslo libraries.
Updated setup.cfg entry points.
We'll cleanup old imports from oslo-incubator modules on demand or
if/when oslo officially deprecates old namespace in one of the next
cycles.
[1]: https://blueprints.launchpad.net/oslo-incubator/+spec/drop-namespace-packages
Depends-On: https://review.openstack.org/#/c/147248/
Depends-On: https://review.openstack.org/#/c/152292/
Depends-On: https://review.openstack.org/#/c/147240/
Closes-Bug: #1409733
Change-Id: If0dce29a0980206ace9866112be529436194d47e
This change moves metadata driver related config options to metadata
driver module to prepare the use of metadata driver method in the dhcp
agent (daughter change). The metadata_port option is not moved as the
dhcp agent uses a non-configurable port (80).
Change-Id: Ie45fdad86f33d35fca3096c4c52fae941a279e76
Partial-Bug: #1187107
Commit 6ca8cb84fd moved the patching early in the loading process.
After some reshuffling in the agents, this might not be as early
as possible, so move them closer to their main()'s.
Change-Id: Ifddd3a3d87ef6cc06788e9b4c40d6a6d981ebe0c
This is done in order to show the split between a main() and
the agent's inner workings. At the same time the L3 agent for
Neutron is also being restructured and to this aim, its config
is also moved out of the agent file.
Keep main() for the time being until we get dependent
services updated.
Partially-Implements: bp restructure-l3-agent
Partially-Implements: bp core-vendor-decomposition
Change-Id: I0d07d91ba301ee1aa51dabcf964a96edc0d6a3e0
The file l3_agent.py has become too large. This patch is a simple
pure refactor to move some of the functionality in to other files
where things aren't too tangled up. There is no functional change
with this patch and I avoided gratuitous other fixups in this patch in
order to make it easier to review.
I plan to follow up on the new l3_dvr and l3_agent_router modules with
more restructuring in the near future.
Partially-Implements: bp restructure-l3-agent
Change-Id: I3529fe4146c50c940f41eb26d0b5efc5870b3af9
- After l3_agent is refactored, need to remove services/firewall
- After vmware plugin moves services out of monolothic,
remove model copies and services/loadbalancer/constants,
and re-enable unit tests.
- After alembic chain gets split in four, tweak models/head and
fix heal/current chain.
- Re-factor test_routerserviceinsertion into one of the service repos
Partially-Implements: blueprint services-split
Change-Id: I5466984a9e57128266f97e9bd5c265f4dc3cba7b
Convert several places in the code that were using the RpcCallback
compatibility class. This class eased the transition to
oslo.messaging. This patch drops the usage of this class in favor of
direct usage of the appropriate oslo.messaging API.
Part of blueprint drop-rpc-compat.
Change-Id: I955958e836635d43dabc7b39d050c1418b18cf8f
Mostly trivial import changes.
- oslo.i18n no longer provide install() method to inject _() into
globals(), so removed all calls to it;
- removed Babel from dependencies (it will now be grabbed by oslo.i18n);
- updated tox.ini to ignore import violations for oslo.i18n.
Change-Id: I6623d551f512fb7fe9bf35ee734ed6d4c6cbc287
Before this fix an exception was raised when
creating the veth between the fip namespace and
the qrouter namespace when the veth was already
present.
This fix add a check to only create the veth if
not present.
Change-Id: Iefea9778223aac885bedfdacfdeacfce74776333
Closes-Bug: #1376013
HA failover testing requires the ability for two L3 agents to
co-exist on the same machine. The following changes were made
to support this:
* Add l3 agent ability to simulate nested namespaces by suffixing
each router namespace with '@host'. For example, if a router
with id '1' is created, its namespace will be:
'qrouter-1@host'
* Added test.common submodule which will hold common code which is
shared between the different kind of tests (unit and funtional).
* Added tests.common.agents submodule which holds test-only agents, and
added a test L3NAT agent to be used in the HA functional testings.
Co-Authored-By: Assaf Muller <amuller@redhat.com>
Closes-bug: #1374946
Change-Id: I2185fe276d9d32b34c407396e7ed5d6e2801340c
The try is too broad. This commit moves code outside of the try block
where exceptions are not expected to be raised. This only leaves the
rpc call in the current code. In legacy code, there used to be more
but now this code merely queues updates for workers to handle
in other threads.
Change-Id: I15407553fc6ad6eeb5b483499fe1a5adffe1edca
These indirections serve no useful purpose. Normally, I wouldn't
bother to change them but these make other refactoring efforts a bit
more of a pain.
Change-Id: Ia73c93eb2eaa81d5772b3e0178fdf18f0c275b4c
On a compute node when the last VM with a floating IP association
is deleted, the L3 agent did not delete the fip namespace. However
the api server has already deleted the fip agent external gateway
port from the database.
This problem is happening on DVRs because the deletion of a VM port,
in addition to a floating IP disassociation, may also result in the
removal of the external gateway port binding AND the removal of the
fip agent external gateway port.
When the L3 agent is handling a routers_updated notification, it is
not processing floating ip address updates when the router has both
a floating ip disassociated and a external gateway port deleted.
This patch corrects this problem.
Closes-bug: #1377156
Change-Id: I86bdef7c9d988cb9d87c88adde55548d459f29a5
Make outgoing packets to an external interface SNATed regardless
of source address of the packets. As a result of deep review,
any problem was not found with this change.
Change-Id: I71a1288633bb6af2951d571540bbb9ec5e1270e2
Closes-bug: #1386041
Drop the usage of the RpcProxy compatibility class from the
L3PluginApi. The equivalent direct usage of the oslo.messaging APIs
are now being used instead.
Part of blueprint drop-rpc-compat.
Change-Id: I6639d1aa8acca8c0544020e28489e71f3d5d2955
Don't translate debug level logs and enforce log hints
Our translation policy
(https://wiki.openstack.org/wiki/LoggingStandards#Log_Translation) calls
for not translating debug level logs. This is to help prioritize log
translation. Furthermore translation has a performance overhead, even if
the log isn't used (since neutron doesn't support lazy translation yet).
NOTE: this is done on a directory by directory basis to ensure that we
do not have too many conflicts and rebases.
Add a local hacking rule to enforce this.
This patch set enforces the directory neutron/agents
Partial-bug: #1320867
Change-Id: I4bd562e5138c2d2850072440aa121f27e902463a
For DVR, FIPs should be hosted on the single node
which hosts the VM assigned with the fixed_ip of the FIP.
The l3_agent should only take action on the correct FIP per
host by filtering the FIPs based on the 'host' value
of the FIP.
A recent refactor on the l3_agent moved the host filtering logic
from process_router_floating_ip_addresses() to
_get_external_device_interface_name(). The local floating_ips var
was not altered as it was before the refactor.
This resulted in network disruption across multiple hosts
since more than one namespace contained the FIP. This problem
would only be seen in a mutli-host environment where the same
router hosting FIPs was present on more than one node.
The fix is to return the host filtering logic by adding a
call to get_floating_ips(). In addition, the unit test
test_process_router_dist_floating_ip_add() was modified to
pass two FIPs instead of one. One FIP matches the host
of the agent, one does not. Only one should be processed,
not two.
Change-Id: I67b19f6228af392519fff89b13283b43921552bf
Closes-bug: #1389880
Remove RemoteError from neutron.common.rpc, which was just mapped
directly to oslo.messaging.RemoteError.
Part of blueprint drop-rpc-compat.
Change-Id: Ic4d7ea0adfd68eb56100e32ce0d8e59aa2aae5b1
Remove RPCException, which was just mapped directly to
oslo.messaging.MessagingException for the purposes of minimizing the
impact to the code base when moving from openstack.common.rpc to
oslo.messaging.
Part of blueprint drop-rpc-compat.
Change-Id: Idc5af503f24217fbd2d982c338434c200d025521
The use_namespaces property is a property of the agent, not the
router. The router doesn't need to know if namespaces are in use by
the agent.
Change-Id: I50a1069fc654eeaafaa4098d926ae9e96e5d400f
Guillaume Espanel