Update hacking to a more recent version, along with
flake8-import-order.
Remove N347 (import mock library) check as that is the
default with later hacking versions.
Update the builtins override of '_' to be the neutron.i18n
version due to the code triggering a false positive. This
is done in a couple of other projects as well.
Fix a number of new warnings it found.
Added some ignore directives for new whitespace issues
found in the test tree, can fix later.
TrivialFix
Change-Id: I5923255af86cf1fa11ab8e3b03bb9efac7dd7b58
The expresion "select 1;" can be replaced using the
``expression.select`` method.
Closes-Bug: #2031646
Change-Id: I7d82d5c399de5c3fb320f130a5c5f65abf147cbd
When neutron-server is down, ovs-agent waits for it to become available
during agent startup. When neutron-server is up, but it cannot reach the
DB, it can do nothing pretty much the same way. However ovs-agent
reacted differently to this failure. With this patch it reacts the same
way and delays its startup until neutron-server is up together with its
DB.
Change-Id: Ia55e82540aedc236e9b016bb58047d0b437eeb99
Closes-Bug: #2025341
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/db.
Trivialfix
Change-Id: I9311cfe5efc51552008072d84aa238e5d0c9de60
The goal of this patch is to make the Neutron code compliant
with SQLAlchemy 2.0.
All SQL transactions must be executed inside an explicit
writer/reader context. SQLAlchemy no longer will create an
implicit transaction if the session has no active transaction.
A warning message, only available in debug mode, is added. When
an ORM session calls "do_orm_execute", if there is no active
transaction, a warning message with a traceback will be logged
to help to debug the regression introduced.
Related-Bug: #1964575
Change-Id: I3da37fee205b8d67d10673075b9130147d9eab5f
When:
* the segments service plugin is enabled and
* we have multiple rpc worker processes (as in the sum of rpc_workers
and rpc_state_report_workers, since both kind processes agent
state_reports) and
* many ovs-agents report physnets,
then rabbitmq dispatches the state_report messages between the workers
in a round robin fashion, therefore eventually the state_reports of the
same agent will hit all rpc workers.
Unfortunately all worker processes have a 'reported_hosts' set to
remember from which host it has seen agent reports already. But right
after a server start when that set is still empty, each worker will
unconditionally write the received physnet-segment information into
the db. This means we multiply the load on the db and rpc workers by
a factor of the rpc worker count.
This patch tries to reduce the load on the db by adding another early
return before the unconditional db write.
Change-Id: I935186b6ee95f0cae8dc05869d9742c8fb3353c3
Closes-Bug: #1952730
Floating IP agent gateway ports are created for each external network
for each node where DVR L3 agent is running and where there is some FIP
from the ext_net.
But even, if L3 agent is removed (e.g. when scaling down the cluster),
such floating IP gateway port is never removed so it consumes IP address
from the external network.
With this patch when the DVR L3 agent is deleted, all such fip gateway
ports owned by that agent will be deleted.
When new L3 agent is created (registered in the DB), Neutron will check
if there are any floating IPs on that host and will recreate such FIP
gateway ports for it.
Closes-Bug: #1891360
Change-Id: If6ef990baf039c556d7420962ac4c54608711f06
The extension in neutron-lib was released in 1.17.0 so it is time to
use the extension from neutron-lib.
Change-Id: Ia414c4c6870897ac6f3dfdc8e33c355eba7f34ff
Related-Bug: #1733362
Since plugin agents are a global resource, relying just on the
'type' field for test_agent_show may end up finding an agent that
we don't know about, and that agent could be deleted by another
test. This reworks test_agent_show to sepecifically look for its
own OVN controller agent and test agent.
This also adds the 'id' field to the returned agent_status from
create_or_update_agent() to make it possible to look for the agent
that was just created.
Change-Id: Ib840e7c51f7b918b5e17ce9deff9ceafacf063cc
Closes-Bug: #1899004
Commit Ifc88dfb3041aa07b197f395172b69399796ba46a recently added a
new agent_timestamp parameter to create_or_update_agent.
As some other projects use this api, the change should have been
backwards compatible.
Adding default to None will solve this issue.
Closes-Bug: #1833589
Change-Id: I2d17f8eaf4d74ae2081f82e3c1a09bdc80b1d701
When an agent reports the state, the timestamp is sent along with the
agent status. This timestamp now is logged if "log_agent_heartbeats" is
activated.
Change-Id: Ifc88dfb3041aa07b197f395172b69399796ba46a
Related-Bug: #1799555
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
Agents supporting the guaranteed minimum bandwidth feature need to share
their resource view with neutron-server and in turn with Placement too.
The resource information is synchronized to neutron-server via the
periodic agent heartbeat therefore transient synchronization errors are
fixed by the next heartbeat. But synchronization to Placement is not
done periodically, but on a (mostly) on demand basis. Therefore to fix
transient errors of the synchronization to Placement we must remember
the success/failure of the last synchronization attempt.
This change implements the extension 'agent-resources-synced' and
therefore extends the agent db model and object with a new attribute:
'resources_synced'. This attribute in only meant to be updated
internally. But it can be read via the API for debugging purposes.
APIImpact: The agent resource has a new attribute: resources_synced.
Change-Id: I757d659cea63c8172ca3618d1f581d10236f5e71
Depends-On: https://review.openstack.org/626210
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
The functionality from model_query is already in neutron-lib and
consumers are using it. This patch removes the _model_query module
from neutron and updates all imports to use neutron-lib's version of
it instead.
NeutronLibImpact
Change-Id: Ib2eae9edb009a93e60b3b0d63ca365056138566b
This patch switches callbacks over to the payload object style events
for AGENT AFTER_CREATE and AFTER_UPDATE based notifications. To do
so a DBEventPayload object is used with the publish() method to
pass along the API related data.
Change-Id: Ibefa495be41c91957c2e8d797130e569bccc3765
The ovs agent will install some basic drop flows first for the
physical bridge mappings during the init procedure. If message
queue is not connected, or neutron-servers are all down, real
traffic flows will not be refreshed anymore. This will cause
the data plane down if tenant network and provider network are
sharing the physical NICs.
This patch adds a RPC check during init L2 agent. When restart
the ovs-agent, if the MQ is OK and we have available neutron-server,
go next step. Otherwise, a rpc timeout will be raised. L2 agent
will start fail, physical bridge mapping drop flows will not be
installed. The original flows will not be replaced, so the traffic
can still work properly.
Closes-Bug: #1803919
Change-Id: Ie15cf625b3710eaf290d6aafecb3f65df664b9df
This patch changed the old engine facade to new engine
fade in neutron/db/agents_db.py.
Partially-Implements blueprint: enginefacade-switch
Change-Id: Iee6fbee98fae41d75b4e8eb0fbcf2283da545023
Make the same status information available to notification consumers
(resource==agent, event==after_create/after_update) as it was already
available where the notification is sent in class AgentDbMixin.
Change-Id: Ie74091da934c7e49fd29ae4c6f930a7eb47e14b2
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
In some cases it may happen that port is "binding_failed"
because L2 agent running on destination host was down but
this is "temporary" issue.
It is like that for example in case when using L3 HA and when
master and backup network nodes were e.g. rebooted.
L3 agent might start running before L2 agent on host in such case
and if it's new master node, router ports will have "binding_failed"
state.
When agent sends heartbeat and is getting back to live,
ML2 plugin will try to bind all ports with "binding_failed"
from this host.
Change-Id: I3bedb7c22312884cc28aa78aa0f8fbe418f97090
Closes-Bug: #1794809
The retry_if_session_inactive decorator was rehomed into neutron-lib
[1]. This patch consumes it by removing the function from neutron and
using neutron-libs version where appropriate.
NeutronLibImpact
[1] https://review.openstack.org/#/c/557040/
Change-Id: I3e3289f33e62d45933d0fbf165bb4b25078f22d5
This patch switches BEFORE_DELETE callback events for AGENT resources
over to the payload style args use a DBEventPayload object. In addition,
the _delete_mac_associated_with_agent is now refactored into it's own
method to allow the context parameter to be parsed by
retry_if_session_inactive.
NeutronLibImpact
Change-Id: I0af951f4346f9290c2eba6aad17f772fbf27bb2f
The externally consumed APIs from neutron.db.api were rehomed into
neutron-lib with https://review.openstack.org/#/c/557040/
This patch consumes the retry_db_errors function from lib by:
- Removing retry_db_errors from neutron.db.api
- Updating the imports for retry_db_errors to use it from lib
- Using the DB API retry fixture from lib in the UTs where applicable
- Removing the UTs for neutron.db.api as they are now covered in lib
NeutronLibImpact
Change-Id: I1feb842d3e0e92c945efb01ece29856335a398fe
The APIs our consumers are using from neutron.db_utils were rehomed into
neutron-lib with https://review.openstack.org/#/c/540161/
This patch consumes them by removing the rehomed APIs and using lib's
implementation where applicable.
NeutronLibImpact
Change-Id: I7ee53bce917feae8e37bf278eb3121a5af47131c
In before, filtering of AZs is using the filter implementation of
agents. To filter the AZs, users need to find the AZ fields to filter,
locate their corresponding fields in agent, and compile the filters
based on the mapping between AZ and agent. This is undesirable.
This patch improve the filtering of AZ by converting the AZ filters
to agent filters. The supporting AZ filters are:
* name: the name of the availability zone
* state: the value is either 'available' or 'unavailable'
* resource: the value is either 'network' or 'router'
NOTE: For backward-compatibility, the old filters still works but
its usage is discourage.
APIImpact: Add filters to specify attributes of availability zones
Needed-By: Icbf427f20ca912a40d68e8042abeeabffeb3005f
Change-Id: Id882f949cc73a34290c311f3ce3d69d1b809c29f
Fixed all pep8 E265 errors and changed tox.ini to no longer
ignore them. Also removed an N536 comment missed from a
previous change.
Change-Id: Ie6db8406c3b884c95b2a54a7598ea83476b8dba1
The agent extension's API definition was moved into neutron-lib with
commit Iae4746aa2e3f68a7120696a12a6ae717115dbc31
This patch consumes it by removing the rehomed code and using the
APIExtensionDescriptor for the parent class of the extension.
NeutronLibImpact
Change-Id: I18aebec1e2b5b361d9b96192d4a59e5424c6a80b
Agent object has been merged [1].
This patch uses Agent object in agents_db and test_agents_db.
We also introduce a new function (get_agents_object) and keep
the old function (get_agents_db) for backward compatibility.
[1] https://review.openstack.org/#/c/297887/
Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Change-Id: I4c4283cb1aa05d52dca00cc249e094ea7d55b1d3
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Commit I1d4ded9959c05c65b04b118b1c31b8e6db652e67 rehomed the
availability zone extension's API definition into neutron-lib. This
patch consumes it, removing the rehomed logic in neutron and switching
over to lib's version of it.
NeutronLibImpact
Change-Id: I761381de0d6e26a0380386700e7921b824991669
The neutron-lib commit I360545b6ee4291547e0c5c8e668ad03d3efa4725 moved
the externally consumed globals from neutron.common.constants into lib.
With the exception of PROVISIONAL_IPV6_PD_PREFIX all other constants
in neutron.common.constants should only be used in neutron, and will
hopefully remain that way. External consumers needing access to other
common constants should move them into lib first.
NeutronLibImpact
Change-Id: Ie4bcffccf626a6e1de84af01f3487feb825f8b65
Changing rpc_api.rst file path from doc/source/devref/rpc_api.rst
to /doc/source/contributor/internals/rpc_api.rst. Because rpc_api.rst
file is located at this path
doc/source/contributor/internals/rpc_api.rst.
Closes-Bug #1722072
Change-Id: Ic243aab9e3428bfec69db61a94b4129cd768e233
Refactoring neutron db config opts to be in neutron/conf/db/ and
neutron/conf/agent/database/ so that all the configuration options
reside in a centralized location. This simplifies the process of
looking up the config opts and provides an easy way to import.
Change-Id: I4da9bb48d49b99e8c2b34a5c1b83e7eb95b70b82
Partial-Bug: #1563069
The usage of is_agent_down in neutron.db.agents_db.AgentDbMixin
is deprecated for removal in version ocata, and usage of the
is_agent_down neutron.agent.common.utils is recommended in future
release. And there is no usage of is_agent_down via
neutron.db.agents_db.AgentDbMixin in other projects [1]. It's safe
to remove it now.
[1]
http://codesearch.openstack.org/?q=is_agent_down&i=nope&files=&repos=
Change-Id: I7fbd2aa7294f284de66b131b59be9150e0538251
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
This logging statement was being called on every agent update
even if no versions changed, resulting in a bunch of useless
messages.
We already log when the actual versions change in the _set_version
call here: [1]
To cover the cases where versions are removed, this adds two logging
statements so we can be sure we see when the tracked versions do
change but not be bothered by version logs in any other condition.
1.
http://git.openstack.org/cgit/openstack/neutron/tree/neutron/api/rpc/callbacks/version_manager.py?h=d225b86738a9db56bc643dfe637b1f21393ad27e#n113
Partial-Bug: #1707307
Change-Id: I4761d21257dabfbd661bba1b24fe09d208d16ae6
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
Now that CommonDbMixin is just a shim we can stop using it.
Instead, use the model_query and resource_extend functions directly.
Related-Blueprint: neutron-lib
Change-Id: If1252c42c49cd59dba7ec7c02c9b887fdc169f51
according to https://wiki.openstack.org/wiki/Python3, now we should avoid
using six.iteritems and replace it with dict.items.
Change-Id: I58a399baa2275f280acc0e6d649f81838648ce5c
Closes-Bug: #1680761
Those were moved under neutron.db.models start of Ocata. We should be
able to clean them up now.
NeutronLibImpact
Change-Id: Iadbf44d52ee8e30712807384152a29ce1a8b8f72
Brian Haley