13 KiB
API Microversioning
Background
The API Microversioning is a framework in Senlin to enable
smooth evolution of the Senlin REST API while preserving its backward
compatibility. The basic idea is that a user has to explicitly specify
the particular version of API requested in the request. Disruptive
changes to the API can then be added without breaking existing users who
don't specifically ask for it. This is done with an HTTP header
OpenStack-API-Version
as suggested by the OpenStack API
Working Group. The value of the header should contain the service name
(clustering
) and the desired API version which is a
monotonically increasing semantic version number starting from
1.0
.
If a user makes a request without specifying a version, they will get
the DEFAULT_API_VERSION
as defined in
senlin.api.common.wsgi
. This value is currently
1.0
and is expected to remain so for quite a long time.
There is a special value "latest
" which can be
specified, which will allow a client to always invoke the most recent
version of APIs from the server.
Warning
The latest
value is mostly meant for integration testing
and would be dangerous to rely on in client code since Senlin
microversions are not following semver and therefore backward
compatibility is not guaranteed. Clients, like python-senlinclient or
openstacksdk, python-openstackclient should always require a specific
microversion but limit what is acceptable to the version range that it
understands at the time.
When to Bump the Microversion
A microversion is needed when the contract to the user is changed. The user contract covers many kinds of information such as:
- the Request
the list of resource URLs which exist on the server
Example: adding a new
GET clusters/{ID}/foo
resource which didn't exist in a previous version of the codethe list of query parameters that are valid on URLs
Example: adding a new parameter
is_healthy
when querying a node byGET nodes/{ID}?is_healthy=True
the list of query parameter values for non-freeform fields
Example: parameter
filters
takes a small set of properties "A
", "B
", "C
", now support for new property "D
" is addednew headers accepted on a request
the list of attributes and data structures accepted.
Example: adding a new attribute
'locked': True/False
to a request body
- the Response
the list of attributes and data structures returned
Example: adding a new attribute
'locked': True/False
to the output ofGET clusters/{ID}
the allowed values of non-freeform fields
Example: adding a new allowed "
status
" field toGET servers/{ID}
the list of status codes allowed for a particular request
Example: an API previously could return 200, 400, 403, 404 and the change would make the API now also be allowed to return 409.
changing a status code on a particular response
Example: changing the return code of an API from 501 to 400.
Note
According to the OpenStack API Working Group, a 500 Internal Server Error should NOT be returned to the user for failures due to user error that can be fixed by changing the request on the client side. This kind of a fix doesn't require a change to the microversion.
new headers returned on a response
The following flow chart attempts to walk through the process of "do we need a microversion".
digraph states {
label="Do I need a microversion?"
silent_fail[shape="diamond", style="", group=g1, label="Did we silently
- fail to do what is asked?"];
-
ret_500[shape="diamond", style="", group=g1, label="Did we return a 500
- before?"];
-
new_error[shape="diamond", style="", group=g1, label="Are we changing the status code returned?"]; new_attr[shape="diamond", style="", group=g1, label="Did we add or remove an attribute to a resource?"]; new_param[shape="diamond", style="", group=g1, label="Did we add or remove an accepted query string parameter or value?"]; new_resource[shape="diamond", style="", group=g1, label="Did we add or remove a resource url?"];
no[shape="box", style=rounded, label="No microversion needed"]; yes[shape="box", style=rounded, label="Yes, you need a microversion"]; no2[shape="box", style=rounded, label="No microversion needed, it's a bug"];
silent_fail -> ret_500[label=" no"]; silent_fail -> no2[label="yes"];
ret_500 -> no2[label="yes [1]"]; ret_500 -> new_error[label=" no"];
new_error -> new_attr[label=" no"]; new_error -> yes[label="yes"];
new_attr -> new_param[label=" no"]; new_attr -> yes[label="yes"];
new_param -> new_resource[label=" no"]; new_param -> yes[label="yes"];
new_resource -> no[label=" no"]; new_resource -> yes[label="yes"];
{rank=same; yes new_attr} {rank=same; no2 ret_500} {rank=min; silent_fail} }
Note
The reason behind such a strict contract is that we want application developers to be sure what the contract is at every microversion in Senlin.
When in doubt, consider application authors. If it would work with no client side changes on both Nova versions, you probably don't need a microversion. If, however, there is any ambiguity, a microversion is likely needed.
When a Microversion Is Not Needed
A microversion is not needed in the following situations:
- the response
- Changing the error message without changing the response code does not require a new microversion.
- Removing an inapplicable HTTP header, for example, suppose the Retry-After HTTP header is being returned with a 4xx code. This header should only be returned with a 503 or 3xx response, so it may be removed without bumping the microversion.
Working with Microversions
In the senlin.api.common.wsgi
module, we define an
@api_version
decorator which is intended to be used on
top-level methods of controllers. It is not appropriate for lower-level
methods.
Adding a New API Method
In the controller class:
@wsgi.Controller.api_version("2.4")
def my_api_method(self, req, id):
....
This method is only available if the caller had specified a request
header OpenStack-API-Version
with value
clustering <ver>
and <ver>
is
>= 2.4
. If they had specified a lower version (or
omitted it thus got the default of 1.0
) the server would
respond with HTTP 404.
Removing an API Method
In the controller class:
@wsgi.Controller.api_version("2.1", "2.4")
def my_api_method(self, req, id):
....
This method would only be available if the caller had specified an
OpenStack-API-Version
with value
clustering <ver>
and the <ver>
is
<= 2.4
. If 2.5
or later is specified the
server will respond with HTTP 404.
Changing a Method's Behavior
In the controller class:
@wsgi.Controller.api_version("1.0", "2.3")
def my_api_method(self, req, id):
.... method_1 ...
@wsgi.Controller.api_version("2.4") # noqa
def my_api_method(self, req, id):
.... method_2 ...
If a caller specified 2.1
, 2.2
or
2.3
(or received the default of 1.0
) they
would see the result from method_1
, 2.4
or
later method_2
.
It is vital that the two methods have the same name, so the second
one will need # noqa
to avoid failing flake8's
F811
rule. The two methods may be different in any kind of
semantics (schema validation, return values, response codes, etc.)
When Not Using Decorators
When you don't want to use the @api_version
decorator on
a method or you want to change behavior within a method (say it leads to
simpler or simply a lot less code) you can directly test for the
requested version with a method as long as you have access to the API
request object. Every API method has an version_request
object attached to the Request
object and that can be used
to modify behavior based on its value:
import senlin.api.common.version_request as vr
def index(self, req):
# common code ...
= req.version_request
req_version = vr.APIVersionRequest("2.1")
req1_min = vr.APIVersionRequest("2.5")
req1_max = vr.APIVersionRequest("2.6")
req2_min = vr.APIVersionRequest("2.10")
req2_max
if req_version.matches(req1_min, req1_max):
# stuff...
elif req_version.matches(req2min, req2_max):
# other stuff...
elif req_version > vr.APIVersionRequest("2.10"):
# more stuff...
# common code ...
The first argument to the matches method is the minimum acceptable version and the second is maximum acceptable version. A specified version can be null:
= APIVersionRequest() null_version
If the minimum version specified is null then there is no restriction on the minimum version, and likewise if the maximum version is null there is no restriction the maximum version. Alternatively an one sided comparison can be used as in the example above.
Planning and Committing Changes
Once the idea of an API change is discussed with the core team and the consensus has been reached to bump the micro-version of Senlin API, you can start working on the changes in the following order:
- Prepare the engine and possibly the action layer for the change. One STRICT requirement is that the newly proposed change(s) should not break any existing users.
- Add a new versioned object if a new API is introduced; or modify the
fields of an existing object representing the API request. You are
expected to override the
obj_make_compatible()
method to ensure the request formed will work on an older version of engine. - If the change is about modifying an existing API, you will need to
bump the version of the request object. You are also required to add or
change the
VERSION_MAP
dictionary of the request object class where the key is the API microversion and the value is the object version. For example:
@base.SenlinObjectRegistry.register
class ClusterDanceRequest(base.SenlinObject):
# VERSION 1.0: Initial version
# VERSION 1.1: Add field 'style'
= '1.1'
VERSION = {
VERSION_MAP 'x.y': '1.1'
}
= {
fields
...'style': fields.StringField(nullable=True),
}
def obj_make_compatible(self, primitive, target_version):
# add the logic to convert the request for a target version
...
- Patch the API layer to introduce the change. This involves changing
the
senlin/api/openstack/history.rst
file to include the descriptive information about the changes made. - Revise the API reference documentation so that the changes are properly documented.
- Add a release note entry for the API change.
- Add tempest based API test and functional tests.
- Update
_MAX_API_VERSION
insenlin.api.openstack.versions
, if needed. Note that each time we bump the API microversion, we may introduce two or more changes rather than one single change, the update of_MAX_API_VERSION
needs to be done only once if this is the case. - Commit patches to the
openstacksdk
project so that new API changes are accessible from client side. - Wait for the new release of
openstacksdk
project that includes the new changes and then propose changes topython-senlinclient
project.
Allocating a microversion
If you are adding a patch which adds a new microversion, it is
necessary to allocate the next microversion number. Except under
extremely unusual circumstances, the minor number of
_MAX_API_VERSION
will be incremented. This will also be the
new microversion number for the API change.
It is possible that multiple microversion patches would be proposed
in parallel and the microversions would conflict between patches. This
will cause a merge conflict. We don't reserve a microversion for each
patch in advance as we don't know the final merge order. Developers may
need over time to rebase their patch calculating a new version number as
above based on the updated value of _MAX_API_VERSION
.