Policy engine API

Change-Id: I271d54758c0b03bba422e7c7d7226534803ec0b9
This commit is contained in:
Tim Hinrichs 2015-05-17 16:04:00 -07:00
parent 8eca2f350f
commit 8e5e721f77
1 changed files with 325 additions and 0 deletions

View File

@ -0,0 +1,325 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Add policy engine class to API
==========================================
This blueprint makes it possible to access policy-engines (other than
the domain-agnostic policy engine) via the API.
Problem description
===================
Currently there is no way to access policy engines other than the current
domain-agnostic policy engine through the API. Now that we are adding
other policy engines, we need to give the user a way to interact with them.
In addition to being useful for domain-specific policy engines, enabling
multiple policy engines should make it easier to handle upgrades for the
domain-agnostic policy engine: bring up the new one alongside the old one,
then swap the old one with the new one atomically.
Proposed change
===============
Here we discuss two proposals. We list tradeoffs in the next section.
Proposal 1: Type-based
--------------------------
Here we simply add a top-level 'policy-engines' API endpoint, giving us ...
/v1/policy-engines/<engine-id>/...
/v1/data-sources/<datasource-id>/...
/v1/system/...
For example:
/data-sources/nova/...
/data-sources/neutron/...
/policy-engines/agnostic/...
Data-sources would support:
* schema: available tables
* actions: available actions
* status: status
Policy-engines would support:
* schema: available tables (e.g. classification:connected_to_internet)
* actions: available actions (e.g. scripts built into a policy-engine for
carrying out some task)
* status: status
* policies: available policies
That is, the only difference between a policy-engine and a data-source is that
the datasource doesn't support 'policies'. If we were to think of 'policies'
as simply 'modules' then we could imagine a datasource-driver exposing
hierarchical tables, just like a policy-engine does. And in that case,
the data-source would have something analogous to the policy-engine 'policies'.
Proposal 2: Service-based
--------------------------
Here we do away with the types policy-engine and data-sources and give clients
the ability to access both policy-engines and datasources from the same
endpoint. This is possible because you cannot give a policy-engine and
a datasource the same name (a restriction in place to ensure references to
other services in policy are unambiguous).
/v1/services/<service-id>/...
/v1/system/...
For example:
/v1/services/nova/...
/v1/services/neutron/...
/v1/services/agnostic/...
All services would support:
* schema: available tables (e.g. classification:connected_to_internet)
* actions: available actions (e.g. scripts built into a policy-engine for
carrying out some task)
* status: status
* policies: available policies
In short, this proposal treats everything as if it is a policy-engine. The
datasource policy engines prohibit users from making changes directly (and
even if they did, they would only accept a very restricted form of policy
statements: ground facts).
This approach can be backwards compatible as well. We can still support
/v1/datasources/<datasource-id>/...
which gets routed to
/v1/services/<datasource-id>.
And we can support
/v1/policies
/v1/actions
/v1/tables
/v1/status
as syntactic sugar for
/v1/services/<default-service>/policies
/v1/services/<default-service>/actions
/v1/services/<default-service>/tables
/v1/services/<default-service>/status
This would enable the user to choose a single touchpoint for managing policy
in the datacenter, while at the same time enabling them direct access to all
the services in the datacenter. (The only worry here is that )
To set the <default-service>, we'd want a parameter the user can set
dynamically (to help with upgrade).
Right now we would set that to /services/agnostic.
And maybe we can have arbitrary aliases for services as well, so that we can
upgrade any service without changing policy.
One worry with providing the /v1/policies, etc. endpoints is that it may
seem to mask Congress's overall status, policies, actions, and tables. That
is, people might expect those endpoints to aggregate all the potential policies,
actions, tables, and statuses. But if such functionality ever becomes
necessary, we can attach those endpoints to /v1/services, giving us the
following end points.
/services/policies
/services/actions
/services/tables
/services/status
Here is an example of the entry points if we had Nova, GBP, and our policy engine. The name of the service is whatever name DSE expects.
/policies
/actions
/tables
/services/policies
/services/actions
/services/tables
/services/nova/policies -> empty
/services/nova/actions -> createVM, deleteVM, migrateVM, etc.
/services/nova/tables -> servers, hosts, etc.
/services/gbp/policies
/services/gbp/actions
/services/gbp/tables
/services/engine/policies
/services/engine/actions
/services/engine/tables
/system/drivers/
/system/engine-drivers/nova-uber-driver
/system/datasource-drivers/nova-uber-driver
/system/action-drivers/nova-uber-driver
/users
/stats
One benefit to enabling people to modify domain-specific policy engines
through the Congress API is that we provide a single policy language for
managing all the policy engines running in the datacenter. For delegation,
we already need adapters that translate Datalog into the native language of
each policy engine, so here we expose that functionality directly to the user
as well.
Tradeoffs
------------
Pros for the type-based approach:
* Easy for users to understand
* Simple extension of the current API
Cons for the type-based approach:
* Awkward that data-sources and policy-engines implement almost exactly the
same interface and have separate namespaces, but are represented as
distinct classes in the API.
* Enables us to build datasources with significantly different programmatic
interface than policy-engines. If at the API-layer the two classes of
objects were almost indistinguishable, it would lead to better abstraction
and interfaces in the underlying implementation.
Pros for the the service-based approach:
* All services running on the DSE are accessed identically from the API. This
is a more natural reflection of the reality of the nature of those services.
Cons for the service-based approach:
* Bigger change
* May be more difficult for users to understand initially.
* Eventually the policy-engine class will include functionality that the
datasource class does not. Executing that functionality on a datasource
will cause a 404, and we cannot predict which will occur based on just the
URL.
* Doing something like listing all the datasources will require an API like
/v1/services?action=list&type=datasource instead of the more obvious
/v1/data-sources/.
Overall, the types (datasource vs. policyengine) will be present in both
proposals, but they will be emphasized much less in the service-based approach.
The service-based approach is closer to Python in that the system isn't able
to look at the code you've written (the URL) and check if the method you asked
for exists. The type-based approach is closer to C/Java in that the system IS
able to tell you if the method exists by just looking at the code (URL).
Typically policy systems are quite dynamic in nature (you can change the
policy/code at runtime), and hence are closer to dynamic programming languages
like Python than to static languages like C/Java. We therefore typically
bias our decisions toward dynamism, which in this case would mean the
service-based approach.
Policy
------
N/A
Policy actions
--------------
N/A
Data sources
------------
N/A
Data model impact
-----------------
N/A
REST API impact
---------------
See above. No changes to API results--just the paths for invoking them.
Security impact
---------------
N/A
Notifications impact
--------------------
N/A
Other end user impact
---------------------
N/A
Performance impact
------------------
N/A
Other deployer impact
---------------------
See above.
Developer impact
----------------
N/A
Implementation
==============
Assignee(s)
-----------
Primary assignee:
<launchpad-id or None>
Other contributors:
<launchpad-id or None>
Work items
----------
Once we decide on the approach, we will figure out the necessary work items.
But here's a rough cut.
- type-based approach: add routes, create congress/api/engine_model.py, modify
congress/api/*_model to enable tables/actions/policies/etc. for engines.
- service-based approach: add routes, create congress/api/service_model.py,
(including an API to list different types of objects), modify the
congress/api/*_model to eliminate distinction between datasources and policies
Dependencies
============
Assumes that we have added an API call for 'actions', though this work could
be done without that: add-action-listing.
Testing
=======
- Change unit tests in congress/tests/test_congress.py
- Change congress_pythonclient (which will handle tempest tests)
Documentation impact
====================
Many URLs may change.
References
==========
N/A