diff --git a/specs/liberty/policy-engine-api.rst b/specs/liberty/policy-engine-api.rst new file mode 100644 index 0000000..712d32c --- /dev/null +++ b/specs/liberty/policy-engine-api.rst @@ -0,0 +1,325 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +========================================== +Add policy engine class to API +========================================== + +This blueprint makes it possible to access policy-engines (other than +the domain-agnostic policy engine) via the API. + + +Problem description +=================== + +Currently there is no way to access policy engines other than the current +domain-agnostic policy engine through the API. Now that we are adding +other policy engines, we need to give the user a way to interact with them. + +In addition to being useful for domain-specific policy engines, enabling +multiple policy engines should make it easier to handle upgrades for the +domain-agnostic policy engine: bring up the new one alongside the old one, +then swap the old one with the new one atomically. + + +Proposed change +=============== + +Here we discuss two proposals. We list tradeoffs in the next section. + +Proposal 1: Type-based +-------------------------- +Here we simply add a top-level 'policy-engines' API endpoint, giving us ... + +/v1/policy-engines//... +/v1/data-sources//... +/v1/system/... + +For example: +/data-sources/nova/... +/data-sources/neutron/... +/policy-engines/agnostic/... + +Data-sources would support: +* schema: available tables +* actions: available actions +* status: status + +Policy-engines would support: +* schema: available tables (e.g. classification:connected_to_internet) +* actions: available actions (e.g. scripts built into a policy-engine for + carrying out some task) +* status: status +* policies: available policies + +That is, the only difference between a policy-engine and a data-source is that +the datasource doesn't support 'policies'. If we were to think of 'policies' +as simply 'modules' then we could imagine a datasource-driver exposing +hierarchical tables, just like a policy-engine does. And in that case, +the data-source would have something analogous to the policy-engine 'policies'. + + +Proposal 2: Service-based +-------------------------- +Here we do away with the types policy-engine and data-sources and give clients +the ability to access both policy-engines and datasources from the same +endpoint. This is possible because you cannot give a policy-engine and +a datasource the same name (a restriction in place to ensure references to +other services in policy are unambiguous). + +/v1/services//... +/v1/system/... + +For example: +/v1/services/nova/... +/v1/services/neutron/... +/v1/services/agnostic/... + +All services would support: + +* schema: available tables (e.g. classification:connected_to_internet) +* actions: available actions (e.g. scripts built into a policy-engine for + carrying out some task) +* status: status +* policies: available policies + +In short, this proposal treats everything as if it is a policy-engine. The +datasource policy engines prohibit users from making changes directly (and +even if they did, they would only accept a very restricted form of policy +statements: ground facts). + +This approach can be backwards compatible as well. We can still support + +/v1/datasources//... + +which gets routed to + +/v1/services/. + +And we can support + +/v1/policies +/v1/actions +/v1/tables +/v1/status + +as syntactic sugar for + +/v1/services//policies +/v1/services//actions +/v1/services//tables +/v1/services//status + +This would enable the user to choose a single touchpoint for managing policy +in the datacenter, while at the same time enabling them direct access to all +the services in the datacenter. (The only worry here is that ) + +To set the , we'd want a parameter the user can set +dynamically (to help with upgrade). +Right now we would set that to /services/agnostic. +And maybe we can have arbitrary aliases for services as well, so that we can +upgrade any service without changing policy. + +One worry with providing the /v1/policies, etc. endpoints is that it may +seem to mask Congress's overall status, policies, actions, and tables. That +is, people might expect those endpoints to aggregate all the potential policies, +actions, tables, and statuses. But if such functionality ever becomes +necessary, we can attach those endpoints to /v1/services, giving us the +following end points. + +/services/policies +/services/actions +/services/tables +/services/status + +Here is an example of the entry points if we had Nova, GBP, and our policy engine. The name of the service is whatever name DSE expects. + +/policies +/actions +/tables +/services/policies +/services/actions +/services/tables + +/services/nova/policies -> empty +/services/nova/actions -> createVM, deleteVM, migrateVM, etc. +/services/nova/tables -> servers, hosts, etc. +/services/gbp/policies +/services/gbp/actions +/services/gbp/tables +/services/engine/policies +/services/engine/actions +/services/engine/tables + +/system/drivers/ + +/system/engine-drivers/nova-uber-driver +/system/datasource-drivers/nova-uber-driver +/system/action-drivers/nova-uber-driver + + +/users +/stats + +One benefit to enabling people to modify domain-specific policy engines +through the Congress API is that we provide a single policy language for +managing all the policy engines running in the datacenter. For delegation, +we already need adapters that translate Datalog into the native language of +each policy engine, so here we expose that functionality directly to the user +as well. + + + + +Tradeoffs +------------ + +Pros for the type-based approach: +* Easy for users to understand +* Simple extension of the current API + +Cons for the type-based approach: +* Awkward that data-sources and policy-engines implement almost exactly the + same interface and have separate namespaces, but are represented as + distinct classes in the API. +* Enables us to build datasources with significantly different programmatic + interface than policy-engines. If at the API-layer the two classes of + objects were almost indistinguishable, it would lead to better abstraction + and interfaces in the underlying implementation. + +Pros for the the service-based approach: +* All services running on the DSE are accessed identically from the API. This + is a more natural reflection of the reality of the nature of those services. + +Cons for the service-based approach: +* Bigger change +* May be more difficult for users to understand initially. +* Eventually the policy-engine class will include functionality that the + datasource class does not. Executing that functionality on a datasource + will cause a 404, and we cannot predict which will occur based on just the + URL. +* Doing something like listing all the datasources will require an API like + /v1/services?action=list&type=datasource instead of the more obvious + /v1/data-sources/. + + +Overall, the types (datasource vs. policyengine) will be present in both +proposals, but they will be emphasized much less in the service-based approach. +The service-based approach is closer to Python in that the system isn't able +to look at the code you've written (the URL) and check if the method you asked +for exists. The type-based approach is closer to C/Java in that the system IS +able to tell you if the method exists by just looking at the code (URL). + +Typically policy systems are quite dynamic in nature (you can change the +policy/code at runtime), and hence are closer to dynamic programming languages +like Python than to static languages like C/Java. We therefore typically +bias our decisions toward dynamism, which in this case would mean the +service-based approach. + + +Policy +------ + +N/A + +Policy actions +-------------- + +N/A + +Data sources +------------ + +N/A + +Data model impact +----------------- + +N/A + +REST API impact +--------------- + +See above. No changes to API results--just the paths for invoking them. + +Security impact +--------------- + +N/A + +Notifications impact +-------------------- + +N/A + +Other end user impact +--------------------- + +N/A + +Performance impact +------------------ + +N/A + +Other deployer impact +--------------------- + +See above. + +Developer impact +---------------- + +N/A + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + + +Other contributors: + + +Work items +---------- + +Once we decide on the approach, we will figure out the necessary work items. +But here's a rough cut. + +- type-based approach: add routes, create congress/api/engine_model.py, modify + congress/api/*_model to enable tables/actions/policies/etc. for engines. +- service-based approach: add routes, create congress/api/service_model.py, + (including an API to list different types of objects), modify the + congress/api/*_model to eliminate distinction between datasources and policies + + + +Dependencies +============ + +Assumes that we have added an API call for 'actions', though this work could +be done without that: add-action-listing. + + +Testing +======= + +- Change unit tests in congress/tests/test_congress.py +- Change congress_pythonclient (which will handle tempest tests) + +Documentation impact +==================== + +Many URLs may change. + +References +========== + +N/A