diff options
authorMonty Taylor <>2016-11-08 09:15:07 -0600
committerMonty Taylor <>2016-11-08 15:34:23 -0600
commit5f8ea0588ae5e54c70e28d5ebb1b56ac4bf494ba (patch)
parent7ee4290eb620bf82d54bcd881e313565be2085c7 (diff)
Add a few more docs
Notes (review): Code-Review+2: Monty Taylor <> Workflow+1: Monty Taylor <> Verified+2: Jenkins Submitted-by: Jenkins Submitted-at: Tue, 08 Nov 2016 22:34:27 +0000 Reviewed-on: Project: openstack/oaktree Branch: refs/heads/master
8 files changed, 146 insertions, 12 deletions
diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index 2a8be82..237692a 100644
@@ -12,6 +12,6 @@ submitted for review via the Gerrit tool:
12 12
13Pull requests submitted through GitHub will be ignored. 13Pull requests submitted through GitHub will be ignored.
14 14
15Bugs should be filed on Launchpad, not GitHub: 15Bugs should be filed on Storyboard, not GitHub:
16 16
17 17!/project/855
diff --git a/doc/source/ b/doc/source/
index 132cf55..3f1ba36 100755
--- a/doc/source/
+++ b/doc/source/
@@ -23,7 +23,6 @@ sys.path.insert(0, os.path.abspath('../..'))
23extensions = [ 23extensions = [
24 'sphinx.ext.autodoc', 24 'sphinx.ext.autodoc',
25 #'sphinx.ext.intersphinx', 25 #'sphinx.ext.intersphinx',
26 'oslosphinx'
27] 26]
28 27
29# autodoc generation is a bit aggressive and a nuisance when doing heavy 28# autodoc generation is a bit aggressive and a nuisance when doing heavy
diff --git a/doc/source/design.rst b/doc/source/design.rst
new file mode 100644
index 0000000..ef064ce
--- /dev/null
+++ b/doc/source/design.rst
@@ -0,0 +1,39 @@
2Oaktree Design
5Once 1.0.0 is released, oaktree pledges to never break backwards compatability.
7Oaktree is intended to be safe for deployers to run CD from master. In fact,
8a deployer running a kilo OpenStack should be able to install tip of master
9of oaktree and have everything be perfectly fine.
11Oaktree must be simple to install and operate. A single node install with no
12shared caching or locking is likely fine for most smaller clouds. For larger
13clouds, shared caching and locking are essential for scale out. Both must be
14supported, and simple.
16Oaktree is not pluggable.
18Oaktree does not allow selectively enabling or disabling features or part of
19its API.
21Oaktree should be runnable by an end user pointed at a local clouds.yaml file.
23Oaktree should be able to talk to other oaktrees.
25Oaktree users should never need to know any information about the cloud other
26than the address of the oaktree endpoint. Cloud-specific information the
27user needs to know must be exposed via a capabilities API. For instance, in
28order for a user to upload an image to a cloud, the user must know what format
29the cloud requires the image to be in. The user must be able to ask oaktree
30what image format(s) the cloud accepts.
32Data returned from oaktree should be normalized such that it is consistent
33no matter what drivers the cloud in question has chosen. This work is done in
34shade, but shapes the design of the protobuf messages.
36All objects in oaktree should have a Location. A Location defines the cloud,
37the region, the zone and the project that contains the object. For objects
38that exist at a region and not a zone level, like flavors and images, zone
39will be null. For objects that exist at a cloud level, region will be null.
diff --git a/doc/source/faq.rst b/doc/source/faq.rst
new file mode 100644
index 0000000..9ada518
--- /dev/null
+++ b/doc/source/faq.rst
@@ -0,0 +1,51 @@
2Frequently Asked Questions
5Why gRPC and not REST?
8There are three main reasons.
10We already have REST APIs. oaktree is not intended to replace them, but to
11supplement them to grease the 80% case that can be inter-operable.
13gRPC comes out of the gate with direct support for a pile of languages, so
14supporting our non-Python friends is direct and straightforward.
16A TON of time is spent in shade polling OpenStack for results. That may not
17sound like a problem - but when you spin up thousands of VMs a day like Infra
18does, the polling becomes a major engineering challenge. gRPC operates over
19http/2 and has support for bi-directional channels - which means you can just
20have a function notify you when something is done. That's a win for everyone
22Why write it in Python rather than XXX?
25The hard part of this isn't the gRPC api - it's the business logic that's in
26the shade library. If we wrote oaktree from scratch in C++ (because hello
27super-high-performance gRPC backend!) - we'd be faced with the task of
28re-implementing all of the shade business logic in C++. If you haven't looked,
29there is a LOT.
31shade is what infra uses for nodepool. It has copious features in it already
32to deal with extremely high scale - including configurable caching, batched
33list update operations to prevent thundering herds and well exercised
34multi-threaded support.
36The interesting part also isn't the server (it's a simple proxy layer) - it's
37the clients. THOSE definitely want much love in the different languages. The
38infrastructure is in place for Python, C++ and Go. Ruby, javascript and C#
39should follow asap.
41Can I add support for my project?
44Yes. It has to be added to shade first, which accepts patches from anything
45that can be tested consistently in a devstack job. We require all new features
46in shade to come with functional tests. Once it's in shade, it can be added as
47an API to oaktree.
49However ... oaktree and shade both promise 100% backwards compatibility at all
50times. If your project is still young, be aware that once an API is added to
51shade or oaktree it will need to be supported until the end of time.
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 7dfe9fc..1f9ff69 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -13,7 +13,9 @@ Contents:
13 13
14 readme 14 readme
15 installation 15 installation
16 usage 16 design
17 faq
18 todo
17 contributing 19 contributing
18 20
19Indices and tables 21Indices and tables
diff --git a/doc/source/todo.rst b/doc/source/todo.rst
new file mode 100644
index 0000000..d5f3ce2
--- /dev/null
+++ b/doc/source/todo.rst
@@ -0,0 +1,51 @@
2Work Needed
5Design the auth story
8The native/default auth for gRPC is oauth. It has the ability for pluggable
9auth, but that would raise the barrier for new languages. I'd love it if we
10can come up with a story that involves making API users in keystone and
11authorizing them to use oaktree via an oauth transaction. The keystone auth
12backends currently are all about integrating with other auth management
13systems, which is great for environments where you have a web browser, but not
14so much for ones where you need to put your auth credentials into a file so
15that your scripts can work. I'm waving my hands wildly here - because all I
16really have are problems to solve and none of the solutions I have are great.
18Design Glance Image / Swift Object Uploads and Downloads
21Having those two data operations go through an API proxy seems inefficient.
22However, having them not in the API seems like a bad user experience. Perhaps
23if we take advantage of the gRPC streaming protocol support doing a direct
24streaming passthrough actually wouldn't be awful. Or maybe the better approach
25would be for the gRPC call to return a URL and token for a user to POST/PUT to
26directly. Literally no clue.
28Design and implement Capabilities API
31shade and the current oaktree codebase rely on os-client-config and clouds.yaml
32for information about the cloud and what it can do. As a service, some of the
33pieces of information in os-client-config need to be queriable by the user.
35Implement API surfaces
38In general, all of the API operations shade can perform should be exposed in
39oaktree. In order to shape that work, we should tackle them in the following
42#. API surface needed for nodepool
43#. API surface needed for existing Ansible modules
44#. Everything else
46Implement oaktree backend in shade
49It's turtles all the way down. If shade sees that a cloud has an oaktree
50service, shade should talk to it over gRPC instead of talking to the REST
51APIs directly.
diff --git a/doc/source/usage.rst b/doc/source/usage.rst
deleted file mode 100644
index 7208c70..0000000
--- a/doc/source/usage.rst
+++ /dev/null
@@ -1,7 +0,0 @@
5To use oaktree in a project::
7 import oaktree
diff --git a/test-requirements.txt b/test-requirements.txt
index 0c885d3..85ecf12 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -8,7 +8,6 @@ coverage>=3.6
8discover 8discover
9python-subunit>=0.0.18 9python-subunit>=0.0.18
10sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2 10sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2
11oslosphinx>=2.5.0 # Apache-2.0
12oslotest>=1.10.0 # Apache-2.0 11oslotest>=1.10.0 # Apache-2.0
13testrepository>=0.0.18 12testrepository>=0.0.18
14testscenarios>=0.4 13testscenarios>=0.4