summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorDoug Szumski <doug@stackhpc.com>2018-10-04 08:47:31 +0100
committerWitold Bedyk <witold.bedyk@est.fujitsu.com>2018-11-28 14:29:56 +0100
commitb285ca6486ad3cedfc182170d7970cc5d0ce590e (patch)
tree1f19292a2b3a63f2ad45969de0f7d4b6222f56c2
parentd3613f3ba9f9a9b447564019a9fd47290fda934d (diff)
Merge Monasca APIs into a single unified API
This commit serves as a straw man for merging the Monasca APIs into a single unified API. APIImpact Story: 2003881 Task: 26742 Change-Id: Id08a46bb5b54db5baa5c3595100b73bab85ea8ff
Notes
Notes (review): Code-Review+1: Doug Szumski <doug@stackhpc.com> Code-Review+2: Witold Bedyk <witold.bedyk@est.fujitsu.com> Code-Review+1: Joseph Davis <joseph.davis@suse.com> Workflow+1: Dobroslaw Zybort <dobroslaw.zybort@ts.fujitsu.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Fri, 30 Nov 2018 13:38:26 +0000 Reviewed-on: https://review.openstack.org/609055 Project: openstack/monasca-specs Branch: refs/heads/master
-rw-r--r--specs/stein/approved/merge_apis.rst251
1 files changed, 251 insertions, 0 deletions
diff --git a/specs/stein/approved/merge_apis.rst b/specs/stein/approved/merge_apis.rst
new file mode 100644
index 0000000..b1e9209
--- /dev/null
+++ b/specs/stein/approved/merge_apis.rst
@@ -0,0 +1,251 @@
1..
2 This work is licensed under a Creative Commons Attribution 3.0 Unported
3 License.
4
5 http://creativecommons.org/licenses/by/3.0/legalcode
6
7==================
8Merge Monasca APIs
9==================
10
11https://storyboard.openstack.org/#!/story/2003881
12
13Monasca currently has 3 APIs; one for metrics, one for logs and one for
14events. Historically, this has created additional overhead for developers
15upgrading or adding new features to the APIs, for system administrators
16configuring the APIs, and for packagers generating Monasca distributions.
17The purpose of this spec is to consider whether the Monasca APIs should
18be merged into a single, unified API.
19
20Problem description
21===================
22
23The Monasca APIs are written in Python and share a lot of functionality,
24some, but not all of which has been factored out to the Monasca common
25library. Here lies the first problem:
26
271. There exists significant technical debt.
28
29There is a significant amount of common functionality that has not been
30factored out to Monasca common. For example, the Monasca API repo contains
31code for validating metrics and so does the Monasca common library. The
32same is true for query authorisation, where validation code lives in
33both repos. This technical debt could of course be addressed, but how
34did it come about?
35
362. Updating the APIs to support common features incurs significant
37 developer and reviewer overhead compared to updating a single API.
38
39The APIs are all written slightly differently. Adding support for
40Oslo Policy or uWSGI to one API is not the same as adding it to another
41API. Furthermore, changes frequently have to be made across multiple
42repos. Whilst Zuul is well suited to coordinating this task, it requires
43careful synchronisation of versions by developers and meticulous
44attention from reviewers. If one modifies code which is common to all
45three APIs, they must systematically verify that in each case the changes
46work as intended. Of course, automated testing can provide relief here, but
47it doesn't make the burden disappear entirely.
48
493. Historically it has been difficult to maintain a standard experience
50 across multiple APIs.
51
52At times, APIs have diverged when in an ideal world they should not have.
53For example, when adding support for a common feature, work has traditionally
54been focused on one API to keep the task simple, with the view to adding
55support for the other APIs at a later date. If works stops, for example due
56to a release, or due to a developer moving on, the APIs can be left in a
57diverged state for a significant amount of time. Of course, one solution
58is to block merging of a common feature until the work is complete
59in all three APIs, and all common code is added to the Monasca
60common library. In practice, this has been prevented by the number of man
61hours available. For example, a contributor may only be interested in
62metrics. They may not be able to justify spending time working on the other
63APIs.
64
654. Packaging, deploying, and configuring 3 APIs is more complex than
66 deploying a single, unified API.
67
68This is an obvious point, but it can be made worse by 3). For example,
69historically, it has not always been possible to run all APIs in the same
70way. Configuration of common functionality such as Kafka has not
71always been uniform. There is extra build system configuration,
72additional keystone endpoints that need to be registered and monitoring
73the availability of 3 APIs is more difficult than one.
74
75Use Cases
76---------
77
78As a developer I would save time by implementing a feature common
79to all APIs in a single repo, rather than across four repos.
80
81As a reviewer I will save time by not having to think about how
82a change may impact multiple repos.
83
84As a deployer I will save time by having two fewer services to deploy
85and configure.
86
87As a security analyst I can focus my efforts reviewing a single API,
88rather than three.
89
90As a user of Monasca I would like a consistent experience, including usability
91and documentation.
92
93As a packager I would like to have a single Monasca API, rather than
94three to save time configuring and maintaining my build system.
95
96Proposed change
97===============
98
99Merge all APIs into a single unified API. Merge all common API code from the
100Monasca common repo into the unified API repository. Specifically, it is
101proposed to merge all relevant code into the Monasca API.
102
103Alternatives
104------------
105
1061. Refactor all APIs so that the code is standard. Prevent merging of common
107 features until the work is complete across all APIs, and all common code
108 has been factored out into the Monasca common library. From historical
109 experience this will be difficult without additional developers.
110
1112. Don't do anything and carry on working around the technical debt. In the
112 long term this is likely to make it more difficult to add new features, and
113 require more time for maintenance.
114
115Data model impact
116-----------------
117
118None
119
120REST API impact
121---------------
122
123Aside from the fact that a single service will implement the combined schema
124from all APIs, the calls should not change. We should be careful when merging,
125for example dimension validation code, that we do not break things which were
126accepted in one API, but not another. An ideal result would be that we use
127the same code for parsing common fields such as dimensions for all API calls.
128
129Security impact
130---------------
131
132If a single API was previously exposed publicly, deploying the unified API
133will increase the surface area for attack. However, in general, it will be
134easier to review changes to make security improvements due to the reduced
135developer and reviewer overhead.
136
137During the deprecation period of the Events and Log API, security fixes made
138to the unified API may need to be backported.
139
140Other end user impact
141---------------------
142
143All services talking to the Monasca Events and Log APIs will need
144to be directed at the unified API. For services which use Keystone
145to lookup the Monasca endpoints this should occur automatically. Other
146services such as the Fluentd Monasca output plugin will need to be
147reconfigured manually. A grace period where the Monasca Events and Log
148APIs are still supported is one possibility to make this transition
149easier.
150
151Performance Impact
152------------------
153
154No direct impact.
155
156In general, it will be less effort to profile and optimise a single API than
157it would be to do the same for all three APIs.
158
159In clustered deployments with specialised nodes (for example a dedicated
160logging node, hosting the Monasca Log API) the unified API can be deployed
161as a direct replacement, and the additional functionality can simply be
162ignored.
163
164Other deployer impact
165---------------------
166
167For Monasca users supporting legacy releases, any security or bug fixes
168made to the unified API may need to be backported to the individual APIs.
169
170All actively maintained deployment solutions will need to be updated to
171deploy the unified API. For example, Monasca-Docker, DevStack, Kolla,
172OpenStack Ansible, and Helm.
173
174In the case of DevStack we should merge the three existing plugins into
175one. The resulting plugin should have options like `log_pipeline_enabled`
176and `metrics_pipeline_enabled` to support enabling those pipelines
177separately. This is useful, for example, when DevStack is used in OpenStack
178CI to allow testing changes localised to specific areas more efficiently.
179
180Developer impact
181----------------
182
183The motiviation behind this change is to reduce the burden placed on
184developers and reviewers when making improvements to the Monasca APIs. It
185is hoped that this will lead to an increase in developer productivity.
186
187Implementation
188==============
189
190Assignee(s)
191-----------
192
193Primary assignee:
194 <launchpad-id or None>
195
196Other contributors:
197 <launchpad-id or None>
198
199Work Items
200----------
201
202* Review test coverage of APIs, and add coverage for any missing areas.
203* Review test coverage of Monasca common and add coverage for any missing areas.
204* Merge Monasca common Python code into the Monasca API. Common functionality
205 should include:
206
207 * authorisation
208 * validation
209 * OpenStack policy
210 * WSGI deployment
211
212* Implement Log API schema in the Monasca API and port tests.
213* Implement Events API schema and port tests.
214* Merge DevStack plugins into a single plugin and add support for enabling
215 pipelines individually.
216* Deprecate Monasca Log and Event APIs.
217* Merge and update documentation
218
219Dependencies
220============
221
222No additional dependencies are added. The dependency on Monasca common can
223be removed.
224
225Testing
226=======
227
228The Monasca API and Log API Tempest test plugins have already been merged into
229one plugin. Any Tempest tests which exist for the Events API should also be
230merged into the unified plugin. The Tempest plugin will need to be updated to
231use the unified API.
232
233Unit tests from the Log API and Events API repo will need to be ported to the
234Monasca API (unified) repo. Some of these tests may be redundant.
235
236Documentation Impact
237====================
238
239Three sets of documentation will be reduced to one. Whilst it will take
240some effort to merge the documentation, it should hopefully be more
241consistent.
242
243References
244==========
245
246None
247
248History
249=======
250
251None