summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJan Klare <j.klare@x-ion.de>2015-08-14 11:02:10 +0200
committerJan Klare <j.klare@x-ion.de>2015-09-24 20:25:16 +0200
commit994f8f220c4f2f2bd31fae1fa15a60ce9e1591ad (patch)
tree4ac3679b53a9fbc23fcebfa020267c59f27f14fd
parent7445f03a1984b169e5bb04073e94493c9305ecc9 (diff)
add spec for refactoring configuration templates
Notes
Notes (review): Verified+2: Jenkins Code-Review+2: Mark Vanderwiel <vanderwl@us.ibm.com> Workflow+1: JJ Asghar <jj@getchef.com> Submitted-by: Jenkins Submitted-at: Thu, 24 Sep 2015 19:26:01 +0000 Reviewed-on: https://review.openstack.org/213042 Project: openstack/openstack-chef-specs Branch: refs/heads/master
-rw-r--r--specs/liberty/all/refactor_config_templates.rst232
1 files changed, 232 insertions, 0 deletions
diff --git a/specs/liberty/all/refactor_config_templates.rst b/specs/liberty/all/refactor_config_templates.rst
new file mode 100644
index 0000000..db43892
--- /dev/null
+++ b/specs/liberty/all/refactor_config_templates.rst
@@ -0,0 +1,232 @@
1..
2 This work is licensed under a Creative Commons Attribution 3.0 Unported
3 License.
4
5 http://creativecommons.org/licenses/by/3.0/legalcode
6
7=================================================
8Refactor configuration templates in all cookbooks
9=================================================
10
11URL of launchpad blueprint:
12
13https://blueprints.launchpad.net/openstack-chef/+spec/refactor-configuration-templates
14
15Problem description
16===================
17
18Our templates for the configurations files for all services have grown through
19the years and it is getting harder and harder to figure out where to set a
20specific option that goes into the config files. The same is true for reading
21these configuration files after they have been rendered, since they contain a
22lot more options and code than actually needed or used. Additionally every time
23one of the OpenStack service projects adds or removes an configuration option,
24we manually need to add an attribute, some conditions and an entry in the
25template for this. Same goes for every change in the defaults, that is either
26not reflected at all in our config files, or set explicitly to the same thing,
27which is just unneeded code after all. Finally, the fact that all of our
28templates are quite big blocks of code and even contain some hardcoded values
29for specific configuration options, which came from the original configuration
30files imported years ago, does not help with keeping up with the multiple changes
31in the service projects.
32
33Proposed change
34===============
35
36The templates for OpenStack service configurations files should implement or use
37a logic of setting the needed key-value configuration options in the appropriate
38section automatically from properly defined chef attributes.
39
40* logic to render proper configuration files needs to be defined and added
41 (example given below)
42* attributes used for or in configurations templates need to be checked and
43 refactored to fit this logic
44* commonly used configuration options should be handled in one place
45* only needed configuration options should be set in the configuration files
46 after rendering the template
47* configuration options that are specific to only a few setups should be removed
48 from the default attributes and recipes (especially the switchcases for them)
49 and moved to the documentation. Although this will mean to remove
50 functionality in the sense of removing some switchcases and attributes in the
51 original cookbooks, this should not mean to loose these completely, but rather
52 move them to the documentation to allow an easy implementation in wrapper
53 cookbooks.
54
55Scope
56-----
57
58This spec tries to define a refactoring process that implements a more elegant
59handling of template files for OpenStack service configurations throughout all
60cookbooks used to deploy OpenStack.
61
62
63End user impact
64---------------
65
66* Users of the cookbooks will be able to wrap all OpenStack chef cookbook more
67 easily and set configuration options if needed without patching the actual
68 templates.
69* Old wrapper cookbooks before this change will stop working partially and
70 would need to be refactored accordingly. The same is true for all environment
71 files that include configuration options for the services via attributes.
72* Reading configuration files will become a lot easier, since only specifically
73 wanted and needed options will be included in them.
74* The user can add environment and company specific comments in the service
75 configuration files via attributes.
76
77
78Developer impact
79----------------
80
81Cookbook developers will not need to add attributes for configuration templates,
82nor should there be any need to change existing attributes to fit the changing
83defaults in OpenStack service projects. It should become easier to read the
84overall cookbook, since a big part of the code in the templates and attribute
85files will be cut or moved with this feature. Therefore it might be easier to
86contribute as a newcomer without spending hours to understand how all the
87attributes work with each other (given that the logic used to render the
88templates is easier to understand of course).
89
90Implementation
91==============
92
93Assignee(s)
94-----------
95
96Primary assignee:
97 <j-klare>
98
99Other contributors:
100
101Work Items
102----------
103
104* define a generic structure for templates, that can be used for most of the
105 configuration files
106* main config files for services look like this for all configuration
107 options:
108
109 [section]
110 # optional comment or description
111 key = value (usually simple boolean, numeric or string)
112 # optional comment or description
113 key = [ array which could be large ]
114
115* create a logic to render configuration files from properly set chef attributes
116* attributes could look like this for main configuration files (e.g. nova
117 or neutron.conf):
118
119::
120
121 default['openstack'][service]['conf'][section][key] =
122 value
123
124without a comment:
125
126::
127
128 default['openstack'][service]['conf'][section]['debug'] =
129 true
130
131with a comment:
132
133::
134
135 default['openstack'][service]['conf'][section]['debug'] =
136 { set_to: true, comment: 'this option is the switch to enable/disable debug loggging' }
137
138cookbook-openstack-common/templates/common.conf.erb :
139
140::
141
142 <% @service_config.each do |section, values| -%>
143 [<%= section %>]
144 <% values.each do |key, value| -%>
145 <% if value.class == Hash -%>
146 <%= "# {value['comment']}" -%>
147 <%= key %> = <%= value['set_to'] %>
148 <% else -%>
149 <%= key %> = <%= value %>
150 <% end -%>
151 <% end -%>
152 <% end -%>
153
154cookbook-openstack-compute/recipes/common.rb :
155
156::
157
158 ...
159 # activesupport currently needs to be upgraded, since the one used in
160 # chef 12 is too old (3.2.19) and does not include the needed deep_merge
161 chef_gem 'activesupport' do
162 action :upgrade
163 version '4.2.4'
164 end
165 require 'active_support'
166 ...
167
168 neutron_admin_password = get_password 'service', 'openstack-network'
169 identity_admin_endpoint = admin_endpoint 'identity-admin'
170 identity_uri = identity_uri_transform(identity_admin_endpoint)
171 ...
172
173 secrets = {
174 neutron: {
175 admin_password: neutron_admin_password,
176 ...
177 },
178 keystone_authtoken: {
179 identity_uri: identity_uri,
180 ...
181 },
182 ...
183 }
184
185 # merge node attributes with secrets like passwords etc. for
186 # usage in template['/etc/nova/nova.conf']
187 nova_config_options =
188 node['openstack']['compute']['conf'].deep_merge(secrets)
189
190 template '/etc/nova/nova.conf' do
191 source 'common.conf.erb'
192 cookbook 'openstack-common'
193 owner node['openstack']['compute']['user']
194 group node['openstack']['compute']['group']
195 mode 00640
196 variables(
197 service_config: nova_config_options
198 )
199 end
200 ...
201
202cookbook-openstack-compute/templates/nova.conf.erb -- not needed anymore
203
204* add a link to documentation/config reference to all config files
205* refactor currently used attributes to fit into that logic
206* adapt specs
207* define a set of minimal needed attributes to create a working stack and move
208 the rest of the attributes into documentation
209* remove attributes that would set configuration options that are equal to the
210 defaults
211* propagate not backward compatible change at a fitting point in time without
212 making a lot of people angry
213
214
215Testing
216=======
217
218* lint and style tests with rubocop (as is)
219* unit tests with chefspec with special focus on testing the proper rendering of
220 the configuration templates (including the sections)
221* integration testing with chef provisioning
222
223
224Documentation Impact
225====================
226
227These patches should move a good part of the currently defined attributes to the
228documentation as examples for specific setups.
229
230
231References
232==========