Fuel HA Fencing plugin for puppet

All documentation provided in README.md

Implements blueprint fencing-in-puppet-manifests

* Use Fuel corosync from 5.1.1
* Add cluster-recheck-interval 3 min setting
* Add parser functions and facts from Fuel library
* Add pre-build hook for dependencies:
  * puppetlabs/stdlib v 4.5.0
  * Fuel corosync v 5.1.1
* Add examples of YAML for fence_virsh, fence_ipmilan,
  fence_apc_snmp and fence topology

Change-Id: I15dc9ff747957f7d22ca3ccd12628423c3c5c8cc
Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
This commit is contained in:
Bogdan Dobrelya 2014-12-19 13:50:10 +01:00
commit 7382d88ccf
39 changed files with 2085 additions and 0 deletions

5
.gitignore vendored Normal file
View File

@ -0,0 +1,5 @@
deployment_scripts/puppet/modules/corosync
deployment_scripts/puppet/modules/stdlib
.build
tmp
*.fp

202
LICENSE Normal file
View File

@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright {yyyy} {name of copyright owner}
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

178
README.md Normal file
View File

@ -0,0 +1,178 @@
Fuel Fencing Plugin
===================
#### Table of Contents
1. [Overview - What is the Fuel fencing plugin?](#overview)
2. [Plugin Description - What does the plugin do?](#plugin-description)
3. [Setup - The basics of getting started with Fuel fencing plugin](#setup)
4. [Implementation - An under-the-hood peek at what the plugin is doing](#implementation)
5. [Limitations - OS compatibility, etc.](#limitations)
6. [Development - Guide for contributing to the plugin](#development)
7. [Contributors - Those with commits](#contributors)
8. [Release Notes - Notes on the most recent updates to the plugin](#release-notes)
Overview
--------
The Fuel fencing plugin is a Puppet configuration module being executed as an
additional post deployment step in order to provide HA fencing (STONITH based)
of a failed cluster nodes.
The plugin itself is used to describe a datacenter's HW power management
configuration - such as PDU outlets, IPMI and other agents - and represent
it as a fencing topology for Corosync & Pacemaker cluster.
[About Fuel plugins](https://software.mirantis.com/mirantis-openstack-fuel-plug-in-development/)
Plugin Description
------------------
The Fuel fencing plugin is intended to provide STONITHing of the failed nodes
in Corosync & Pacemaker cluster.
Fencing plugin operates the YAML data structures and extends the Fuel YAML
configuration file.
It suggests the manual definition of the YAML data structures with all required
parameters for existing power management (PM) devices for every controller node.
It is up to the user to collect and verify all the needed IP adresses, credentials,
power outlets layouts, BM hosts to PM devices mappings and other parameters.
This plugin also installs fence-agents package and assumes
there is the one avaiable in the OS repositories.
Setup
-----
### Installing Fencing plugin
Please refer to the [plugins dev guide](http://docs.mirantis.com/fuel/fuel-6.0/plugin-dev.html#what-is-pluggable-architecture)
Note that in order to build this plugin the following tools must present:
* rsync
* wget
### Beginning with Fencing plugin
* Create an HA environment and select the fencing policy (reboot, poweroff or
disabled) at the settings tab.
* Assign roles to the nodes as always, but use Fuel CLI instead of Deploy button
to provision all nodes in the environment. Please note, that the power management
devices should be reachable from the management network via TCP protocol.
* Define YAML configuration files for controller nodes and existing power management
(PM aka STONITH) devices. See an example in
``deployment_scripts/puppet/modules/pcs_fencing/examples/pcs_fencing.yaml``.
In the given example we assume 'reboot' policy, which is a hard resetting of
the failed nodes in Pacemaker cluster. We define IPMI reset action and PSU OFF/ON
actions for ``fence_ipmilan`` and ``fence_apc_snmp`` agent types.
These agents will be contacted by Pacemaker stonith-ng daemon to STONITH controller
nodes (there are 3 of them according to the given fence topology) in the following
order:
* If IPMI device reported OK on reset action requested, STONITH is completed.
* if IPMI device cannot succeed on reset action for some reason, PSU OFF action
will be requested.
* In the case of PSU OFF action success, PSU ON action will be requested as well.
* In the case of both OFF and ON actions success, STONITH is completed OK.
* If either of them failed, repeat from the step 1, untill timeout exceeded.
(if timeout exceeded, STONITH is failed)
For other controllers, the same configuration stanza should be manually populated.
IP addresses, credentials, delay and indexes of the power outlets (in case of PDU/PSU)
of STONISH devices being connected by these agents should be updated as well as
node names.
Please note, that each controller node should have configured all of its fence agent
types ``delay`` parameters with an increased values. That is required in order to
resolve a mutual fencing situations then the nodes are triyng to STONITH each other.
For example, if you have 3 controllers, set all delay values as 0 for 1st controller,
10 - for 2nd one and 20 seconds - for the last one. The other timeouts should be
as well adjusted as the following:
```
delay + shell_timeout + login_timeout < power_wait < power_timeout
```
Fencing topology could vary for controller nodes but usually it is the same.
It provides an ordering of STONITH agents to call in case of the fencing actions.
It is recommended to configure several types of the fencing devices and put
them to the dedicated admin network. This network should be either directly connected
or reached from the management interfaces of controller nodes in order to provide a
connectivity to the fencing devices.
In the given example we define the same topology for node-10 and node-11 and slightly
different one for node-12 - just to illustrate that each node could have a different
fence agent types configured, hence, the different topology as well. So, we configure
nodes 10 and 11 to rely on IPMI and PSU devices, while the node 12 is a virtual node
and relies on virsh agent.
Please also note, that the names of nodes in fence topology stanza and ``pcmk_*``
parameters should be specified as FQDN names in case of RedHat OS family and as a
short names in case of Debian OS family. That is related to the node naming rules in
Pacemaker cluster in different OS types.
* Put created fencing configuration YAML files as ``/etc/pcs_fencing.yaml``
for corresponding controller nodes.
* Deploy HA environment either by CLI command or Deploy button
TODO(bogdando) finish the guide, add agents and devices verification commands
Implementation
--------------
### Fuel Fencing plugin
This plugin is a combination of Puppet module and metadata required to
describe and configure the fencing topology for Corosync & Pacemaker
cluster. The plugin includes custom puppet module pcs_fencing and as a dependencies,
custom corosync module and puppetlabs/stdlib module v4.5.0.
It changes global cluster properties:
* cluster-recheck-interval = 3 minutes
* stonith-enabled = True
It creates a set of STONITH primitives in Pacemaker cluster and runs them in a way,
that ensures the node will never try to shoot itself (-inf location constraint).
It configures a fencing topology singleton primitive in Pacemaker cluster.
It uses crm command line tool which is deprecated and will be replaced to pcs later.
Limitations
-----------
* It is not recommended to use this plugin, if controller nodes contain any additional
roles (such as storage, monitoring, compute) in Openstack environment, because
STONITH'ed node in Pacemaker cluster will bring these additional roles residing at
this node down as well.
* Can be used only with the Debian and RedHat OS families with crm command line tool
available.
Development
-----------
Developer documentation for the entire Fuel project.
* https://wiki.openstack.org/wiki/Fuel#Where_can_documentation_be_found
Contributors
------------
Will be added later
Versioning
----------
This module has been given version 6 to track the Fuel releases. The
versioning for plugin releases are as follows:
```
Plugin :: Fuel version
6.0.0 -> 6.0
```
Release Notes
-------------
*** 6.0.0 ***
* This is the initial release of this plugin.

View File

@ -0,0 +1,21 @@
$fuel_settings = parseyaml($astute_settings_yaml)
# Fetch fencing policy and settings
$fence_policy = $::fuel_settings['ha_fencing']['fence_policy']
$fencing_enabled = $fence_policy ? { 'disabled'=>false, 'reboot'=>true, 'poweroff'=>true, default=>false }
if $fencing_enabled {
$fencing_settings = parseyaml($fencing_settings_yaml)
$fence_primitives = $::fencing_settings['fence_primitives']
$fence_topology = $::fencing_settings['fence_topology']
$nodes_hash = $::fuel_settings['nodes']
$controllers = concat(filter_nodes($nodes_hash,'role','primary-controller'), filter_nodes($nodes_hash,'role','controller'))
include stdlib
class { '::pcs_fencing::fencing_primitives':
fence_primitives => $fence_primitives,
fence_topology => $fence_topology,
nodes => $controllers,
}
}

View File

@ -0,0 +1,9 @@
fixtures:
repositories:
#corosync: 'https://github.com/puppetlabs/puppetlabs-corosync.git'
#stdlib: 'https://github.com/puppetlabs/puppetlabs-stdlib.git'
symlinks:
pcs_fencing: "#{source_dir}"
corosync: "#{source_dir}/../corosync"
stdlib: "#{source_dir}/../stdlib"

View File

@ -0,0 +1,3 @@
Gemfile.lock
spec/fixtures
.bundle

View File

@ -0,0 +1,16 @@
source 'https://rubygems.org'
group :development, :test do
gem 'puppetlabs_spec_helper', :require => false
gem 'puppet-lint', '~> 0.3.2'
gem 'rake', '10.1.1'
gem 'rspec', '< 2.99'
gem 'json'
gem 'webmock'
end
if puppetversion = ENV['PUPPET_GEM_VERSION']
gem 'puppet', puppetversion, :require => false
else
gem 'puppet', :require => false
end

View File

@ -0,0 +1,56 @@
GEM
remote: https://rubygems.org/
specs:
addressable (2.3.6)
crack (0.4.2)
safe_yaml (~> 1.0.0)
diff-lcs (1.2.5)
facter (2.3.0)
hiera (1.3.4)
json_pure
json (1.8.1)
json_pure (1.8.1)
metaclass (0.0.4)
mocha (1.1.0)
metaclass (~> 0.0.1)
puppet (3.7.3)
facter (> 1.6, < 3)
hiera (~> 1.0)
json_pure
puppet-lint (0.3.2)
puppet-syntax (1.3.0)
rake
puppetlabs_spec_helper (0.8.2)
mocha
puppet-lint
puppet-syntax
rake
rspec
rspec-puppet
rake (10.1.1)
rspec (2.14.1)
rspec-core (~> 2.14.0)
rspec-expectations (~> 2.14.0)
rspec-mocks (~> 2.14.0)
rspec-core (2.14.8)
rspec-expectations (2.14.5)
diff-lcs (>= 1.1.3, < 2.0)
rspec-mocks (2.14.6)
rspec-puppet (1.0.1)
rspec
safe_yaml (1.0.4)
webmock (1.20.4)
addressable (>= 2.3.6)
crack (>= 0.3.2)
PLATFORMS
ruby
DEPENDENCIES
json
puppet
puppet-lint (~> 0.3.2)
puppetlabs_spec_helper
rake (= 10.1.1)
rspec (< 2.99)
webmock

View File

@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2014 OpenStack Foundation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@ -0,0 +1,87 @@
pcs_fencing
===========
#### Table of Contents
1. [Overview - What is the pcs_fencing module?](#overview)
2. [Module Description - What does the module do?](#module-description)
3. [Setup - The basics of getting started with pcs_fencing](#setup)
4. [Implementation - An under-the-hood peek at what the module is doing](#implementation)
5. [Limitations - OS compatibility, etc.](#limitations)
6. [Development - Guide for contributing to the module](#development)
7. [Contributors - Those with commits](#contributors)
8. [Release Notes - Notes on the most recent updates to the module](#release-notes)
Overview
--------
TODO(bogdando) provide a link to Fuel plugin repo then ready.
The pcs_fencing module is a part of Fencing plugin for Fuel and have no
a separate code repository.
The module itself is used to configure fencing primitives in Pacemaker
and combine them into the Fencing topology.
Module Description
------------------
The pcs_fencing module is intended to provide STONITH based HA fencing
of the failed nodes in Corosync & Pacemaker cluster. This module
cannot be used separately from Fuel Fencing plugin.
Pcs_fencing module operates the data structures which are tight to the ones
in Fuel YAML configuration file and has no its own parameters.
This module also installs fence-agents package and assumes there is
the one avaiable in the OS repositories.
Setup
-----
### Installing pcs_fencing
This module is being installed automatically as a part of Fuel Fencing
plugin.
The module's rspec tests could be run only after the plugin is built as
its pre build hook will download required Fuel custom puppet module for corosync
and puppetlabs/stdlib module.
### Beginning with pcs_fencing
Instructions for beginning with pcs_fencing will be added later.
Implementation
--------------
### pcs_fencing
pcs_fencing is a combination of Puppet manifest and ruby code to delivery
configuration and extra functionality through custom types, providers, parser
functions and facts from Fuel library of puppet manifests.
Note that it requires a custom module for corosync and includes a custom
provider for fencing topology.
Limitations
-----------
Limitations will be added as they are discovered.
Development
-----------
Developer documentation for the entire Fuel project.
* https://wiki.openstack.org/wiki/Fuel#Where_can_documentation_be_found
Contributors
------------
Will be added later
Versioning
----------
This module is being versioned as well as Fuel Fencing plugin.
Release Notes
-------------
This module has no a separate release notes. See the release notes for
Fuel Fencing plugin.

View File

@ -0,0 +1,7 @@
require 'rubygems'
require 'puppetlabs_spec_helper/rake_tasks'
require 'puppet-lint/tasks/puppet-lint'
PuppetLint.configuration.fail_on_warnings = true
PuppetLint.configuration.send('disable_80chars')
PuppetLint.configuration.ignore_paths = ["spec/**/*.pp", "pkg/**/*.pp"]

View File

@ -0,0 +1,117 @@
#fence_policy: reboot
fence_topology:
node-10.test.local:
'1':
- ipmi_reset
'2':
- psu_off
- psu_on
node-11.test.local:
'1':
- ipmi_reset
'2':
- psu_off
- psu_on
node-12.test.local:
'1':
- virsh_reset
fence_primitives:
ipmi_reset:
agent_type: fence_ipmilan
operations:
monitor:
interval: 3600s
timeout: 120s
start:
interval: '0'
timeout: 120s
on-fail: restart
stop:
interval: '0'
timeout: 1800s
on-fail: restart
meta:
migration-threshold: '5'
failure-timeout: '180'
parameters:
ipaddr: 10.11.12.13
login: ipmi_user
passwd: ipmi_pass
privlvl: operator
auth: password
power_wait: '15'
delay: '300'
action: reboot
pcmk_reboot_action: reboot
pcmk_off_action: reboot
pcmk_host_list: node-10.test.local
psu_off:
agent_type: fence_apc_snmp
operations:
monitor:
interval: 3600s
timeout: 120s
start:
interval: '0'
timeout: 120s
on-fail: restart
stop:
interval: '0'
timeout: 1800s
on-fail: fence
meta:
migration-threshold: '5'
failure-timeout: '180'
parameters:
ipaddr: 10.11.12.14
login: tripplite
community: QWErty123
snmp_auth_prot: MD5
snmp_priv_prot: DES
port: '5'
snmp_sec_level: authPriv
passwd: QWErty123
power_timeout: '30'
shell_timeout: '10'
login_timeout: '10'
power_wait: '15'
delay: '300'
action: 'off'
pcmk_reboot_action: 'off'
pcmk_off_action: 'off'
pcmk_host_list: node-10.test.local
psu_on:
agent_type: fence_apc_snmp
operations:
monitor:
interval: 3600s
timeout: 120s
start:
interval: '0'
timeout: 120s
on-fail: restart
stop:
interval: '0'
timeout: 1800s
on-fail: fence
meta:
migration-threshold: '5'
failure-timeout: '180'
parameters:
ipaddr: 10.11.12.14
login: tripplite
community: QWErty123
snmp_auth_prot: MD5
snmp_priv_prot: DES
port: '5'
snmp_sec_level: authPriv
passwd: QWErty123
power_timeout: '30'
shell_timeout: '10'
login_timeout: '10'
power_wait: '15'
delay: '300'
action: 'on'
pcmk_reboot_action: 'on'
pcmk_off_action: 'on'
pcmk_host_list: node-10.test.local

View File

@ -0,0 +1,43 @@
#fence_policy: reboot
fence_topology:
node-7:
'1':
- virsh_reset
node-8:
'1':
- virsh_reset
node-9:
'1':
- virsh_reset
fence_primitives:
virsh_reset:
agent_type: fence_virsh
operations:
monitor:
interval: 3600s
timeout: 120s
start:
interval: '0'
timeout: 120s
on-fail: restart
stop:
interval: '0'
timeout: 1800s
on-fail: restart
meta:
migration-threshold: '5'
failure-timeout: '180'
parameters:
ipaddr: 10.108.5.1
login: virsh_ssh_user
passwd: virsh_ssh_pass
power_wait: '15'
power_timeout: '20'
shell_timeout: '10'
login_timeout: '5'
secure: true
delay: '300'
action: reboot
pcmk_reboot_action: reboot
pcmk_off_action: reboot
pcmk_host_map: 'node-7:env60_slave-07'

View File

@ -0,0 +1,15 @@
require 'facter'
fencing_settings_path = ['/etc/pcs_fencing.yaml']
fencing_settings_path.each do |fencing_file|
if File.exist?(fencing_file)
Facter.add('fencing_settings_file') do
setcode { fencing_file }
end
Facter.add('fencing_settings_yaml') do
setcode { File.read(fencing_file) }
end
break
end
end

View File

@ -0,0 +1,16 @@
require 'facter'
# This file is created and managed by Astute
astute_settings_path = ['/etc/fuel/astute.yaml', '/etc/astute.yaml']
astute_settings_path.each do |astute_file|
if File.exist?(astute_file)
Facter.add('astute_settings_file') do
setcode { astute_file }
end
Facter.add('astute_settings_yaml') do
setcode { File.read(astute_file) }
end
break
end
end

View File

@ -0,0 +1,9 @@
# Fact: pacemaker_hostname
#
# Purpose: Return name of the node used by Pacemaker
#
Facter.add(:pacemaker_hostname) do
setcode do
rv = Facter::Util::Resolution.exec('crm_node -n')
end
end

View File

@ -0,0 +1,13 @@
module Puppet::Parser::Functions
newfunction(:filter_hash, :type => :rvalue, :doc => <<-EOS
Map array of hashes $arg0 to an array yielding
an element from each hash by key $arg1
EOS
) do |args|
hash = args[0]
field = args[1]
hash.map do |e|
e[field]
end
end
end

View File

@ -0,0 +1,9 @@
module Puppet::Parser::Functions
newfunction(:filter_nodes, :type => :rvalue) do |args|
name = args[1]
value = args[2]
args[0].select do |it|
it[name] == value
end
end
end

View File

@ -0,0 +1,24 @@
#
# parseyaml.rb
#
module Puppet::Parser::Functions
newfunction(:parseyaml, :type => :rvalue, :doc => <<-EOS
This function accepts YAML as a string and converts it into the correct
Puppet structure.
EOS
) do |arguments|
if (arguments.size != 1) then
raise(Puppet::ParseError, "parseyaml(): Wrong number of arguments "+
"given #{arguments.size} for 1")
end
require 'yaml'
YAML::load(arguments[0])
end
end
# vim: set ts=2 sw=2 et :

View File

@ -0,0 +1,144 @@
require 'pp'
require 'open3'
require 'rexml/document'
class Puppet::Provider::Corosync < Puppet::Provider
def self.dump_cib
self.block_until_ready
stdout = Open3.popen3("#{command(:crm)} configure show xml")[1].read
return stdout, nil
end
def try_command(command,resource_name,should=nil,cib=nil,timeout=120)
cmd = "#{command(:crm)} configure #{command} #{resource_name} #{should} ".rstrip
env = {}
if cib
env["CIB_shadow"]=cib.to_s
end
Timeout::timeout(timeout) do
debug("Issuing #{cmd} for CIB #{cib} ")
loop do
break if exec_withenv(cmd,env) == 0
sleep 2
end
end
end
def exec_withenv(cmd,env=nil)
self.class.exec_withenv(cmd,env)
end
def self.exec_withenv(cmd,env=nil)
Process.fork do
ENV.update(env) if !env.nil?
Process.exec(cmd)
end
Process.wait
$?.exitstatus
end
# Corosync takes a while to build the initial CIB configuration once the
# service is started for the first time. This provides us a way to wait
# until we're up so we can make changes that don't disappear in to a black
# hole.
def self.block_until_ready(timeout = 120)
cmd = "#{command(:crm_attribute)} --type crm_config --query --name dc-version 2>/dev/null"
Timeout::timeout(timeout) do
until exec_withenv(cmd) == 0
debug('Corosync not ready, retrying')
sleep 2
end
# Sleeping a spare two since it seems that dc-version is returning before
# It is really ready to take config changes, but it is close enough.
# Probably need to find a better way to check for reediness.
sleep 2
end
end
def self.prefetch(resources)
instances.each do |prov|
if res = resources[prov.name.to_s]
res.provider = prov
end
end
end
def exists?
self.class.block_until_ready
Puppet.debug "Call exists? on cs_resource '#{@resource[:name]}'"
out = !(@property_hash[:ensure] == :absent or @property_hash.empty?)
Puppet.debug "Return: #{out}"
Puppet.debug "Current state:\n#{@property_hash.pretty_inspect}" if @property_hash.any?
out
end
def get_scope(type)
case type
when 'resource'
scope='resources'
when /^(colocation|order|location)$/
scope='constraints'
when 'rsc_defaults'
scope='rsc_defaults'
else
fail('unknown resource type')
scope=nil
end
return scope
end
def apply_changes(res_name,tmpfile,res_type)
env={}
shadow_name="#{res_type}_#{res_name}"
original_cib="/tmp/#{shadow_name}_orig.xml"
new_cib="/tmp/#{shadow_name}_new.xml"
begin
debug('trying to delete old shadow if exists')
crm_shadow("-b","-f","-D",shadow_name)
rescue Puppet::ExecutionFailure
debug('delete failed but proceeding anyway')
end
if !get_scope(res_type).nil?
cibadmin_scope = "-o #{get_scope(res_type)}"
else
cibadmin_scope = nil
end
crm_shadow("-b","-c",shadow_name)
env["CIB_shadow"] = shadow_name
orig_status = exec_withenv("#{command(:cibadmin)} #{cibadmin_scope} -Q > /tmp/#{shadow_name}_orig.xml", env)
#cibadmin returns code 6 if scope is empty
#in this case write empty file
if orig_status == 6 or File.open("/tmp/#{shadow_name}_orig.xml").read.empty?
cur_scope=REXML::Element.new(get_scope(res_type)).to_s
emptydoc=REXML::Document.new(cur_scope)
emptydoc.write(File.new("/tmp/#{shadow_name}_orig.xml",'w'))
end
exec_withenv("#{command(:crm)} configure load update #{tmpfile.path.to_s}",env)
exec_withenv("#{command(:cibadmin)} #{cibadmin_scope} -Q > /tmp/#{shadow_name}_new.xml",env)
patch = Open3.popen3("#{command(:crm_diff)} --original #{original_cib} --new #{new_cib}")[1].read
if patch.empty?
debug("no difference - nothing to apply")
return
end
xml_patch = REXML::Document.new(patch)
wrap_cib=REXML::Element.new('cib')
wrap_configuration=REXML::Element.new('configuration')
wrap_cib.add_element(wrap_configuration)
wrap_cib_a=Marshal.load(Marshal.dump(wrap_cib))
wrap_cib_r=Marshal.load(Marshal.dump(wrap_cib))
diff_a=REXML::XPath.first(xml_patch,'//diff-added')
diff_r=REXML::XPath.first(xml_patch,'//diff-removed')
diff_a_elements=diff_a.elements
diff_r_elements=diff_r.elements
wrap_configuration_a=REXML::XPath.first(wrap_cib_a,'//configuration')
wrap_configuration_r=REXML::XPath.first(wrap_cib_r,'//configuration')
diff_a_elements.each {|element| wrap_configuration_a.add_element(element)}
diff_r_elements.each {|element| wrap_configuration_r.add_element(element)}
diff_a.add_element(wrap_cib_a)
diff_r.add_element(wrap_cib_r)
cibadmin '--patch', '--sync-call', '--xml-text', xml_patch
end
end

View File

@ -0,0 +1,134 @@
require 'pathname'
require 'open3'
require Pathname.new(__FILE__).dirname.dirname.expand_path + 'corosync'
require 'rexml/document'
Puppet::Type.type(:cs_fencetopo).provide(:crm, :parent => Puppet::Provider::Corosync) do
desc 'Specific provider for a rather specific type since I currently have no plan to
abstract corosync/pacemaker vs. keepalived. This provider will create or destroy
a singleton for fencing topology configuration.'
# Path to the crm binary for interacting with the cluster configuration.
commands :crm => 'crm'
commands :cibadmin => 'cibadmin'
commands :crm_attribute => 'crm_attribute'
def self.instances
block_until_ready
raw, status = dump_cib
doc = REXML::Document.new(raw)
nodes = []
fence_topology = {}
# return empty array, if there is no topology singleton configured in cib
stanzas = doc.root.elements['configuration/fencing-topology'] rescue nil
return [] if stanzas.nil?
# otherwise, parse cib for existing topology singleton and return it as provider instance
stanzas.each_element do |e|
items = e.attributes
line = { :fence_primitives => items['devices'], :node => items['target'], :index => items['index'] }
primitives = line[:fence_primitives].split(',')
if primitives.length > 1 then
agents = []
primitives.each { |primitive| agents << (/^stonith__([^__].+)__.*$/.match(primitive)[1] rescue 'primitive_name_parse_error') }
else
agents = [(/^stonith__([^__].+)__.*$/.match(primitives[0])[1] rescue 'primitive_name_parse_error')]
end
nodes.push(line[:node]) unless nodes.include?(line[:node])
fence_topology[line[:node]] = {} if fence_topology[line[:node]].nil?
fence_topology[line[:node]][line[:index]] = agents
end
property_instance = {
:name => 'myfencetopo',
:ensure => :present,
:fence_topology => fence_topology,
:nodes => nodes,
:provider => self.name
}
[new(property_instance)]
end
# SET
def nodes=(should)
@property_hash[:nodes] = should
end
def fence_topology=(should)
@property_hash[:fence_topology] = should
end
#GET
def nodes
@property_hash[:nodes]
end
def fence_topology
@property_hash[:fence_topology]
end
def create
@property_hash = {
:name => @resource[:name],
:ensure => :present,
:fence_topology => @resource[:fence_topology],
:nodes => @resource[:nodes]
}
@property_hash[:cib] = @resource[:cib] if ! @resource[:cib].nil?
end
def destroy
debug("Removing fencing topology")
env = {}
env["CIB_shadow"] = @resource[:cib].to_s if !@resource[:cib].nil?
commands_to_exec = ''
commands_to_exec << "#{command(:cibadmin)} --scope fencing-topology --delete-all --force --xpath //fencing-level 2>&1"
commands_to_exec << "\n"
commands_to_exec << "#{command(:cibadmin)} --delete --xml-text '<fencing-topology/>' 2>&1"
exec_withenv(commands_to_exec, env)
@property_hash.clear
end
def exists?
self.class.block_until_ready
debug(@property_hash.inspect)
env = {}
env["CIB_shadow"] = @resource[:cib].to_s if !@resource[:cib].nil?
commands_to_exec = "#{command(:cibadmin)} --query --scope fencing-topology"
exec_withenv(commands_to_exec, env) == 0
end
def flush
unless @property_hash.empty? or self.class.instances != []
self.class.block_until_ready
args = ''
@property_hash[:nodes].each do |node|
# extract node's short name from its fqdn, if defined
shortname = /^([^.]+)\..*$/.match(node)[1] rescue node
pos = 1
# start crafting node's topology from position #1,
# nodes' topology lines should be separated by whitespace
line = " #{node}: "
@property_hash[:fence_topology][node].sort.each do |index, primitives|
primitives.each do |primitive|
line += case pos
# first primitive should be put after its node fqdn
when 1 then "stonith__#{primitive}__#{shortname}"
# all primitives with the same indexes should be grouped together, coma separated
when index then ",stonith__#{primitive}__#{shortname}"
# all groups with different indexes should be separated by whitespace
else " stonith__#{primitive}__#{shortname}"
end
pos = index if index != pos
end
end
args += line
# proceed to the next node
end
# send topology lines crafted for all nodes to crm
env = {}
env["CIB_shadow"] = @resource[:cib].to_s if !@resource[:cib].nil?
command_to_exec = "#{command(:crm)} --force configure fencing_topology#{args} 2>&1"
exec_withenv(command_to_exec, env)
end
end
end

View File

@ -0,0 +1,55 @@
module Puppet
newtype(:cs_fencetopo) do
@doc = "Type for manipulating corosync/pacemaker configuration for fencing topology.
More information on fencing topologies can be found here:
* http://clusterlabs.org/wiki/Fencing_topology
"
ensurable
newparam(:name) do
desc "Fencing topology name reference."
isnamevar
end
newparam(:cib) do
desc "Corosync applies its configuration immediately. Using a CIB allows
you to group multiple primitives and relationships to be applied at
once. This can be necessary to insert complex configurations into
Corosync correctly.
This paramater sets the CIB this order should be created in. A
cs_shadow resource with a title of the same name as this value should
also be added to your manifest."
end
newproperty(:nodes, :array_matching=>:all) do
desc "An array with cluster nodes' fqdns"
isrequired
end
newproperty(:fence_topology) do
desc "A hash with predefined fence topology."
isrequired
validate do |fence_topology|
raise Puppet::Error, "Puppet::Type::Cs_FenceTopo: fencing topology entries must be a hashes." unless fence_topology.is_a? Hash
end
defaultto Hash.new
end
autorequire(:service) do
[ 'corosync' ]
end
autorequire(:cs_shadow) do
autos = []
if @parameters[:cib]
autos << @parameters[:cib].value
end
autos
end
end
end

View File

@ -0,0 +1,44 @@
# == Define: pcs_fencing::fencing
#
# Configure STONITH resources for corosync/pacemaker.
#
define pcs_fencing::fencing (
$agent_type,
$parameters = false,
$operations = false,
$meta = false,
){
$res_name = "stonith__${title}__${::hostname}"
cs_resource { $res_name:
ensure => present,
provided_by => 'pacemaker',
primitive_class => 'stonith',
primitive_type => $agent_type,
parameters => $parameters,
operations => $operations,
metadata => $meta,
}
cs_location {"location__prohibit__${res_name}":
node_name => $::pacemaker_hostname,
node_score => '-INFINITY',
primitive => $res_name,
}
cs_location {"location__allow__${res_name}":
primitive => $res_name,
rules => [
{
'score' => '100',
'boolean' => '',
'expressions' => [
{'attribute'=>"#uname",'operation'=>'ne','value'=>$::pacemaker_hostname},
],
},
],
}
Cs_resource[$res_name] ->
Cs_location<||>
}

View File

@ -0,0 +1,41 @@
# Creates fencing primitives and topology for given nodes.
# Assumes all nodes have the same OS installed
#
class pcs_fencing::fencing_primitives (
$fence_primitives,
$fence_topology,
$nodes,
) {
case $::osfamily {
'RedHat': {
$names = filter_hash($nodes, 'fqdn')
}
'Debian': {
$names = filter_hash($nodes, 'name')
}
default: {
fail("Unsupported osfamily: ${::osfamily} operatingsystem: ${::operatingsystem}, module ${module_name} only support osfamily RedHat and Debian")
}
}
anchor {'Fencing primitives start':}
anchor {'Fencing primitives end':}
create_resources('::pcs_fencing::fencing', $fence_primitives)
cs_fencetopo { 'fencing_topology':
ensure => present,
fence_topology => $fence_topology,
nodes => $names,
}
cs_property { 'stonith-enabled': value => 'true' }
cs_property { 'cluster-recheck-interval': value => '3min' }
package {'fence-agents':}
Anchor['Fencing primitives start'] ->
Package['fence-agents'] ->
Pcs_fencing::Fencing<||> ->
Cs_fencetopo['fencing_topology'] ->
Cs_property<||> ->
Anchor['Fencing primitives end']
}

View File

@ -0,0 +1,36 @@
{
"name": "pcs_fencing",
"version": "6.1.0",
"author": "Bogdan Dobrelya <bdobrelia@mirantis.com>",
"summary": "Puppet Pacemaker fencing Module for Fuel fencing plugin",
"license": "Apache License 2.0",
"source": "git://github.com/bogdando/pcs_fencing.git",
"project_page": "none",
"issues_url": "none",
"requirements": [
{ "name": "pe","version_requirement": "3.x" },
{ "name": "puppet","version_requirement": "3.x" }
],
"operatingsystem_support": [
{
"operatingsystem": "Debian",
"operatingsystemrelease": ["7"]
},
{
"operatingsystem": "Fedora",
"operatingsystemrelease": ["20"]
},
{
"operatingsystem": "RedHat",
"operatingsystemrelease": ["6.5","7"]
},
{
"operatingsystem": "Ubuntu",
"operatingsystemrelease": ["12.04","14.04"]
}
],
"description": "Puppet module for configuring Pacemaker HA fencing in Fuel plugin",
"dependencies": [
{ "name":"puppetlabs/stdlib","version_requirement": "4.x" }
]
}

View File

@ -0,0 +1,66 @@
require 'spec_helper'
describe 'pcs_fencing::fencing_primitives' do
let :params do
{
:fence_primitives => {
'ipmi_off' => {
'agent_type' => 'fence_ipmilan',
'operations' => false,
'meta' => false,
'parameters' => false
}
},
:fence_topology => {
'node-1.foo.bar' => {
'1' => [ 'ipmi_off' ]
}
},
:nodes => [
{
'fqdn' => 'node-1.foo.bar',
'name' => 'node-1',
'role' => 'primary-controller'
}
]
}
end
let(:names) { [ 'node-1.foo.bar' ] }
let(:facts) {{ :osfamily => 'RedHat' }}
context 'then configuring fencing' do
it 'should install fence-agents' do
should contain_package('fence-agents')
end
it 'should contain its class' do
should contain_class('pcs_fencing::fencing_primitives').with(params)
end
it 'should create fencing primitives' do
should contain_pcs_fencing__fencing('ipmi_off').with(
params[:fence_primitives]['ipmi_off']
)
end
it 'should enable fencing' do
should contain_cs_property('stonith-enabled')
end
it 'should update cluster recheck interval' do
should contain_cs_property('cluster-recheck-interval')
end
it 'should create a topology' do
should contain_cs_fencetopo('fencing_topology').with(
{
:ensure => 'present',
:fence_topology => params[:fence_topology],
:nodes => names,
}
)
end
end
end

View File

@ -0,0 +1,87 @@
require 'spec_helper'
describe 'pcs_fencing::fencing', :type => :define do
let (:title) { 'virsh_off' }
let (:node) { 'node-1' }
let (:res_name) { "stonith__#{title}__#{node}" }
let :params do
{
:agent_type => 'fence_virsh',
:parameters => false,
:operations => false,
:meta => false
}
end
let :primitive_params do
{
:ensure => 'present',
:provided_by => 'pacemaker',
:primitive_class => 'stonith',
:primitive_type => params[:agent_type],
:parameters => params[:parameters],
:operations => params[:operations],
:metadata => params[:meta]
}
end
let :location_prohibit_params do
{
:node_name => node,
:score => '-INFINITY',
:primitive => res_name
}
end
let :location_allow_params do
{
:primitive => res_name,
:rules => [
{
'score' => '100',
'boolean' => '',
'expressions' => [
{'attribute'=>"#uname",'operation'=>'ne','value'=>node}
]
}
]
}
end
let(:facts) {{ :osfamily => 'Debian' }}
let(:facts) {{ :pacemaker_hostname => node }}
context 'then configuring STONITH primitive' do
it 'should contain its definition' do
should contain_pcs_fencing__fencing(title).with(params)
end
it 'should create a pacemaker primitive' do
should contain_cs_resource(res_name).with(
{
'ensure' => primitive_params[:ensure],
'primitive_class' => primitive_params[:primitive_class],
'primitive_type' => primitive_params[:primitive_type],
'provided_by' => primitive_params[:provided_by],
'parameters' => primitive_params[:parameters],
'operations' => primitive_params[:operations],
'metadata' => primitive_params[:metadata]
}
)
end
it 'should create a prohibit location' do
should contain_cs_location("location__prohibit__#{res_name}").with(
{
'node_name' => location_prohibit_params[:node_name],
'node_score' => location_prohibit_params[:score],
'primitive' => location_prohibit_params[:primitive]
}
)
end
it 'should create an allow location' do
should contain_cs_location("location__allow__#{res_name}").with(
{
'primitive' => location_allow_params[:primitive],
'rules' => location_allow_params[:rules]
}
)
end
end
end

View File

@ -0,0 +1,60 @@
<cib epoch="521" num_updates="0" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.5" update-origin="ubuntu-1" update-client="crmd" cib-last-written="Mon Jan 28 20:34:13 2013" have-quorum="1" dc-uuid="ubuntu-1">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c"/>
</cluster_property_set>
</crm_config>
<nodes>
<node id="ubuntu-1" type="normal" uname="ubuntu-1"/>
<node id="ubuntu-2" type="normal" uname="ubuntu-2"/>
<node id="vvk-workstation" type="normal" uname="vvk-workstation"/>
</nodes>
<resources>
<primitive class="ocf" id="bar" provider="pacemaker" type="Dummy">
<operations>
<op id="bar-monitor-20" interval="20" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="foo" provider="pacemaker" type="Dummy"/>
<clone id="clone_blort">
<primitive class="ocf" id="blort" provider="pacemaker" type="Dummy">
<operations>
<op id="blort-start-0" interval="0" name="start" timeout="20"/>
<op id="blort-monitor-20" interval="20" name="monitor"/>
</operations>
</primitive>
</clone>
<group id="mygroup">
<primitive class="ocf" id="baz_1" provider="pacemaker" type="Dummy"/>
<primitive class="ocf" id="baz_2" provider="pacemaker" type="Dummy"/>
</group>
</resources>
<constraints>
<rsc_order first="foo" id="foo-before-bar" score="INFINITY" then="bar"/>
<rsc_colocation id="foo-with-bar" rsc="foo" score="INFINITY" with-rsc="bar"/>
<rsc_location id="l_11" rsc="master_bar">
<rule id="l_11-rule" score="INFINITY">
<expression attribute="#uname" id="l_11-expression" operation="ne" value="ubuntu-1"/>
<date_expression id="l_11-expression-0" operation="date_spec">
<date_spec hours="10" id="l_11-date_spec" weeks="5"/>
</date_expression>
<date_expression id="l_11-expression-1" operation="in_range" start="20121212" end="20131212" />
<date_expression id="l_11-expression-2" operation="gt" start="20121212"/>
<date_expression id="l_11-expression-3" operation="lt" end="20131212" />
<date_expression id="l_11-expression-4" operation="in_range" end="" start="20121212">
<duration id="l_11-duration" years="10"/>
</date_expression>
</rule>
</rsc_location>
<rsc_location id="l_12" rsc="master_bar" node="ubuntu-1" score="INFINITY"/>
</constraints>
<fencing-topology>
<fencing-level devices="stonith__ipmi_reset__node-1" id="fencing" index="1" target="node-1.test.local"/>
<fencing-level devices="stonith__psu_off__node-1,stonith__psu_on__node-1" id="fencing-0" index="2" target="node-1.test.local"/>
<fencing-level devices="stonith__ilo_reset__node-2" id="fencing-1" index="1" target="node-2.test.local"/>
<fencing-level devices="stonith__psu_snmp_off__node-2,stonith__psu_snmp_on__node-2" id="fencing-2" index="2" target="node-2.test.local"/>
</fencing-topology>
</configuration>
</cib>

View File

@ -0,0 +1,54 @@
<cib epoch="521" num_updates="0" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.5" update-origin="ubuntu-1" update-client="crmd" cib-last-written="Mon Jan 28 20:34:13 2013" have-quorum="1" dc-uuid="ubuntu-1">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c"/>
</cluster_property_set>
</crm_config>
<nodes>
<node id="ubuntu-1" type="normal" uname="ubuntu-1"/>
<node id="ubuntu-2" type="normal" uname="ubuntu-2"/>
<node id="vvk-workstation" type="normal" uname="vvk-workstation"/>
</nodes>
<resources>
<primitive class="ocf" id="bar" provider="pacemaker" type="Dummy">
<operations>
<op id="bar-monitor-20" interval="20" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="foo" provider="pacemaker" type="Dummy"/>
<clone id="clone_blort">
<primitive class="ocf" id="blort" provider="pacemaker" type="Dummy">
<operations>
<op id="blort-start-0" interval="0" name="start" timeout="20"/>
<op id="blort-monitor-20" interval="20" name="monitor"/>
</operations>
</primitive>
</clone>
<group id="mygroup">
<primitive class="ocf" id="baz_1" provider="pacemaker" type="Dummy"/>
<primitive class="ocf" id="baz_2" provider="pacemaker" type="Dummy"/>
</group>
</resources>
<constraints>
<rsc_order first="foo" id="foo-before-bar" score="INFINITY" then="bar"/>
<rsc_colocation id="foo-with-bar" rsc="foo" score="INFINITY" with-rsc="bar"/>
<rsc_location id="l_11" rsc="master_bar">
<rule id="l_11-rule" score="INFINITY">
<expression attribute="#uname" id="l_11-expression" operation="ne" value="ubuntu-1"/>
<date_expression id="l_11-expression-0" operation="date_spec">
<date_spec hours="10" id="l_11-date_spec" weeks="5"/>
</date_expression>
<date_expression id="l_11-expression-1" operation="in_range" start="20121212" end="20131212" />
<date_expression id="l_11-expression-2" operation="gt" start="20121212"/>
<date_expression id="l_11-expression-3" operation="lt" end="20131212" />
<date_expression id="l_11-expression-4" operation="in_range" end="" start="20121212">
<duration id="l_11-duration" years="10"/>
</date_expression>
</rule>
</rsc_location>
<rsc_location id="l_12" rsc="master_bar" node="ubuntu-1" score="INFINITY"/>
</constraints>
</configuration>
</cib>

View File

@ -0,0 +1 @@
require 'puppetlabs_spec_helper/module_spec_helper'

View File

@ -0,0 +1,150 @@
require 'spec_helper'
describe Puppet::Type.type(:cs_fencetopo).provider(:crm) do
$fence_topology = {
'node-1.test.local' => {
'1' => [
'ipmi_reset',
],
'2' => [
'psu_off','psu_on'
],
},
'node-2.test.local' => {
'1' => [
'ilo_reset',
],
'2' => [
'psu_snmp_off','psu_snmp_on'
],
},
}
$nodes = [ 'node-1.test.local', 'node-2.test.local' ]
$foo_topology = {
'node-1.foo-test.local' => {
'1' => [
'ipmi_off', 'dirac_off', 'ilo_off'
],
'2' => [
'psu1_off','psu2_off'
],
},
'node-2.foo-test.local' => {
'1' => [
'ipmi_off', 'dirac_off', 'ilo_off'
],
'2' => [
'psu1_off','psu2_off'
],
},
'node-3.foo-test.local' => {
'1' => [
'ipmi_off', 'dirac_off', 'ilo_off'
],
'2' => [
'psu1_off','psu2_off'
],
},
}
$foo_nodes = [ 'node-1.foo-test.local', 'node-2.foo-test.local', 'node-3.foo-test.local' ]
let(:resource) { Puppet::Type.type(:cs_fencetopo).new(
:name=>'myfencetopo',
:provider=>:crm,
:ensure=>:present,
:nodes=>$nodes,
:fence_topology=>$fence_topology) }
let(:provider) { resource.provider }
let(:instance) { provider.class.instances }
let(:foo_resource) { Puppet::Type.type(:cs_fencetopo).new(
:name=>'myfootopo',
:provider=>:crm,
:ensure=>:present,
:nodes=>$foo_nodes,
:fence_topology=>$foo_topology) }
let(:foo_provider) { foo_resource.provider }
let(:foo_instance) { foo_provider.class.instances }
describe "#create" do
before(:each) do
provider.class.stubs(:exec_withenv).returns(0)
end
it "should create topology singleton with corresponding nodes list and fence primitives" do
provider.class.stubs(:block_until_ready).returns(true)
provider.class.stubs(:instances).returns([])
provider.expects(:exec_withenv).with(' --force configure fencing_topology node-1.test.local: stonith__ipmi_reset__node-1 stonith__psu_off__node-1,stonith__psu_on__node-1 node-2.test.local: stonith__ilo_reset__node-2 stonith__psu_snmp_off__node-2,stonith__psu_snmp_on__node-2 2>&1', {})
provider.create
provider.flush
end
it "should not try to recreate the same topology (idempotency test)" do
provider.class.stubs(:block_until_ready).returns(true)
provider.class.stubs(:instances).returns([instance])
provider.create
provider.flush
instance.instance_eval{@property_hash}.should be_nil
end
it "should not create new topology, if one already exists (singleton test)" do
foo_provider.class.stubs(:block_until_ready).returns(true)
foo_provider.class.stubs(:instances).returns([instance])
foo_provider.create
foo_provider.flush
foo_instance.instance_eval{@property_hash}.should be_nil
end
end
describe "#destroy" do
it "should destroy topology singleton" do
expected = ''
expected << ' --scope fencing-topology --delete-all --force --xpath //fencing-level 2>&1'
expected << "\n"
expected << " --delete --xml-text '<fencing-topology/>' 2>&1"
provider.expects(:exec_withenv).with(expected, {})
provider.destroy
end
end
describe "#instances" do
it "should find topology singleton" do
provider.class.stubs(:block_until_ready).returns(true)
out=File.open(File.dirname(__FILE__) + '/../../../../fixtures/cib/cib.xml')
provider.class.stubs(:dump_cib).returns(out,nil)
expected = {:name=>"myfencetopo", :fence_topology=>$fence_topology, :nodes=>$nodes, :ensure=>:present, :provider=>:crm}
instance[0].instance_eval{@property_hash}.should eql(expected)
end
it "should not find topology singleton" do
provider.class.stubs(:block_until_ready).returns(true)
out=File.open(File.dirname(__FILE__) + '/../../../../fixtures/cib/cib_no_topo.xml')
provider.class.stubs(:dump_cib).returns(out,nil)
instance[0].instance_eval{@property_hash}.should be_nil
end
end
describe '#exists?' do
it 'checks if topology singleton exists' do
provider.class.stubs(:block_until_ready).returns(true)
out=File.open(File.dirname(__FILE__) + '/../../../../fixtures/cib/cib.xml')
provider.class.stubs(:dump_cib).returns(out,nil)
provider.class.stubs(:exec_withenv).with(' --query --scope fencing-topology', {}).returns(0)
provider.exists?.should be_true
end
it 'checks if topology singleton does not exist' do
provider.class.stubs(:block_until_ready).returns(true)
out=File.open(File.dirname(__FILE__) + '/../../../../fixtures/cib/cib_no_topo.xml')
provider.class.stubs(:dump_cib).returns(out,nil)
provider.class.stubs(:exec_withenv).with(' --query --scope fencing-topology', {}).returns(6)
provider.exists?.should be_false
end
end
end

View File

@ -0,0 +1,92 @@
require 'spec_helper'
describe Puppet::Type.type(:cs_fencetopo) do
subject do
Puppet::Type.type(:cs_fencetopo)
end
$fence_topology = {
'node-1.test.local' => {
'1' => [
'ipmi_reset',
],
'2' => [
'psu_off','psu_on'
],
},
'node-2.test.local' => {
'1' => [
'ilo_reset',
],
'2' => [
'psu_snmp_off','psu_snmp_on'
],
}
}
$nodes = [ 'node-1.test.local', 'node-2.test.local' ]
$foo_topology = {
'node-1.foo-test.local' => {
'1' => [
'ipmi_off', 'dirac_off', 'ilo_off'
],
'2' => [
'psu1_off','psu2_off'
],
},
'node-2.foo-test.local' => {
'1' => [
'ipmi_off', 'dirac_off', 'ilo_off'
],
'2' => [
'psu1_off','psu2_off'
],
},
'node-3.foo-test.local' => {
'1' => [
'ipmi_off', 'dirac_off', 'ilo_off'
],
'2' => [
'psu1_off','psu2_off'
],
},
}
$foo_nodes = [ 'node-1.foo-test.local', 'node-2.foo-test.local', 'node-3.foo-test.local' ]
it "should have a 'name' parameter" do
subject.new(:name => 'mock_resource')[:name].should == 'mock_resource'
end
describe "basic structure" do
it "should be able to create a singleton instance" do
provider_class = Puppet::Type::Cs_fencetopo.provider(Puppet::Type::Cs_fencetopo.providers[0])
Puppet::Type::Cs_fencetopo.expects(:defaultprovider).returns(provider_class)
subject.new(:name => "mock_resource").should_not be_nil
end
#it "should not be able to create other instances" do
#TODO verify if fencetopo has a singleton nature
#end
[:cib, :name ].each do |param|
it "should have a #{param} parameter" do
subject.validparameter?(param).should be_true
end
it "should have documentation for its #{param} parameter" do
subject.paramclass(param).doc.should be_instance_of(String)
end
end
[ :nodes, :fence_topology ].each do |prop|
it "should have a #{prop} property" do
subject.validproperty?(prop).should be_true
end
it "should have documentation for its #{prop} property" do
subject.propertybyname(prop).doc.should be_instance_of(String)
end
end
end
end

View File

@ -0,0 +1,24 @@
node default {
class { 'pcs_fencing::fencing_primitives':
fence_primitives => {
'ipmi_off' => {
'agent_type' => 'fence_ipmilan',
'operations' => false,
'meta' => false,
'parameters' => false,
},
},
fence_topology => {
'node-1' => {
'1' => [ 'ipmi_off' ],
}
},
nodes => [
{
'name' => 'node-1',
'fqdn' => 'node-1.foo.bar',
'role' => 'primary-controller',
},
],
}
}

7
environment_config.yaml Normal file
View File

@ -0,0 +1,7 @@
attributes:
fence_policy:
value: 'reboot'
label: 'Policy for HA fencing [disabled, reboot, poweroff]'
description: 'Pick a type of HA fencing policy'
weight: 25
type: "text"

25
metadata.yaml Normal file
View File

@ -0,0 +1,25 @@
# Plugin name
name: ha_fencing
title: HA fencing for Pacemaker cluster
# Plugin version
version: 6.0.0
# Description
description: Enables STONITH of the failed nodes in Corosync & Pacemaker cluster
# Required fuel version
fuel_version: ['6.0']
# The plugin is compatible with releases in the list
releases:
- os: ubuntu
version: 2014.2-6.0
mode: ['ha']
deployment_scripts_path: deployment_scripts/
repository_path: repositories/ubuntu
- os: centos
version: 2014.2-6.0
mode: ['ha']
deployment_scripts_path: deployment_scripts/
repository_path: repositories/centos
# Version of plugin package
package_version: '1.0.0'

26
pre_build_hook Executable file
View File

@ -0,0 +1,26 @@
#!/bin/bash
set -eux
ROOT="$(dirname `readlink -f $0`)"
MODULES="${ROOT}"/deployment_scripts/puppet/modules/
TMP_DIR="${ROOT}"/tmp
mkdir -p "${MODULES}"
mkdir -p "${TMP_DIR}"
#Puppetlabs/stdlib 4.5.0
REPO_PATH='https://github.com/puppetlabs/puppetlabs-stdlib/tarball/80f09623b63cf6946b5913b629911e2c49b5d1dd'
wget -qO- "${REPO_PATH}" | \
tar -C "${MODULES}" -zxvf - \
puppetlabs-puppetlabs-stdlib-80f0962 && \
mv "${MODULES}puppetlabs-puppetlabs-stdlib-80f0962" "${MODULES}stdlib"
#Fuel 5.1.1 puppet-corosync
REPO_PATH='https://github.com/stackforge/fuel-library/tarball/a3043477337b4a0a8fd166dc83d6cd5d504f5da8'
MODULES="${ROOT}"/deployment_scripts/puppet/modules/
wget -qO- "${REPO_PATH}" | \
tar -C "${MODULES}" --strip-components=3 -zxvf - \
stackforge-fuel-library-a304347/deployment/puppet/corosync

View File

View File

8
tasks.yaml Normal file
View File

@ -0,0 +1,8 @@
# Deployment is required for controllers
- role: ['controller']
stage: post_deployment
type: puppet
parameters:
puppet_manifest: puppet/manifests/site.pp
puppet_modules: puppet/modules
timeout: 720