Add upstream puppet modules

for parsing text config files by mapping to hash
and for boolean values normalization in types

Fuel-CI: disable
Change-Id: I9b0d71eea357e9e27141caacf307cd0e40b9f5df
Blueprint: refactor-l23-linux-bridges
This commit is contained in:
Sergey Vasilenko 2014-12-17 19:29:17 +03:00
parent 9b1948b210
commit 879910023c
12 changed files with 1324 additions and 0 deletions

View File

@ -0,0 +1 @@
Gemfile.local

View File

@ -0,0 +1,8 @@
---
language: ruby
script: "bundle exec rspec --color --format documentation"
notifications:
email: false
rvm:
- 1.9.3
- 1.8.7

View File

@ -0,0 +1,63 @@
CHANGELOG
=========
1.1.3
-----
2014-09-02
This is a backwards compatible bugfix release.
* Invoke super in self.initvars to initialize `@defaults`
Thanks to Igor Galić for his work on this release.
1.1.2
-----
2013-07-04
This is a backwards compatible bugfix release.
* Update permissions of built modules to be a+rX.
1.1.1
-----
2012-12-30
This is a backwards compatible bugfix release.
* (filemapper-#4) Add resource failure when in error state
Thanks to Reid Vandewiele for his contribution for this release.
1.1.0
-----
2012-12-07
This is a backwards compatible feature release.
* Add Apache 2.0 LICENSE
* Add Gemfile
* (filemapper-#3) Add `unlink_empty_files` attribute
* (maint) spec cleanup for readability
* (filemapper-#2) Add pre and post flush hook support
1.0.2
-----
This is a backwards compatible maintenance release.
* Update metadata to reference forge username
* Ensure implementing classes return a string from format_file
1.0.1
-----
This is a backwards compatible maintenance release.
* Remove call `#symbolize` method; said method was removed in Puppet 3.0.0
* Fail fast if an including class returns bad data from Provider.parse_file
* Don't try to fall back to `@resource.should` value for properties

View File

@ -0,0 +1,15 @@
source 'https://rubygems.org'
gem 'puppet', '>= 2.7.0'
gem 'facter', '>= 1.6.2'
group :test, :development do
gem 'yard', '~> 0.8.3'
gem 'redcarpet', '~> 2.3.0'
gem 'rspec', '~> 2.10.0'
gem 'mocha', '~> 0.10.5'
end
if File.exists? "#{__FILE__}.local"
eval(File.read("#{__FILE__}.local"), binding)
end

View File

@ -0,0 +1,9 @@
# A sample Guardfile
# More info at https://github.com/guard/guard#readme
guard 'rspec', :version => 2 do
watch(%r{^spec/.+_spec\.rb$})
watch(%r{^lib/(.+)\.rb$}) { |m| "spec/unit/#{m[1]}_spec.rb" }
watch('spec/spec_helper.rb') { "spec" }
end

View File

@ -0,0 +1,14 @@
Copyright 2012 Adrien Thebo
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@ -0,0 +1,10 @@
name 'adrien-filemapper'
version '1.1.3'
author 'Adrien Thebo <adrien@somethingsinistral.net>'
license 'Apache 2.0'
summary 'Puppet provider file manipulation extension'
description 'Developer extension for permitting complex manipulation of file contents as resources'
source 'https://github.com/adrienthebo/puppet-filemapper'
project_page 'https://github.com/adrienthebo/puppet-filemapper'

View File

@ -0,0 +1,252 @@
Puppet FileMapper
=================
Synopsis
--------
Map files to resources and back with this handy dandy mixin!
Documentation is available at [http://adrienthebo.github.com/puppet-filemapper/](http://adrienthebo.github.com/puppet-filemapper/)
Travis Test status: [![Build Status](https://travis-ci.org/adrienthebo/puppet-filemapper.png)](https://travis-ci.org/adrienthebo/puppet-filemapper)
Description
-----------
Things that are harder than they should be:
* Acquiring a pet monkey
* Getting anywhere in Los Angeles
* Understanding the ParsedFile provider
* Writing Puppet providers that directly manipulate files
The solution for this is to completely bypass parsing in any sort of base
provider, and delegate the role of parsing and generating to including classes.
You figure out how to parse and write the file, and this will do the rest.
Synopsis of implementation requirements
---------------------------------------
Providers using the Filemapper extension need to implement the following
methods.
### `self.target_files`
This should return an array of filenames specifying which files should be
prefetched.
### `self.parse_file(filename, file_contents)`
This should take two values, a string containing the file name, and a string
containing the contents of the file. It should return an array of hashes,
where each hash represents {property => value} pairs.
### `select_file`
This is a provider instance method. It should return a string containing the
filename that the provider should be flushed to.
### `self.format_file(filename, providers)`
This should take two values, a string containing the file name to be flushed,
and an array of providers that should be flushed to this file. It should return
a string containing the contents of the file to be written.
Synopsis of optional implementation hooks
-----------------------------------------
### `self.pre_flush_hook(filename)` and `self.post_flush_hook(filename)`
These methods can be implemented to add behavior right before and right after
filesystem operations. Both methods take a single argument, a string
containing the name of the file to be flushed.
If `self.pre_flush_hook` raises an exception, the flush will not occur and the
provider will be marked as failed and will refuse to perform any more flushes.
If some sort of critical error occurred, this can force the provider to error
out before it starts stomping on files.
`self.post_flush_hook` is guaranteed to run after any filesystem operations
occur. This can be used for recovery if something goes wrong during the flush.
If this method raises an exception, the provider will be marked as failed and
will refuse to perform any more flushes.
Removing empty files
--------------------
If a file is empty, it's often reasonable to just delete it. The Filemapper
mixin implements `attr_accessor :unlink_empty_files`. If that value is set to
true, then if `self.format_file` returns the empty string then the file will be
deleted from the file system.
How it works
------------
[transaction]: http://somethingsinistral.net/blog/reading-puppet-the-transaction/
The Filemapper extension takes advantage of hooks within the
[Transaction][transaction] to reduce the number of reads and writes needed to
perform operations.
### prefetching
When a catalog is being applied, providers can define the `prefetch` method to
load all resources before runtime. The Filemapper extension uses this method to
preemptively read all files that the provider requires, and generates and stores
the state of the requested resources. This means that if you have a few thousand
resources in 20 files, you only need to do 20 reads for the entire Puppet run.
### post-evaluation flushing
When resources are normally evaluated, each time a property is synchronized it's
expected that an action will be run right then. The Filemapper extension instead
records all the requested changes and defers operating on them. When the
resource is finished, it will be flushed, at which time all of the requested
changes will be applied in one pass. Given a resource with 10 properties, all of
which are out of sync, the file will be written only once. If no properties are
out of sync, the file will be untouched.
To ensure that the system state matches what Puppet thinks is going on, any file
that has changed resources will be re-written after each resource is flushed.
That means that if you have 20 resources out of sync, that file will have to be
written 20 times. While it's technically possible to write the file in a single
pass, this means that some resources will be applied either early or late, which
utterly smashes POLA.
### Use on the command line
The Filemapper extension implements the `instances` method, which means that you
can use the `puppet resource` command to interact with the associated provider
without having to perform a full blown Puppet run.
### Selecting files to load
In order to provide prefetching and `puppet resource` in a clean manner, the
Filemapper extension has to have a full list of what files to read. Implementing
classes need to implement the `target_files` method which returns a list of
files to read. The implementation is entirely up to the implementing class; it
can return a single file every time, such as "/etc/inittab", or it can generate
that information on the fly, by returning `Dir["/etc/sysconfig/network/ifcfg-*"]`.
Basically, files that will be used as a source of data can be as complex or
simple as you need.
### Writing back files
In a similar vein, resources can be written back to files in whatever method you
need. Implementing classes need to implement the *instance method* `#select_file`
so that when that resource is changed, the correct file is modified.
### Parsing
When parsing a file, the implementing class needs to implement the `parse_file`
method. It will get the name of the file being parsed as well as the contents.
It can parse this file in whatever manner needed, and should return an array of
any provider instances generated. If the file only contains a single provider
instance, then just wrap that instance in an array.
### Writing
Whenever a file is marked as dirty, that is a resource associated with that file
has changed, the `format_file` method will be called. The implementing class
needs to implement a method that takes the filename and an array of provider
instances associated with that file, and return a string. The method needs to
determine how that file should be written to disk and then return the contents.
This can be as complex as needed.
Under no conditions should implementing classes modify any files directly. No,
seriously, don't do it. The Filemapper extension uses the built in methods for
modifying files, which will back up changed files to the filebucket. This is for
your own safety, so if you bypass this then you are on your own.
### Storing state outside of resources
It's more or less expected that there will be no state outside of the provider
instances, but there are plenty of cases where this could be the case. For
instance, if one wanted to preserve the comments in a file but didn't directly
associate them with resource attributes, the `parse_file` method can store data
in an instance variable, such as `@comments = my_list_of_comments`. When
formatting the file, the implementing class can read the `@comments` variable
and re-add that data to the content that will be written back.
Basically, you can store whatever data you need in these methods and pass things
around to maintain more complex state.
Using this sort of operation of reading outside state, you can theoretically
have multiple Filemapper extensions that work on shared files. By communicating
the state between them, you can manage multiple different resources in one file.
**HOWEVER**, this will require careful communication, so don't take this sort of
thing lightly. However, I don't thing that anything else in Puppet can provide
this sort of behavior. YMMV.
### Why a mixin?
While the ParsedFile provider is supposed to be inherited, this class is a mixin
and needs to be included. This is done because the Filemapper extension only
*adds* behavior, and isn't really an object or entity in its own right. This way
you can use the Filemapper extension while inheriting from something like the
Puppet::Provider::Package provider.
The Backstory
-------------
Managing Unix-ish systems generally means dealing with one of two things:
1. Processes - starting them, stopping them, monitoring them, etc.
1. Files - Creating them, editing, deleting them, specifying permissions, etc.
Puppet has pretty good support in the provider layer for running commands, but
the file manipulation layer has been lacking. The long-standing approach for
manipulating files has been to select one of the following, and hope for the best.
### Shipping flat files to the client
Using the `File` resource to ship flat files is a really common solution, and
it's very easy. It also has the finesse of a brick thrown through a window.
There is very little customizability here, aside from the array notation for
[specifying the `source` field](http://docs.puppetlabs.com/references/latest/type.html#file).
### Using ERB templates to customize files
The File resource can also take a content field, to which you can pass the
output of a template. This allows more sophistication, but not much. It also
adds more of a burden to your master; template rendering happens on the master
and if you're doing really crazy number crunching then this pain will be
centralized.
### Using Augeas
Augeas is a very powerful tool that allows you to manipulate files, and the
`Augeas` type allows you to harness this inside of Puppet. However, it has a
rather byzantine syntax, and is dependent on lenses being available.
### Sed
I personally love sed, but sed a file configuration management tool is not.
### Using the ParsedFile provider
[parsedfile]: https://github.com/puppetlabs/puppet/blob/2.7.19/lib/puppet/provider/parsedfile.rb "Puppet 2.7.19 - ParsedFile provider"
Puppet has a provider extension called the [ParsedFile provider][parsedfile]
that's used to manipulate text like crontabs and so forth. It also uses a number
of advanced features in puppet, which makes it quite powerful. However, it's
incredibly complex, tightly coupled with the FileParsing utility language, has
tons of obscure and undocumented hooks that are the only way to do complex
operations, and is entirely record based which makes it unsuitable for managing
files that have complex structure. While it has basic support for managing
multiple files, *basic* is the indicative word.
- - -
The Filemapper extension has been designed as a lower level alternative
to the ParsedFile.
Examples
--------
[puppet-network]: https://github.com/adrienthebo/puppet-network
The Filemapper extension was largely extracted out of the [puppet-network][puppet-network]
module. That code base should display the weird edge cases that this extension
handles.

View File

@ -0,0 +1,322 @@
require 'puppet/util/filetype'
# Forward declaration
module PuppetX; end
module PuppetX::FileMapper
# Copy all desired resource properties into this resource for generation upon flush
#
# This method is necessary for the provider to be ensurable
def create
raise Puppet::Error, "#{self.class} is in an error state" if self.class.failed?
@resource.class.validproperties.each do |property|
if value = @resource.should(property)
@property_hash[property] = value
end
end
self.dirty!
end
# Use the prefetched status to determine of the resource exists.
#
# This method is necessary for the provider to be ensurable
#
# @return [TrueClass || FalseClass]
def exists?
@property_hash[:ensure] and @property_hash[:ensure] == :present
end
# Update the property hash to mark this resource as absent for flushing
#
# This method is necessary for the provider to be ensurable
def destroy
@property_hash[:ensure] = :absent
self.dirty!
end
# Mark the file associated with this resource as dirty
def dirty!
file = select_file
self.class.dirty_file! file
end
# When processing on this resource is complete, trigger a flush on the file
# that this resource belongs to.
def flush
self.class.flush_file(self.select_file)
end
def self.included(klass)
klass.extend PuppetX::FileMapper::ClassMethods
klass.mk_property_methods
klass.initvars
end
module ClassMethods
# @!attribute [rw] unlink_empty_files
# @return [TrueClass || FalseClass] Whether empty files will be removed
attr_accessor :unlink_empty_files
# @!attribute [rw] filetype
# @return [Symbol] The FileType to use when interacting with target files
attr_accessor :filetype
# @!attribute [r] mapped_files
# @return [Hash<filepath => Hash<:dirty => Bool, :filetype => Filetype>>]
# A data structure representing the file paths and filetypes backing this
# provider.
attr_reader :mapped_files
def initvars
super
@mapped_files = Hash.new {|h, k| h[k] = {}}
@unlink_empty_files = false
@filetype = :flat
@failed = false
@all_providers = []
end
def failed?
@failed
end
def failed!
@failed = true
end
# Register all provider instances with the class
#
# In order to flush all provider instances to a given file, we need to be
# able to track them all. When provider#flush is called and the file
# associated with that provider instance is dirty, the file needs to be
# flushed and all provider instances associated with that file will be
# passed to self.flush_file
def new(*args)
obj = super
@all_providers << obj
obj
end
# Returns all instances of the provider using this mixin.
#
# @return [Array<Puppet::Provider>]
def instances
provider_hashes = load_all_providers_from_disk
provider_hashes.map do |h|
h.merge!({:provider => self.name, :ensure => :present})
new(h)
end
rescue
# If something failed while loading instances, mark the provider class
# as failed and pass the exception along
@failed = true
raise
end
# Validate that the required methods are available.
#
# @raise Puppet::DevError if an expected method is unavailable
def validate_class!
required_class_hooks = [:target_files, :parse_file, :format_file]
required_instance_hooks = [:select_file]
required_class_hooks.each do |method|
raise Puppet::DevError, "#{self} has not implemented `self.#{method}`" unless self.respond_to? method
end
required_instance_hooks.each do |method|
raise Puppet::DevError, "#{self} has not implemented `##{method}`" unless self.method_defined? method
end
end
# Reads all files from disk and returns an array of hashes representing
# provider instances.
#
# @return [Array<Hash<String, Hash<Symbol, Object>>>]
# An array containing a set of hashes, keyed with a file path and values
# being a hash containg the state of the file and the filetype associated
# with it.
#
# @example
# IncludingProvider.load_all_providers_from_disk
# # => [
# # { "/path/to/file" => {
# # :dirty => false,
# # :filetype => #<Puppet::Util::FileTypeFlat:0x007fbf5b05ff10>,
# # },
# # { "/path/to/another/file" => {
# # :dirty => false,
# # :filetype => #<Puppet::Util::FileTypeFlat:0x007fbf5b05c108,
# # },
# #
#
def load_all_providers_from_disk
validate_class!
# Retrieve a list of files to fetch, and cache a copy of a filetype
# for each one
target_files.each do |file|
@mapped_files[file][:filetype] = Puppet::Util::FileType.filetype(self.filetype).new(file)
@mapped_files[file][:dirty] = false
end
# Read and parse each file.
provider_hashes = []
@mapped_files.each_pair do |filename, file_attrs|
arr = parse_file(filename, file_attrs[:filetype].read)
unless arr.is_a? Array
raise Puppet::DevError, "expected #{self}.parse_file to return an Array, got a #{arr.class}"
end
provider_hashes.concat arr
end
provider_hashes
end
# Match up all resources that have existing providers.
#
# Pass over all provider instances, and see if there is a resource with the
# same namevar as a provider instance. If such a resource exists, set the
# provider field of that resource to the existing provider.
#
# This is a hook method that will be called by Puppet::Transaction#prefetch
#
# @param [Hash<String, Puppet::Resource>] resources
def prefetch(resources = {})
# generate hash of {provider_name => provider}
providers = instances.inject({}) do |hash, instance|
hash[instance.name] = instance
hash
end
# For each prefetched resource, try to match it to a provider
resources.each_pair do |resource_name, resource|
if provider = providers[resource_name]
resource.provider = provider
end
end
end
# Create attr_accessors for properties and mark the provider as dirty on change.
def mk_property_methods
resource_type.validproperties.each do |attr|
attr = attr.intern if attr.respond_to? :intern and not attr.is_a? Symbol
# Generate the attr_reader method
define_method(attr) do
if @property_hash[attr].nil?
:absent
else
@property_hash[attr]
end
end
# Generate the attr_writer and have it mark the resource as dirty when called
define_method("#{attr}=") do |val|
@property_hash[attr] = val
self.dirty!
end
end
end
# Generate an array of providers that should be flushed to a specific file
#
# Only providers that should be present will be returned regardless of
# the containing file.
#
# @param [String] filename The name of the file to find providers for
#
# @return [Array<Puppet::Provider>]
def collect_providers_for_file(filename)
@all_providers.select do |provider|
provider.select_file == filename and provider.ensure == :present
end
end
def dirty_file!(filename)
@mapped_files[filename][:dirty] = true
end
# Flush provider instances associated with the given file and call any defined hooks
#
# If the provider is in a failure state, the provider class will refuse to
# flush any file, since we're in an unknown state.
#
# This method respects two method hooks: `pre_flush_hook` and `post_flush_hook`.
# These methods must accept one argument, the path of the file being flushed.
# `post_flush_hook` is guaranteed to be called after the flush has occurred.
#
# @param [String] filename The path of the file to be flushed
def flush_file(filename)
if failed?
err "#{self.name} is in an error state, refusing to flush file #{filename}"
return
end
if not @mapped_files[filename][:dirty]
Puppet.debug "#{self.name} was requested to flush the file #{filename}, but it was not marked as dirty - doing nothing."
else
# Collect all providers that should be present and pass them to the
# including class for formatting.
target_providers = collect_providers_for_file(filename)
file_contents = self.format_file(filename, target_providers)
unless file_contents.is_a? String
raise Puppet::DevError, "expected #{self}.format_file to return a String, got a #{file_contents.class}"
end
# Call the `pre_flush_hook` method if it's defined
pre_flush_hook(filename) if self.respond_to? :pre_flush_hook
begin
if file_contents.empty? and self.unlink_empty_files
remove_empty_file(filename)
else
perform_write(filename, file_contents)
end
ensure
post_flush_hook(filename) if self.respond_to? :post_flush_hook
end
end
rescue
# If something failed during the flush process, mark the provider as
# failed. There's not much we can do about any file that's already been
# flushed but we can stop smashing things.
@failed = true
raise
end
# We have a dirty file and the new contents ready, back up the file and perform the flush.
#
# @param [String] filename The destination filename
# @param [String] contents The new file contents
def perform_write(filename, contents)
@mapped_files[filename][:filetype] ||= Puppet::Util::FileType.filetype(self.filetype).new(filename)
filetype = @mapped_files[filename][:filetype]
filetype.backup if filetype.respond_to? :backup
filetype.write(contents)
end
# Back up and remove a file, if it exists
#
# @param [String] filename The file to remove
def remove_empty_file(filename)
if File.exist? filename
@mapped_files[filename][:filetype] ||= Puppet::Util::FileType.filetype(self.filetype).new(filename)
filetype = @mapped_files[filename][:filetype]
filetype.backup if filetype.respond_to? :backup
File.unlink(filename)
end
end
end
end

View File

@ -0,0 +1,2 @@
# In case some tool demands that at least one manifest exists, we add this.
# Nothing interesting happens here. TALK AMONGST YOURSELVES

View File

@ -0,0 +1,12 @@
require 'rubygems'
require 'rspec'
require 'puppet'
require 'mocha_standalone'
PROJECT_ROOT = File.expand_path(File.join(File.dirname(__FILE__), ".."))
$LOAD_PATH.unshift(File.join(PROJECT_ROOT, "lib"))
$LOAD_PATH.unshift(File.join(PROJECT_ROOT, "spec", "lib"))
RSpec.configure do |config|
config.mock_with :mocha
end

View File

@ -0,0 +1,616 @@
require 'spec_helper'
require 'puppetx/filemapper'
describe PuppetX::FileMapper do
before do
@ramtype = Puppet::Util::FileType.filetype(:ram)
@flattype = stub 'Class<FileType<Flat>>'
@crontype = stub 'Class<FileType<Crontab>>'
Puppet::Util::FileType.stubs(:filetype).with(:flat).returns @flattype
Puppet::Util::FileType.stubs(:filetype).with(:crontab).returns @crontype
end
after :each do
dummytype.defaultprovider = nil
end
let(:dummytype) do
Puppet::Type.newtype(:dummy) do
ensurable
newparam(:name, :namevar => true)
newparam(:dummy_param)
newproperty(:dummy_property)
end
end
let(:single_file_provider) do
dummytype.provide(:single) do
include PuppetX::FileMapper
def self.target_files; ['/single/file/provider']; end
def self.parse_file(filename, content)
[{:name => 'yay', :dummy_param => :bla, :dummy_property => 'baz'}]
end
def select_file; '/single/file/provider'; end
def self.format_file(filename, providers); 'flushback'; end
end
end
let(:multiple_file_provider) do
dummytype.provide(:multiple, :resource_type => dummytype) do
include PuppetX::FileMapper
def self.target_files; ['/multiple/file/provider-one', '/multiple/file/provider-two']; end
def self.parse_file(filename, content)
case filename
when '/multiple/file/provider-one' then [{:name => 'yay', :dummy_param => :bla, :dummy_property => 'baz'}]
when '/multiple/file/provider-two' then [{:name => 'whee', :dummy_param => :ohai, :dummy_property => 'wat'}]
end
end
def select_file; '/multiple/file/provider-flush'; end
def self.format_file(filename, providers); 'multiple flush content'; end
end
end
let(:params_yay) { {:name => 'yay', :dummy_param => :bla, :dummy_property => 'baz'} }
let(:params_whee) { {:name => 'whee', :dummy_param => :ohai, :dummy_property => 'wat'} }
let(:params_nope) { {:name => 'dead', :dummy_param => :nofoo, :dummy_property => 'sadprop'} }
after :each do
dummytype.provider_hash.clear
end
describe 'when included' do
describe 'after initilizing attributes' do
subject { dummytype.provide(:foo) { include PuppetX::FileMapper } }
its(:mapped_files) { should be_empty }
its(:unlink_empty_files) { should eq(false) }
its(:filetype) { should eq(:flat) }
it { should_not be_failed }
end
describe 'when generating attr_accessors' do
subject { multiple_file_provider.new(params_yay) }
describe 'for properties' do
it { should respond_to :dummy_property }
it { should respond_to :dummy_property= }
it { should respond_to :ensure }
it { should respond_to :ensure= }
end
describe 'for parameters' do
it { should_not respond_to :dummy_param }
it { should_not respond_to :dummy_param= }
end
end
end
describe 'when validating the class' do
describe "and it doesn't implement self.target_files" do
subject do
dummytype.provide(:incomplete) { include PuppetX::FileMapper }
end
it { expect { subject.validate_class! }.to raise_error Puppet::DevError, /self.target_files/ }
end
describe "and it doesn't implement self.parse_file" do
subject do
dummytype.provide(:incomplete) do
include PuppetX::FileMapper
def self.target_files; end
end
end
it { expect { subject.validate_class! }.to raise_error Puppet::DevError, /self.parse_file/}
end
describe "and it doesn't implement #select_file" do
subject do
dummytype.provide(:incomplete) do
include PuppetX::FileMapper
def self.target_files; end
def self.parse_file(filename, content); end
def self.format_file(filename, resources); 'foo'; end
end
end
it { expect { subject.validate_class! }.to raise_error Puppet::DevError, /#select_file/}
end
describe "and it doesn't implement self.format_file" do
subject do
dummytype.provide(:incomplete) do
include PuppetX::FileMapper
def self.target_files; end
def self.parse_file(filename, content); end
def select_file; '/single/file/provider'; end
end
end
it { expect { subject.validate_class! }.to raise_error Puppet::DevError, /self\.format_file/}
end
end
describe 'when reading' do
describe 'a single file' do
subject { single_file_provider }
it 'should generate a filetype for that file' do
@flattype.expects(:new).with('/single/file/provider').once.returns @ramtype.new('/single/file/provider')
subject.load_all_providers_from_disk
end
it 'should parse each file' do
stub_file = stub(:read => 'file contents')
@flattype.stubs(:new).with('/single/file/provider').once.returns stub_file
subject.expects(:parse_file).with('/single/file/provider', 'file contents').returns []
subject.load_all_providers_from_disk
end
it 'should return the generated array' do
@flattype.stubs(:new).with('/single/file/provider').once.returns @ramtype.new('/single/file/provider')
subject.load_all_providers_from_disk.should == [params_yay]
end
end
describe 'multiple files' do
subject { multiple_file_provider }
it 'should generate a filetype for each file' do
@flattype.expects(:new).with('/multiple/file/provider-one').once.returns(stub(:read => 'barbar'))
@flattype.expects(:new).with('/multiple/file/provider-two').once.returns(stub(:read => 'bazbaz'))
subject.load_all_providers_from_disk
end
describe 'when parsing' do
before do
@flattype.stubs(:new).with('/multiple/file/provider-one').once.returns(stub(:read => 'barbar'))
@flattype.stubs(:new).with('/multiple/file/provider-two').once.returns(stub(:read => 'bazbaz'))
end
it 'should parse each file' do
subject.expects(:parse_file).with('/multiple/file/provider-one', 'barbar').returns []
subject.expects(:parse_file).with('/multiple/file/provider-two', 'bazbaz').returns []
subject.load_all_providers_from_disk
end
it 'should return the generated array' do
data = subject.load_all_providers_from_disk
data.should be_include(params_yay)
data.should be_include(params_whee)
end
end
end
describe 'validating input' do
subject { multiple_file_provider }
before do
@flattype.stubs(:new).with('/multiple/file/provider-one').once.returns(stub(:read => 'barbar'))
@flattype.stubs(:new).with('/multiple/file/provider-two').once.returns(stub(:read => 'bazbaz'))
end
it 'should ensure that retrieved values are in the right format' do
subject.stubs(:parse_file).with('/multiple/file/provider-one', 'barbar').returns Hash.new
subject.stubs(:parse_file).with('/multiple/file/provider-two', 'bazbaz').returns Hash.new
expect { subject.load_all_providers_from_disk }.to raise_error Puppet::DevError, /expected.*to return an Array, got a Hash/
end
end
end
describe 'when generating instances' do
subject { multiple_file_provider }
before do
@flattype.stubs(:new).with('/multiple/file/provider-one').once.returns(stub(:read => 'barbar'))
@flattype.stubs(:new).with('/multiple/file/provider-two').once.returns(stub(:read => 'bazbaz'))
end
it 'should generate a provider instance from hashes' do
params_yay.merge!({:provider => subject.name})
params_whee.merge!({:provider => subject.name})
subject.expects(:new).with(params_yay.merge({:ensure => :present})).returns stub()
subject.expects(:new).with(params_whee.merge({:ensure => :present})).returns stub()
subject.instances
end
it 'should generate a provider instance for each hash' do
provs = subject.instances
provs.should have(2).items
provs.each { |prov| prov.should be_a_kind_of(Puppet::Provider)}
end
[
{:name => 'yay', :dummy_property => 'baz'},
{:name => 'whee', :dummy_property => 'wat'},
].each do |values|
it "should match hash values to provider properties for #{values[:name]}" do
provs = subject.instances
prov = provs.find {|prov| prov.name == values[:name]}
values.each_pair { |property, value| prov.send(property).should == value }
end
end
end
describe 'when prefetching' do
subject { multiple_file_provider }
let(:provider_yay) { subject.new(params_yay.merge({:provider => subject.name})) }
let(:provider_whee) { subject.new(params_whee.merge({:provider => subject.name})) }
before do
subject.stubs(:instances).returns [provider_yay, provider_whee]
end
let(:resources) do
[params_yay, params_whee, params_nope].inject({}) do |h, params|
h[params[:name]] = dummytype.new(params)
h
end
end
it "should update resources with existing providers" do
resources['yay'].expects(:provider=).with(provider_yay)
resources['whee'].expects(:provider=).with(provider_whee)
subject.prefetch(resources)
end
it "should not update resources that don't have providers" do
resources['dead'].expects(:provider=).never
subject.prefetch(resources)
end
end
describe 'on resource state change' do
subject { multiple_file_provider }
before do
dummytype.defaultprovider = subject
subject.any_instance.stubs(:resource_type).returns dummytype
end
describe 'from absent to present' do
let(:resource) { dummytype.new(:name => 'boom', :dummy_property => 'bang') }
it 'should mark the related file as dirty' do
subject.mapped_files['/multiple/file/provider-flush'][:dirty].should be_false
resource.property(:ensure).sync
subject.mapped_files['/multiple/file/provider-flush'][:dirty].should be_true
end
end
describe 'from present to absent' do
it 'should mark the related file as dirty' do
resource = dummytype.new(:name => 'boom', :dummy_property => 'bang', :ensure => :absent)
subject.mapped_files['/multiple/file/provider-flush'][:dirty].should be_false
resource.property(:ensure).sync
subject.mapped_files['/multiple/file/provider-flush'][:dirty].should be_true
end
end
describe 'on a property' do
let(:resource) { resource = dummytype.new(params_yay) }
before do
prov = subject.new(params_yay.merge({:ensure => :present}))
subject.stubs(:instances).returns [prov]
subject.prefetch({params_yay[:name] => resource})
end
it 'should mark the related file as dirty' do
subject.mapped_files['/multiple/file/provider-flush'][:dirty].should be_false
resource.property(:dummy_property).value = 'new value'
resource.property(:dummy_property).sync
subject.mapped_files['/multiple/file/provider-flush'][:dirty].should be_true
end
end
describe 'on a parameter' do
let(:resource) { resource = dummytype.new(params_yay) }
before do
prov = subject.new(params_yay.merge({:ensure => :present}))
subject.stubs(:instances).returns [prov]
subject.prefetch({params_yay[:name] => resource})
end
it 'should not mark the related file as dirty' do
subject.mapped_files['/multiple/file/provider-flush'][:dirty].should be_false
resource.parameter(:dummy_param).value = 'new value'
resource.flush
subject.mapped_files['/multiple/file/provider-flush'][:dirty].should be_false
end
end
end
describe 'when determining whether to flush' do
subject { multiple_file_provider }
before do
dummytype.defaultprovider = subject
subject.any_instance.stubs(:resource_type).returns dummytype
end
let(:resource) { resource = dummytype.new(params_yay) }
it 'should refuse to flush if the provider is in a failed state' do
subject.dirty_file!('/multiple/file/provider-flush')
subject.failed!
subject.expects(:collect_resources_for_provider).never
resource.flush
end
it 'should use the provider instance method `select_file` to locate the destination file' do
resource.provider.expects(:select_file).returns '/multiple/file/provider-flush'
resource.property(:dummy_property).value = 'zoom'
resource.property(:dummy_property).sync
end
it 'should trigger the class dirty_file! method' do
subject.expects(:dirty_file!).with('/multiple/file/provider-flush')
resource.property(:dummy_property).value = 'zoom'
resource.property(:dummy_property).sync
end
end
describe 'when flushing' do
subject { multiple_file_provider }
let(:newtype) { @ramtype.new('/multiple/file/provider-flush') }
let(:resource) { resource = dummytype.new(params_yay) }
before { newtype.stubs(:backup) }
it 'should forward provider#flush to the class' do
subject.expects(:flush_file).with('/multiple/file/provider-flush')
resource.flush
end
it 'should generate filetypes for new files' do
subject.dirty_file!('/multiple/file/provider-flush')
@flattype.expects(:new).with('/multiple/file/provider-flush').returns newtype
resource.flush
end
it 'should use existing filetypes for existing files' do
stub_filetype = stub()
stub_filetype.expects(:backup)
stub_filetype.expects(:write)
subject.dirty_file!('/multiple/file/provider-flush')
subject.mapped_files['/multiple/file/provider-flush'][:filetype] = stub_filetype
resource.flush
end
it 'should trigger a flush on dirty files' do
subject.dirty_file!('/multiple/file/provider-flush')
subject.expects(:perform_write).with('/multiple/file/provider-flush', 'multiple flush content')
resource.flush
end
it 'should not flush clean files' do
subject.expects(:perform_write).never
resource.flush
end
end
describe 'validating the file contents to flush' do
subject { multiple_file_provider }
before do
subject.stubs(:format_file).returns ['definitely', 'not', 'of', 'class', 'String']
subject.dirty_file!('/multiple/file/provider-flush')
end
it 'should raise an error if given an invalid value for file contents' do
subject.expects(:perform_write).with('/multiple/file/provider-flush', %w{invalid data}).never
expect { subject.flush_file('/multiple/file/provider-flush') }.to raise_error Puppet::DevError, /expected .* to return a String, got a Array/
end
end
describe 'when unlinking empty files' do
subject { multiple_file_provider }
let(:newtype) { @ramtype.new('/multiple/file/provider-flush') }
before do
subject.unlink_empty_files = true
newtype.stubs(:backup)
File.stubs(:unlink)
end
describe 'with empty file contents' do
before do
subject.dirty_file!('/multiple/file/provider-flush')
@flattype.stubs(:new).with('/multiple/file/provider-flush').returns newtype
File.stubs(:exist?).with('/multiple/file/provider-flush').returns true
subject.stubs(:format_file).returns ''
end
it 'should back up the file' do
newtype.expects(:backup)
subject.flush_file('/multiple/file/provider-flush')
end
it 'should remove the file' do
File.expects(:unlink).with('/multiple/file/provider-flush')
subject.flush_file('/multiple/file/provider-flush')
end
it 'should not write to the file' do
subject.expects(:perform_write).with('/multiple/file/provider-flush', '').never
subject.flush_file('/multiple/file/provider-flush')
end
end
describe 'with empty file contents and no destination file' do
before do
subject.dirty_file!('/multiple/file/provider-flush')
@flattype.stubs(:new).with('/multiple/file/provider-flush').returns newtype
File.stubs(:exist?).with('/multiple/file/provider-flush').returns false
subject.stubs(:format_file).returns ''
end
it 'should not try to remove the file' do
File.expects(:exist?).with('/multiple/file/provider-flush').returns false
File.expects(:unlink).never
subject.flush_file('/multiple/file/provider-flush')
end
it 'should not try to back up the file' do
newtype.expects(:backup).never
subject.flush_file('/multiple/file/provider-flush')
end
end
describe 'with a non-empty file' do
before do
subject.dirty_file!('/multiple/file/provider-flush')
@flattype.stubs(:new).with('/multiple/file/provider-flush').returns newtype
File.stubs(:exist?).with('/multiple/file/provider-flush').returns true
subject.stubs(:format_file).returns 'not empty'
end
it 'should not remove the file' do
File.expects(:unlink).never
subject.flush_file('/multiple/file/provider-flush')
end
end
end
describe 'when using an alternate filetype' do
subject { multiple_file_provider }
before do
subject.filetype = :crontab
end
it 'should assign that filetype to loaded files' do
@crontype.expects(:new).with('/multiple/file/provider-one').once.returns(stub(:read => 'barbar'))
@crontype.expects(:new).with('/multiple/file/provider-two').once.returns(stub(:read => 'bazbaz'))
subject.load_all_providers_from_disk
end
describe 'that does not implement backup' do
let(:resource) { resource = dummytype.new(params_yay) }
let(:stub_filetype) { stub() }
before :each do
subject.mapped_files['/multiple/file/provider-flush'][:filetype] = stub_filetype
subject.dirty_file!('/multiple/file/provider-flush')
stub_filetype.expects(:respond_to?).with(:backup).returns(false)
stub_filetype.expects(:backup).never
end
it 'should not call backup when writing files' do
stub_filetype.stubs(:write)
resource.flush
end
it 'should not call backup when unlinking files' do
subject.unlink_empty_files = true
subject.stubs(:format_file).returns ''
File.stubs(:exist?).with('/multiple/file/provider-flush').returns true
File.stubs(:unlink)
resource.flush
end
end
end
describe 'flush hooks' do
subject { multiple_file_provider }
before :each do
subject.dirty_file!('/multiple/file/provider-flush')
end
let(:newtype) { @ramtype.new('/multiple/file/provider-flush') }
it 'should be called in order' do
seq = sequence('flush')
subject.expects(:respond_to?).with(:pre_flush_hook).returns true
subject.expects(:respond_to?).with(:post_flush_hook).returns true
subject.expects(:pre_flush_hook).with('/multiple/file/provider-flush').in_sequence(seq)
subject.expects(:perform_write).with('/multiple/file/provider-flush', 'multiple flush content').in_sequence(seq)
subject.expects(:post_flush_hook).with('/multiple/file/provider-flush').in_sequence(seq)
subject.flush_file '/multiple/file/provider-flush'
end
it 'should call post_flush_hook even if an exception is raised' do
subject.stubs(:respond_to?).with(:pre_flush_hook).returns false
subject.stubs(:respond_to?).with(:post_flush_hook).returns true
subject.expects(:perform_write).with('/multiple/file/provider-flush', 'multiple flush content').raises RuntimeError
subject.expects(:post_flush_hook)
expect { subject.flush_file '/multiple/file/provider-flush' }.to raise_error RuntimeError
end
end
describe 'when formatting resources for flushing' do
let(:provider_class) { multiple_file_provider }
let(:new_resource) { dummytype.new(params_yay) }
let(:current_provider) { provider_class.new(params_whee) }
let(:current_resource) { dummytype.new(params_whee) }
let(:remove_provider) { provider_class.new(params_nope) }
let(:remove_resource) { dummytype.new(params_nope.merge({:ensure => :absent})) }
let(:unmanaged_provider) { provider_class.new(:name => 'unmanaged_resource', :dummy_param => 'zoom', :dummy_property => 'squid', :ensure => :present) }
let(:provider_stubs) { [current_provider, remove_provider, unmanaged_provider] }
let(:resource_stubs) { [new_resource, current_resource, remove_resource] }
before do
dummytype.defaultprovider = provider_class
provider_class.any_instance.stubs(:resource_type).returns dummytype
provider_class.stubs(:instances).returns provider_stubs
provider_class.prefetch(resource_stubs.inject({}) { |h, r| h[r.name] = r; h})
# Pretend that we're the resource harness and apply the ensure param
resource_stubs.each { |r| r.property(:ensure).sync }
end
it 'should collect all resources for a given file' do
provider_class.expects(:collect_providers_for_file).with('/multiple/file/provider-flush').returns []
provider_class.stubs(:perform_write)
provider_class.flush_file('/multiple/file/provider-flush')
end
describe 'and selecting' do
subject { multiple_file_provider.collect_providers_for_file('/multiple/file/provider-flush').map(&:name) }
describe 'present resources' do
it { should be_include 'yay' }
it { should be_include 'whee' }
it { should be_include 'unmanaged_resource' }
end
describe 'absent resources' do
it { should_not be_include 'nope' }
end
end
end
end