x
/
fuel-noop-fixtures
Code Issues Proposed changes

Update 

* Improve reporting
* More comments
* Port back file resource reporting features
* Port back data-based testing
* Improve filtering
* Improve docs

Change-Id: I218fce4e6012ce1171a5042eebc8ac95ba734c21
This commit is contained in:
Dmitry Ilyin 2016-02-24 19:53:55 +03:00
parent a76dc00a27
commit 79143636d0
34 changed files with 1296 additions and 525 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -12,7 +12,7 @@
.project
# Doc
doc/build
doc/_build
# Editors
*.swp

3
Gemfile
View File

@ -5,14 +5,13 @@ group :development, :test do
gem 'puppet-lint', '~> 0.3.2'
gem 'rspec-puppet', '~> 2.2.0'
gem 'rspec-puppet-utils', '~> 2.0.0'
gem 'openstack'
gem 'netaddr'
gem 'deep_merge'
gem 'pry'
gem 'simplecov'
gem 'puppet-spec'
gem 'colorize'
gem 'parallel'
gem 'openstack'
end
if ENV['PUPPET_GEM_VERSION']

60
doc/usage.rst
View File

@ -89,7 +89,20 @@ their own.::
tests/noop/noop_tests.sh -y neut_vlan.compute.ssl,neut_vlan.compute.nossl -s firewall/firewall,netconfig/netconfig -p
Filters can use used with a list of elements like this.
Filters can use used with a list of elements or can be given as a regular
expression or a list of regular expressions::
./noop_tests.sh -p -s 'master/.*'
Will filter all tasks in the *master* group.::
./noop_tests.sh -p -s '^ceph.*,^heat.*,glance/db_spec'
Will filter all *ceph*, *heat* tasks and glance/db_spec task individually.::
./noop_tests.sh -p -s '^ceph.*' -y ceph
All *ceph* related tasks only on Hiera files which have Ceph enabled.
Recreating globals yaml files
-----------------------------
@ -223,6 +236,49 @@ and properties actually have after all the catalog logic is processed. It's
very helpful when you are debugging a strange task behaviour or writing a spec
file.
The framework can also gather and report information about *File* resources
that are being installed by Puppet. Using *--save_file_resources* options
will dave the list of files that would be installed by the catalog and
description about their source or template. Using *--puppet_binary_files*
option will enable additional RSpec matcher that will fail if there are
files and, especially, binary files being installed. These ones should be
delivered by fuel packages.
Data-driven catalog tests
-------------------------
Usually the spec files try to repeat the logic found in the tested manifests,
receive the same set of resources and their parameters and compare them to
the set of resources found in the compiled catalog. Then the matchers are used
to check if the catalog contains what is expected from it to contain.
While this method works well in most cases it requires a lot of work and
extensive expertise in the tasks' domain to write a correct and comprehensive
set of spec for a task catalog. Specs also cannot detect if there are several
new resources or properties that have not been described in the spec file.
Data-driven tests can offer an alternative way to ensure that there are
no unwanted changes in the tasks catalogs. The idea behind them is building
catalogs in human-readable format before and after the changes are made. Then
these files can be compared and everything that have been changes will become
visible.
Using the **-V** options will save the current catalog to the *catalogs*
folder. These generated catalogs can either be commited to the repository
or be used as a temporary files that you can make before doing some changes
to the modules or manifests and removed later. Saved catalog files can be
very useful for a developer to just review the catalog contents and check
that every resource or class are receiving the correct property values.
You can also use **-v** option to enable automatic catalog checks. It should be
done after you have generated the initial versions and made some changes.
Running the tests with this option enabled will generate the catalogs again and
compare them to the saved version. If there are differences the test will be
failed and you will be able to locate the failed tasks. If you have catalogs
commited to the repository you can use the *diff* command to review what
changes to the catalog files have been introduced and commit the modified
catalogs if the changes are expected.
Using external environment variables and custom paths
-----------------------------------------------------
@ -241,7 +297,7 @@ Paths related:
library.
- **SPEC_TASK_DIR** Set the path to the task manifests folder.
- **SPEC_DEPLOYMENT_DIR** Set the path to the *deployment* directory. It's
actually use only to find the scripts to udate and reset modules.
actually use only to find the scripts to update and reset modules.
- **WORKSPACE** This variable is passed by the Jenkins jobs or will default to
the *workspece* folder. Currently used only to store the Ruby gems installed
by the *bundler* if *RVM* is not used.

12
doc/utility.rst
View File

@ -27,7 +27,6 @@ Output:::
-O, --report_only_tasks Show only tasks, skip individual examples
-r, --load_saved_reports Read saved report JSON files from the previous run and show tasks report
-R, --run_failed_tasks Run the task that have previously failed again
-M, --list_missing List all task manifests without a spec file
-x, --xunit_report Save report in xUnit format to a file
List options:
-Y, --list_hiera List all hiera yaml files
@ -40,7 +39,7 @@ Output:::
-f, --facts FACTS1,FACTS2 Run only these facts yamls. Example: "ubuntu.yaml,centos.yaml"
Debug options:
-c, --task_console Run PRY console
-C, --rspec_console Run PRY console in the
-C, --rspec_console Run PRY console in the RSpec process
-d, --task_debug Show framework debug messages
-D, --puppet_debug Show Puppet debug messages
--debug_log FILE Write all debug messages to this files
@ -56,7 +55,11 @@ Output:::
--dir_puppet_modules DIR Path to the puppet modules
Spec options:
-A, --catalog_show Show catalog content debug output
-V, --catalog_save Save catalog to the files instead of comparing them with the current catalogs
-v, --catalog_check Check the saved catalog against the current one
-a, --spec_status Show spec status blocks
--puppet_binary_files Check if Puppet installs binary files
--save_file_resources Save file resources list to a report file
Shortcut scripts
----------------
@ -78,5 +81,6 @@ can be used to perform some common actions.
the numbers of tasks in the library.
- **run_failed_tasks.sh** This wrapper will load the saved reports files from
the previous run and will try to run all the failed tasks again.
- **purge_reports.sh** Removes all report files.
- **purge_globals.sh** Removes all globals files.
- **purge_reports.sh** Removes all task report files.
- **purge_globals.sh** Removes all saved globals files.
- **purge_catalogs.sh** Removes all saves catalog files.

5
facts/master_centos6.yaml
View File

@ -19,3 +19,8 @@
:l23_os: 'centos6'
:os_package_type: 'rpm'
:os_service_default: '<SERVICE DEFAULT>'
:interfaces: docker0,eth0,lo
:ipaddress: 172.17.42.1
:ipaddress_docker0: 172.17.42.1
:ipaddress_eth0: 10.20.0.2
:ipaddress_lo: 127.0.0.1

5
facts/master_centos7.yaml
View File

@ -19,3 +19,8 @@
:l23_os: 'centos6'
:os_package_type: 'rpm'
:os_service_default: '<SERVICE DEFAULT>'
:interfaces: docker0,eth0,lo
:ipaddress: 172.17.42.1
:ipaddress_docker0: 172.17.42.1
:ipaddress_eth0: 10.20.0.2
:ipaddress_lo: 127.0.0.1

25
hiera/globals/neut_vlan.cinder-block-device.compute.yaml
View File

@ -389,10 +389,31 @@
- controller
sql_connection: "mysql://nova:wPFuGzlctG0bWM5i94Xsu1ub@192.168.0.2/nova?charset=utf8&read_timeout=60"
storage_hash:
iser: false
volumes_ceph: false
per_pool_pg_nums:
compute: 128
default_pg_num: 128
volumes: 128
images: 128
backups: 128
".rgw": 128
objects_ceph: false
ephemeral_ceph: false
volumes_lvm: true
volumes_block_device: true
images_vcenter: false
osd_pool_size: "2"
pg_num: 128
images_ceph: false
metadata:
group: storage
weight: 60
label: Storage
volume_backend_names:
volumes_ceph: false
volumes_lvm: false
volumes_block_device: false
volumes_lvm: LVM-backend
volumes_block_device: BDD-backend
swift_hash:
user_password: zGlRsQYt0T5Cg8Mmw5bEhtvg
syslog_hash:

147
hiera/master.yaml
View File

@ -1,76 +1,97 @@
# fqdn is needed for tests internals
fqdn: "fuel.domain.tld"
role: "master"
# run only these tasks with the master node manifest
test_tasks:
- master/astute-only
- master/cobbler-only
- master/dhcp-default-range
- master/hiera-for-container
- master/host-only
- master/host-upgrade
- master/keystone-only
- master/nailgun-only
- master/nginx-only
- master/ostf-only
- master/postgres-only
- master/puppetsync-only
- master/rabbitmq-only
- master/rsyslog-only
# Full astute.yaml for new-style manifests that load it via Hiera
"HOSTNAME": "nailgun"
---
"ADMIN_NETWORK":
"cidr": "10.20.0.0/24"
"dhcp_gateway": "10.109.0.1"
"dhcp_pool_end": "10.109.0.254"
"dhcp_pool_start": "10.109.0.3"
"interface": "enp0s3"
"ipaddress": "10.109.0.2"
"mac": "64:85:8e:4f:7c:2e"
"netmask": "255.255.255.0"
"size": "256"
"BOOTSTRAP":
"flavor": "ubuntu"
"http_proxy": ""
"https_proxy": ""
"repos":
- "name": "ubuntu"
"priority": !!null "null"
"section": "main universe multiverse"
"suite": "trusty"
"type": "deb"
"uri": "http://archive.ubuntu.com/ubuntu"
- "name": "ubuntu-updates"
"priority": !!null "null"
"section": "main universe multiverse"
"suite": "trusty-updates"
"type": "deb"
"uri": "http://archive.ubuntu.com/ubuntu"
- "name": "ubuntu-security"
"priority": !!null "null"
"section": "main universe multiverse"
"suite": "trusty-security"
"type": "deb"
"uri": "http://archive.ubuntu.com/ubuntu"
- "name": "mos"
"priority": !!int "1050"
"section": "main restricted"
"suite": "mos9.0"
"type": "deb"
"uri": "http://127.0.0.1:8080/ubuntu/x86_64"
- "name": "mos-updates"
"priority": !!int "1050"
"section": "main restricted"
"suite": "mos9.0-updates"
"type": "deb"
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0"
- "name": "mos-security"
"priority": !!int "1050"
"section": "main restricted"
"suite": "mos9.0-security"
"type": "deb"
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0"
- "name": "mos-holdback"
"priority": !!int "1100"
"section": "main restricted"
"suite": "mos9.0-holdback"
"type": "deb"
"uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0"
"DNS_DOMAIN": "test.domain.local"
"DNS_SEARCH": "test.domain.local"
"DNS_UPSTREAM": "10.109.0.1"
"FEATURE_GROUPS": []
"FUEL_ACCESS":
"password": "admin"
"user": "admin"
"HOSTNAME": "nailgun"
"NTP1": "0.fuel.pool.ntp.org"
"NTP2": "1.fuel.pool.ntp.org"
"NTP3": "2.fuel.pool.ntp.org"
"ADMIN_NETWORK":
"interface": "eth0"
"ipaddress": "10.109.0.2"
"netmask": "255.255.255.0"
"cidr": "10.20.0.0/24"
"size": "256"
"dhcp_gateway": "10.109.0.1"
"dhcp_pool_start": "10.109.0.3"
"dhcp_pool_end": "10.109.0.254"
"mac": "64:60:46:2e:5d:37"
"FUEL_ACCESS":
"user": "admin"
"password": "admin"
"BOOTSTRAP":
"MIRROR_DISTRO": "http://archive.ubuntu.com/ubuntu"
"MIRROR_MOS": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0"
"HTTP_PROXY": ""
"EXTRA_APT_REPOS": ""
"flavor": "centos"
"PRODUCTION": "docker"
"astute":
"password": "4OVylZOm"
"user": "naily"
"cobbler":
"password": "JuYBYPGy"
"user": "cobbler"
"keystone":
"admin_token": "ED6pa5u5"
"monitord_password": "rJ1GbiXg"
"monitord_user": "monitord"
"nailgun_password": "koftnFua"
"nailgun_user": "nailgun"
"ostf_password": "S1V7UGP3"
"ostf_user": "ostf"
"mcollective":
"password": "r3JATyRN"
"user": "mcollective"
"postgres":
"keystone_dbname": "keystone"
"nailgun_user": "nailgun"
"keystone_password": "ab32l0zA"
"keystone_user": "keystone"
"nailgun_password": "CYoU6RS6"
"ostf_user": "ostf"
"nailgun_dbname": "nailgun"
"keystone_password": "cpppakUb"
"ostf_dbname": "ostf"
"ostf_password": "TwfzylM7"
"cobbler":
"password": "0mMXE4t8"
"user": "cobbler"
"astute":
"password": "SwLCUx2H"
"user": "naily"
"keystone":
"nailgun_password": "Zk9zJ6Cy"
"nailgun_user": "nailgun"
"monitord_user": "monitord"
"nailgun_password": "MtC5S2TN"
"monitord_password": "9IR0gsgd"
"ostf_dbname": "ostf"
"ostf_password": "pErpTajp"
"ostf_user": "ostf"
"admin_token": "ZoyxrMO6"
"ostf_password": "7evzsSBv"
"mcollective":
"password": "PPMi1XT2"
"user": "mcollective"

2
hiera/neut_vlan.cinder-block-device.compute.yaml
View File

@ -121,7 +121,7 @@ access:
user: admin
tenant: admin
email: admin@localhost
storage_hash:
storage:
iser: false
volumes_ceph: false
per_pool_pg_nums:

2
lib/noop/manager.rb
View File

@ -1,7 +1,7 @@
require_relative 'config'
require_relative 'manager/library'
require_relative 'manager/options'
require_relative 'manager/actions'
require_relative 'manager/base'
require_relative 'manager/report'
require_relative 'manager/setup'
require_relative 'manager/xunit'

153
lib/noop/manager/actions.rb
View File

@ -1,153 +0,0 @@
require 'parallel'
module Noop
class Manager
def find_tasks_without_specs
task_file_names.reject do |manifest|
spec = Noop::Utils.convert_to_spec manifest
spec_file_names.include? spec
end
end
def debug(message)
Noop::Config.log.debug message
end
def output(message)
puts message
end
def parallel_run?
options[:parallel_run] and options[:parallel_run] > 0
end
def list_hiera_files
hiera_file_names.sort.each do |file_name_hiera|
next unless hiera_included? file_name_hiera
output file_name_hiera
end
exit(0)
end
def list_facts_files
facts_file_names.sort.each do |file_name_facts|
next unless facts_included? file_name_facts
output file_name_facts
end
exit(0)
end
def list_spec_files
spec_file_names.sort.each do |file_name_spec|
next unless spec_included? file_name_spec
output file_name_spec
end
exit(0)
end
def list_task_files
task_file_names.sort.each do |file_name_task|
output file_name_task
end
exit(0)
end
def run_all_tasks
Parallel.map(task_list, :in_threads => options[:parallel_run]) do |task|
task.run unless options[:pretend]
task
end
end
def run_failed_tasks
Parallel.map(task_list, :in_threads => options[:parallel_run]) do |task|
next if task.success?
task.status = :pending
task.run unless options[:pretend]
task
end
end
def load_task_reports
Parallel.map(task_list, :in_threads => options[:parallel_run]) do |task|
task.file_load_report_json
task.determine_task_status
task
end
end
def list_tasks_without_specs
tasks_without_specs = find_tasks_without_specs.to_a
if tasks_without_specs.any?
Noop::Utils.error "There are tasks without specs: #{tasks_without_specs.join ', '}"
end
end
def have_failed_tasks?
task_list.any? do |task|
task.failed?
end
end
def exit_with_error_code
exit 1 if have_failed_tasks?
exit 0
end
#########################################
def main
options
if ENV['SPEC_TASK_CONSOLE']
require 'pry'
binding.pry
exit(0)
end
if options[:list_missing]
list_tasks_without_specs
end
if options[:bundle_setup]
setup_bundle
end
if options[:update_librarian_puppet]
setup_library
end
if options[:self_check]
check_paths
show_filters
show_library
exit(0)
end
list_hiera_files if options[:list_hiera]
list_facts_files if options[:list_facts]
list_spec_files if options[:list_specs]
list_task_files if options[:list_tasks]
if options[:run_failed_tasks]
load_task_reports
run_failed_tasks
task_report
exit_with_error_code
end
if options[:load_saved_reports]
load_task_reports
task_report
save_xunit_report if options[:xunit_report]
exit_with_error_code
end
run_all_tasks
task_report
save_xunit_report if options[:xunit_report]
exit_with_error_code
end
end
end

209
lib/noop/manager/base.rb Normal file
View File

@ -0,0 +1,209 @@
module Noop
class Manager
def initialize
options
colorize_load_or_stub
parallel_load_or_stub
end
# Load the "colorize" gem or just
# stub the colorization method if the gem
# cannot be loaded and don't show colors
def colorize_load_or_stub
begin
require 'colorize'
rescue LoadError
debug 'Could not load "colorize" gem. Disabling colors.'
String.instance_eval do
define_method(:colorize) do |*args|
self
end
end
end
end
# Load the 'parallel' gem or just
# stub the parallel run function to run tasks one by one
def parallel_load_or_stub
begin
require 'parallel'
rescue LoadError
debug 'Could not load "parallel" gem. Disabling multi-process run.'
Object.const_set('Parallel', Module.new)
class << Parallel
def map(data, *args, &block)
data.map &block
end
end
end
end
# Write a debug message to the logger
# @return [void]
def debug(message)
Noop::Config.log.debug message
end
# Output a message to the console
# @return [void]
def output(message)
Noop::Utils.output message
end
# Output an error message to the log file
# and raise the exception
# @return [void]
def error(message)
Noop::Utils.error message
end
# Check if parallel run option is enabled
# @return [true,false]
def parallel_run?
options[:parallel_run] and options[:parallel_run] > 0
end
# Check if there are some filters defined
# @return [true,false]
def has_filters?
options[:filter_specs] or options[:filter_facts] or options[:filter_hiera] or options[:filter_examples]
end
# Output a list of all discovered Hiera file names taking filers into account
# @return [void]
def list_hiera_files
hiera_file_names.sort.each do |file_name_hiera|
next unless hiera_included? file_name_hiera
output file_name_hiera
end
exit(0)
end
# Output a list of all discovered facts file names taking filers into account
# @return [void]
def list_facts_files
facts_file_names.sort.each do |file_name_facts|
next unless facts_included? file_name_facts
output file_name_facts
end
exit(0)
end
# Output a list of all discovered spec file names taking filers into account
# @return [void]
def list_spec_files
spec_file_names.sort.each do |file_name_spec|
next unless spec_included? file_name_spec
output file_name_spec
end
exit(0)
end
# Output a list of all discovered task file names taking filers into account
# @return [void]
def list_task_files
task_file_names.sort.each do |file_name_task|
output file_name_task
end
exit(0)
end
# Try to run all discovered tasks in the task list, using
# parallel run if enabled
# Does not run tasks if :pretend option is given
# return [Array<Noop::Task>]
def run_all_tasks
Parallel.map(task_list, :in_threads => options[:parallel_run]) do |task|
task.run unless options[:pretend]
task
end
end
# Try to run anly those tasks that have failed status by reseting them
# to the :pending status first.
# Does not run tasks if :pretend option is given
# return [Array<Noop::Task>]
def run_failed_tasks
Parallel.map(task_list, :in_threads => options[:parallel_run]) do |task|
next if task.success?
task.status = :pending
task.run unless options[:pretend]
task
end
end
# Ask every task in the task list to load its report file and status
# from the previous run attempt
# return [Array<Noop::Task>]
def load_task_reports
Parallel.map(task_list, :in_threads => options[:parallel_run]) do |task|
task.file_load_report_json
task.determine_task_status
task
end
end
# Check if there are any failed tasks in the list.
# @return [true, false]
def have_failed_tasks?
task_list.any? do |task|
task.failed?
end
end
# Exit with error if there are failed tasks
# or without the error code if none.
def exit_with_error_code
exit 1 if have_failed_tasks?
exit 0
end
#########################################
def main
if ENV['SPEC_TASK_CONSOLE']
require 'pry'
binding.pry
exit(0)
end
if options[:bundle_setup]
setup_bundle
end
if options[:update_librarian_puppet]
setup_library
end
if options[:self_check]
self_check
exit(0)
end
list_hiera_files if options[:list_hiera]
list_facts_files if options[:list_facts]
list_spec_files if options[:list_specs]
list_task_files if options[:list_tasks]
if options[:run_failed_tasks]
load_task_reports
run_failed_tasks
tasks_report
exit_with_error_code
end
if options[:load_saved_reports]
load_task_reports
tasks_report
save_xunit_report if options[:xunit_report] and not options[:pretend]
exit_with_error_code
end
run_all_tasks
tasks_report
save_xunit_report if options[:xunit_report] and not options[:pretend]
exit_with_error_code
end
end
end

184
lib/noop/manager/library.rb
View File

@ -4,67 +4,80 @@ require 'set'
module Noop
class Manager
# Recursively find file in the folder
# @param root [String,Pathname]
# @return [Array<Pathname>]
def find_files(root, path_from=nil, &block)
root = Noop::Utils.convert_to_path root
files = []
begin
root.children.each do |path|
if path.file?
if block_given?
next unless block.call path
end
path = path.relative_path_from path_from if path_from
files << path
else
files << find_files(path, path_from, &block)
end
end
rescue
[]
end
files.flatten
end
# Scan the spec directory and gather the list of spec files
# @return [Array<Pathname>]
def spec_file_names
return @spec_file_names if @spec_file_names
@spec_file_names = []
Noop::Utils.error "No #{Noop::Config.dir_path_task_spec} directory!" unless Noop::Config.dir_path_task_spec.directory?
Noop::Config.dir_path_task_spec.find do |spec_file|
next unless spec_file.file?
next unless spec_file.to_s.end_with? '_spec.rb'
@spec_file_names << spec_file.relative_path_from(Noop::Config.dir_path_task_spec)
error "No #{Noop::Config.dir_path_task_spec} directory!" unless Noop::Config.dir_path_task_spec.directory?
@spec_file_names = find_files(Noop::Config.dir_path_task_spec, Noop::Config.dir_path_task_spec) do |file|
file.to_s.end_with? '_spec.rb'
end
@spec_file_names
end
# Scan the Hiera directory and gather the list of Hiera files
# @return [Array<Pathname>]
def hiera_file_names
return @hiera_file_names if @hiera_file_names
@hiera_file_names = []
Noop::Utils.error "No #{Noop::Config.dir_path_hiera} directory!" unless Noop::Config.dir_path_hiera.directory?
Noop::Config.dir_path_hiera.find do |hiera_name|
next unless hiera_name.file?
next unless hiera_name.to_s.end_with? '.yaml'
base_dir = hiera_name.dirname.basename
next if base_dir == Noop::Config.dir_name_hiera_override
next if base_dir == Noop::Config.dir_name_globals
@hiera_file_names << hiera_name.relative_path_from(Noop::Config.dir_path_hiera)
error "No #{Noop::Config.dir_path_hiera} directory!" unless Noop::Config.dir_path_hiera.directory?
@hiera_file_names = find_files(Noop::Config.dir_path_hiera, Noop::Config.dir_path_hiera) do |file|
file.to_s.end_with? '.yaml'
end
@hiera_file_names
end
# Scan the facts directory and gather the list of facts files
# @return [Array<Pathname>]
def facts_file_names
return @facts_file_names if @facts_file_names
@facts_file_names = []
Noop::Utils.error "No #{Noop::Config.dir_path_facts} directory!" unless Noop::Config.dir_path_facts.directory?
Noop::Config.dir_path_facts.find do |facts_name|
next unless facts_name.file?
next unless facts_name.to_s.end_with? '.yaml'
next if facts_name.dirname.basename == Noop::Config.dir_name_facts_override
@facts_file_names << facts_name.relative_path_from(Noop::Config.dir_path_facts)
error "No #{Noop::Config.dir_path_facts} directory!" unless Noop::Config.dir_path_facts.directory?
@facts_file_names = find_files(Noop::Config.dir_path_facts, Noop::Config.dir_path_facts) do |file|
file.to_s.end_with? '.yaml'
end
@facts_file_names
end
# Scan the tasks directory and gather the list of task files
# @return [Array<Pathname>]
def task_file_names
return @task_file_names if @task_file_names
@task_file_names = []
Noop::Utils.error "No #{Noop::Config.dir_path_tasks_local} directory!" unless Noop::Config.dir_path_tasks_local.directory?
Noop::Config.dir_path_tasks_local.find do |task_name|
next unless task_name.file?
next unless task_name.to_s.end_with? '.pp'
@task_file_names << task_name.relative_path_from(Noop::Config.dir_path_tasks_local)
error "No #{Noop::Config.dir_path_tasks_local} directory!" unless Noop::Config.dir_path_tasks_local.directory?
@task_file_names = find_files(Noop::Config.dir_path_tasks_local, Noop::Config.dir_path_tasks_local) do |file|
file.to_s.end_with? '.pp'
end
@task_file_names
end
# Read the task deployment graph metadata files in the library:
# Find all 'tasks.yaml' files in the puppet directory.
# Read them all to a Hash by their ids.
# Find all 'groups' records and resolve their 'tasks' reference
# by pointing referenced tasks to this group instead.
# @return [Hash<String => Hash>]
def task_graph_metadata
return @task_graph_metadata if @task_graph_metadata
@task_graph_metadata = {}
Noop::Utils.error "No #{Noop::Config.dir_path_modules_local} directory!" unless Noop::Config.dir_path_modules_local.directory?
error "No #{Noop::Config.dir_path_modules_local} directory!" unless Noop::Config.dir_path_modules_local.directory?
Noop::Config.dir_path_modules_local.find do |task_file|
next unless task_file.file?
next unless task_file.to_s.end_with? 'tasks.yaml'
@ -91,13 +104,17 @@ module Noop
@task_graph_metadata
end
# Try to determine the roles each spec should be run in using
# the deployment graph metadata. Take a list of groups or roles
# and form a set of them.
# @return [Hash<Pathname => Set>]
def assign_spec_to_roles
return @assign_spec_to_roles if @assign_spec_to_roles
@assign_spec_to_roles = {}
task_graph_metadata.values.each do |task_data|
roles = task_data['groups'] or task_data['roles']
next unless roles.is_a? Array
roles = (task_data['groups'] or task_data['roles'] or task_data['role'])
next unless roles
roles = [roles] unless roles.is_a? Array
file_path_manifest = task_data.fetch('parameters', {}).fetch('puppet_manifest', nil)
next unless file_path_manifest
file_path_manifest = Pathname.new file_path_manifest
@ -110,6 +127,9 @@ module Noop
@assign_spec_to_roles
end
# Try to determine the roles of each Hiera file.
# Take 'nodes' structure and find 'node_roles' of the current node their.
# Form a set of found values and add root 'role' value if found.
# @return [Hash<Pathname => Set>]
def assign_hiera_to_roles
return @assign_hiera_to_roles if @assign_hiera_to_roles
@ -138,19 +158,29 @@ module Noop
@assign_hiera_to_roles
end
# Determine Hiera files for each spec file by calculating
# the intersection between their roles sets.
# If the spec file contains '*' role it should be counted
# as all possible roles.
# @return [Hash<Pathname => Pathname]
def assign_spec_to_hiera
return @assign_spec_to_hiera if @assign_spec_to_hiera
@assign_spec_to_hiera = {}
assign_spec_to_roles.each do |file_name_spec, spec_roles|
hiera_files = assign_hiera_to_roles.select do |file_name_hiera, hiera_roles|
roles_intersection = hiera_roles & spec_roles
roles_intersection.any?
end.keys
assign_spec_to_roles.each do |file_name_spec, spec_roles_set|
if spec_roles_set.include? '*'
hiera_files = assign_hiera_to_roles.keys
else
hiera_files = assign_hiera_to_roles.select do |file_name_hiera, hiera_roles_set|
roles_intersection = hiera_roles_set & spec_roles_set
roles_intersection.any?
end.keys
end
@assign_spec_to_hiera[file_name_spec] = hiera_files if hiera_files.any?
end
@assign_spec_to_hiera
end
# Read all spec annotations metadata.
# @return [Hash<Pathname => Array>]
def spec_run_metadata
return @spec_run_metadata if @spec_run_metadata
@ -165,7 +195,9 @@ module Noop
@spec_run_metadata
end
# Parse a spec file to find annotation entries.
# @param [Pathname] task_spec
# @return [Hash]
def parse_spec_file(task_spec)
task_spec_metadata = {}
@ -212,6 +244,8 @@ module Noop
task_spec_metadata
end
# Split a space or comma separated list of yaml files
# and form an Array of the yaml file names.
# @return [Array<Pathname>]
def get_list_of_yamls(line)
line = line.split /\s*,\s*|\s+/
@ -222,6 +256,15 @@ module Noop
end
end
# Determine the list of run records for a spec file:
# Take a list of explicitly defined runs if present.
# Make product of allowed Hiera and facts yaml files to
# form more run records.
# Use the default facts file name if there is none
# is given in the annotation.
# Use the list of Hiera files determined by the intersection of
# deployment graph metadata and Hiera yaml contents using roles
# as a common data.
def get_spec_runs(file_name_spec)
file_name_spec = Noop::Utils.convert_to_path file_name_spec
metadata = spec_run_metadata.fetch file_name_spec, {}
@ -242,38 +285,55 @@ module Noop
runs
end
# Check if the given element matches this filter
# @param [Array<String>]
def filter_is_matched?(filter, element)
return true unless filter
filter = [filter] unless filter.is_a? Array
filter.any? do |expression|
expression = Regexp.new expression.to_s
expression =~ element.to_s
end
end
# Use filters to check if this spec file is included
# @return [true,false]
def spec_included?(spec)
filter = options[:filter_specs]
return true unless filter
filter = [filter] unless filter.is_a? Array
filter.include? spec
filter_is_matched? options[:filter_specs], spec
end
# Use filters to check if this facts file is included
# @return [true,false]
def facts_included?(facts)
filter = options[:filter_facts]
return true unless filter
filter = [filter] unless filter.is_a? Array
filter.include? facts
filter_is_matched? options[:filter_facts], facts
end
# Use filters to check if this Hiera file is included
# @return [true,false]
def hiera_included?(hiera)
filter = options[:filter_hiera]
return true unless filter
filter = [filter] unless filter.is_a? Array
filter.include? hiera
filter_is_matched? options[:filter_hiera], hiera
end
# Check if the globals spec should be skipped.
# It should not be skipped only if it's explicitly enabled in the filter.
# @return [true,false]
def skip_globals?(file_name_spec)
return false unless file_name_spec == Noop::Config.spec_name_globals
return true unless options[:filter_specs]
not spec_included? file_name_spec
end
# Check if the spec is disabled using the annotation
# @return [true,false]
def spec_is_disabled?(file_name_spec)
file_name_spec = Noop::Utils.convert_to_path file_name_spec
spec_run_metadata.fetch(file_name_spec, {}).fetch(:disable, false)
end
# Form the final list of Task objects that should be running.
# Take all discovered spec files, get run records for them,
# apply filters to exclude filtered records.
# @return [Array<Noop::Task>]
def task_list
return @task_list if @task_list
@task_list = []
@ -293,5 +353,25 @@ module Noop
@task_list
end
# Loop through all task files and find those that
# do not have a corresponding spec file present
# @return [Array<Pathname>]
def find_tasks_without_specs
task_file_names.reject do |manifest|
spec = Noop::Utils.convert_to_spec manifest
spec_file_names.include? spec
end
end
# Loop through all spec files and find those that
# do not have a corresponding task file present
# @return [Array<Pathname>]
def find_specs_without_tasks
spec_file_names.reject do |spec|
manifest = Noop::Utils.convert_to_manifest spec
task_file_names.include? manifest
end
end
end
end

50
lib/noop/manager/options.rb
View File

@ -3,6 +3,8 @@ require 'optparse'
module Noop
class Manager
# Parse the CLI options
# @return [Hash]
def options
return @options if @options
@options = {}
@ -41,9 +43,6 @@ module Noop
opts.on('-R', '--run_failed_tasks', 'Run the task that have previously failed again') do
@options[:run_failed_tasks] = true
end
opts.on('-M', '--list_missing', 'List all task manifests without a spec file') do
@options[:list_missing] = true
end
opts.on('-x', '--xunit_report', 'Save report in xUnit format to a file') do
@options[:xunit_report] = true
end
@ -64,13 +63,13 @@ module Noop
opts.separator 'Filter options:'
opts.on('-s', '--specs SPEC1,SPEC2', Array, 'Run only these spec files. Example: "hosts/hosts_spec.rb,apache/apache_spec.rb"') do |specs|
@options[:filter_specs] = import_specs_list specs
@options[:filter_specs] = specs
end
opts.on('-y', '--yamls YAML1,YAML2', Array, 'Run only these hiera yamls. Example: "controller.yaml,compute.yaml"') do |yamls|
@options[:filter_hiera] = import_yamls_list yamls
@options[:filter_hiera] = yamls
end
opts.on('-f', '--facts FACTS1,FACTS2', Array, 'Run only these facts yamls. Example: "ubuntu.yaml,centos.yaml"') do |yamls|
@options[:filter_facts] = import_yamls_list yamls
@options[:filter_facts] = yamls
end
# opts.on('-e', '--examples STR1,STR2', Array, 'Run only these spec examples. Example: "should compile"') do |examples|
# @options[:filter_examples] = examples
@ -80,7 +79,7 @@ module Noop
opts.on('-c', '--task_console', 'Run PRY console') do
ENV['SPEC_TASK_CONSOLE'] = 'YES'
end
opts.on('-C', '--rspec_console', 'Run PRY console in the ') do
opts.on('-C', '--rspec_console', 'Run PRY console in the RSpec process') do
ENV['SPEC_RSPEC_CONSOLE'] = 'YES'
end
opts.on('-d', '--task_debug', 'Show framework debug messages') do
@ -126,12 +125,12 @@ module Noop
opts.on('-A', '--catalog_show', 'Show catalog content debug output') do
ENV['SPEC_CATALOG_SHOW'] = 'YES'
end
# opts.on('--catalog_save', 'Save catalog to the files instead of comparing them with the current catalogs') do
# ENV['SPEC_CATALOG_CHECK'] = 'save'
# end
# opts.on('--catalog_check', 'Check the saved catalog against the current one') do
# ENV['SPEC_CATALOG_CHECK'] = 'check'
# end
opts.on('-V', '--catalog_save', 'Save catalog to the files instead of comparing them with the current catalogs') do
ENV['SPEC_CATALOG_CHECK'] = 'save'
end
opts.on('-v', '--catalog_check', 'Check the saved catalog against the current one') do
ENV['SPEC_CATALOG_CHECK'] = 'check'
end
# opts.on('--spec_generate', 'Generate specs for catalogs') do
# ENV['SPEC_SPEC_GENERATE'] = 'YES'
# end
@ -141,30 +140,19 @@ module Noop
# opts.on('--spec_coverage', 'Show spec coverage statistics') do
# ENV['SPEC_COVERAGE'] = 'YES'
# end
# opts.on('--puppet_binary_files', 'Check if Puppet installs binary files') do
# ENV['SPEC_PUPPET_BINARY_FILES'] = 'YES'
# end
# opts.on('--file_resources DIR', 'Save file resources to this dir') do |dir|
# ENV['SPEC_SAVE_FILE_RESOURCES'] = dir
# end
opts.on('--puppet_binary_files', 'Check if Puppet installs binary files') do
ENV['SPEC_PUPPET_BINARY_FILES'] = 'YES'
end
opts.on('--save_file_resources', 'Save file resources list to a report file') do
ENV['SPEC_SAVE_FILE_RESOURCES'] = 'YES'
end
end
optparse.parse!
@options
end
def import_specs_list(specs)
specs.map do |spec|
Noop::Utils.convert_to_spec spec
end
end
def import_yamls_list(yamls)
yamls.map do |yaml|
Noop::Utils.convert_to_yaml yaml
end
end
# Any default options values can be set here
def options_defaults(options)
options[:parallel_run] = 0
end

185
lib/noop/manager/report.rb
View File

@ -1,60 +1,12 @@
require 'erb'
require 'colorize'
require 'rexml/document'
# TODO: cli report should use data from tasks_report_structure instead of reimplementing it
module Noop
class Manager
STATUS_STRING_LENGTH = 8
def tasks_report_structure(tasks)
tasks_report = []
tasks.each do |task|
task_hash = {}
task_hash[:status] = task.status
task_hash[:name] = task.to_s
task_hash[:description] = task.description
task_hash[:spec] = task.file_name_spec.to_s
task_hash[:hiera] = task.file_name_hiera.to_s
task_hash[:facts] = task.file_name_facts.to_s
task_hash[:task] = task.file_name_manifest.to_s
task_hash[:examples] = []
if task.report.is_a? Hash
examples = task.report['examples']
next unless examples.is_a? Array
examples.each do |example|
example_hash = {}
example_hash[:file_path] = example['file_path']
example_hash[:line_number] = example['line_number']
example_hash[:description] = example['description']
example_hash[:status] = example['status']
example_hash[:run_time] = example['run_time']
example_hash[:pending_message] = example['pending_message']
exception_class = example.fetch('exception', {}).fetch('class', nil)
exception_message = example.fetch('exception', {}).fetch('message', nil)
next unless example_hash[:description] and example_hash[:status]
if exception_class and exception_message
example_hash[:exception_class] = exception_class
example_hash[:exception_message] = exception_message
end
task_hash[:examples] << example_hash
end
summary = task.report['summary']
task_hash[:example_count] = summary['example_count']
task_hash[:failure_count] = summary['failure_count']
task_hash[:pending_count] = summary['pending_count']
task_hash[:duration] = summary['duration']
end
tasks_report << task_hash
end
tasks_report
end
COLUMN_WIDTH = 8
# Output a status string for this task.
# Output examples to unless disables.
# @param task [Noop::Task]
def output_task_status(task)
return if options[:report_only_failed] and task.success?
line = task_status_string task
@ -65,6 +17,8 @@ module Noop
output_task_examples task unless options[:report_only_tasks]
end
# Output examples report for this task
# @param task [Noop::Task]
def output_task_examples(task)
return unless task.report.is_a? Hash
examples = task.report['examples']
@ -81,36 +35,47 @@ module Noop
end
end
# Get a colored string with status of this task
# @param task [Noop::Task]
# @return [String]
def task_status_string(task)
if task.pending?
'PENDING'.ljust(STATUS_STRING_LENGTH).colorize :blue
'PENDING'.ljust(COLUMN_WIDTH).colorize :blue
elsif task.success?
'SUCCESS'.ljust(STATUS_STRING_LENGTH).colorize :green
'SUCCESS'.ljust(COLUMN_WIDTH).colorize :green
elsif task.failed?
'FAILED'.ljust(STATUS_STRING_LENGTH).colorize :red
'FAILED'.ljust(COLUMN_WIDTH).colorize :red
else
task.status
end
end
# Colorize the example status string
# @param status [String]
# @return [String]
def example_status_string(status)
if status == 'passed'
status.ljust(STATUS_STRING_LENGTH).colorize :green
status.ljust(COLUMN_WIDTH).colorize :green
elsif status == 'failed'
status.ljust(STATUS_STRING_LENGTH).colorize :red
status.ljust(COLUMN_WIDTH).colorize :red
else
status.ljust(STATUS_STRING_LENGTH).colorize :blue
status.ljust(COLUMN_WIDTH).colorize :blue
end
end
# Return a string showing if the directory is present.
# @param directory [Pathname]
# @return [String]
def directory_check_status_string(directory)
if directory.directory?
'SUCCESS'.ljust(STATUS_STRING_LENGTH).colorize :green
'SUCCESS'.ljust(COLUMN_WIDTH).colorize :green
else
'FAILED'.ljust(STATUS_STRING_LENGTH).colorize :red
'FAILED'.ljust(COLUMN_WIDTH).colorize :red
end
end
# Find the length of the longest spec file name
# @return [Integer]
def max_length_spec
return @max_length_spec if @max_length_spec
@max_length_spec = task_list.map do |task|
@ -118,6 +83,8 @@ module Noop
end.max
end
# Find the length of the longest Hiera file name
# @return [Integer]
def max_length_hiera
return @max_length_hiera if @max_length_hiera
@max_length_hiera = task_list.map do |task|
@ -125,6 +92,8 @@ module Noop
end.max
end
# Find the length of the longest facts file name
# @return [Integer]
def max_length_facts
return @max_length_facts if @max_length_facts
@max_length_facts = task_list.map do |task|
@ -132,6 +101,7 @@ module Noop
end.max
end
# Output a status string with tasks count
def output_task_totals
count = {
:total => 0,
@ -139,24 +109,25 @@ module Noop
:pending => 0,
}
task_list.each do |task|
next unless task.is_a? Noop::Task
count[:pending] += 1 if task.pending?
count[:failed] += 1 if task.failed?
count[:total] += 1
end
output "Tasks: #{count[:total]} Failed: #{count[:failed]} Pending: #{count[:pending]}"
output_stats_string 'Tasks', count[:total], count[:failed], count[:pending]
end
# Output a status string with examples count
def output_examples_total
count = {
:total => 0,
:failed => 0,
:pending => 0,
}
task_list.each do |task|
examples = task.report['examples']
next unless examples.is_a? Array
examples.each do |example|
next unless task.is_a? Noop::Task
next unless task.has_report?
task.report['examples'].each do |example|
count[:total] += 1
if example['status'] == 'failed'
count[:failed] += 1
@ -165,50 +136,71 @@ module Noop
end
end
end
output "Examples: #{count[:total]} Failed: #{count[:failed]} Pending: #{count[:pending]}"
output_stats_string 'Examples', count[:total], count[:failed], count[:pending]
end
def task_report
# Format a status string of examples or tasks
def output_stats_string(name, total, failed, pending)
line = "#{name.to_s.ljust(COLUMN_WIDTH).colorize :yellow}"
line += " Total: #{total.to_s.ljust(COLUMN_WIDTH).colorize :green}"
line += " Failed: #{failed.to_s.ljust(COLUMN_WIDTH).colorize :red}"
line += " Pending: #{pending.to_s.ljust(COLUMN_WIDTH).colorize :blue}"
output line
end
# Show the main tasks report
def tasks_report
output Noop::Utils.separator
task_list.each do |task|
output_task_status task
end
output Noop::Utils.separator
tasks_stats
output Noop::Utils.separator
end
# Show the tasks and examples stats
def tasks_stats
output_examples_total unless options[:report_only_tasks]
output_task_totals
end
# Show report with all defined filters content
def show_filters
if options[:filter_specs]
options[:filter_specs] = [options[:filter_specs]] unless options[:filter_specs].is_a? Array
output "Spec filter: #{options[:filter_specs].join ', '}"
output "Spec filter: #{options[:filter_specs].join(', ').colorize :green}"
end
if options[:filter_facts]
options[:filter_facts] = [options[:filter_facts]] unless options[:filter_facts].is_a? Array
output "Facts filter: #{options[:filter_facts].join ', '}"
output "Facts filter: #{options[:filter_facts].join(', ').colorize :green}"
end
if options[:filter_hiera]
options[:filter_hiera] = [options[:filter_hiera]] unless options[:filter_hiera].is_a? Array
output "Hiera filter: #{options[:filter_hiera].join ', '}"
output "Hiera filter: #{options[:filter_hiera].join(', ').colorize :green}"
end
if options[:filter_examples]
options[:filter_examples] = [options[:filter_examples]] unless options[:filter_examples].is_a? Array
output "Examples filter: #{options[:filter_examples].join ', '}"
output "Examples filter: #{options[:filter_examples].join(', ').colorize :green}"
end
end
# Show the stats of discovered library objects
def show_library
template = <<-'eof'
<%= '=' * 80 %>
Tasks discovered: <%= task_file_names.length %>
Specs discovered: <%= spec_file_names.length %>
Hiera discovered: <%= hiera_file_names.length %>
Facts discovered: <%= facts_file_names.length %>
Tasks in graph metadata: <%= task_graph_metadata.length %>
Tasks with spec metadata: <%= spec_run_metadata.length %>
Total tasks to run: <%= task_list.count %>
Tasks discovered: <%= task_file_names.length.to_s.colorize :green %>
Specs discovered: <%= spec_file_names.length.to_s.colorize :green %>
Hiera discovered: <%= hiera_file_names.length.to_s.colorize :green %>
Facts discovered: <%= facts_file_names.length.to_s.colorize :green %>
Tasks in graph metadata: <%= task_graph_metadata.length.to_s.colorize :yellow %>
Tasks with spec metadata: <%= spec_run_metadata.length.to_s.colorize :yellow %>
Total tasks to run: <%= task_list.count.to_s.colorize :yellow %>
eof
output ERB.new(template, nil, '-').result(binding)
end
# Check the existence of main directories
def check_paths
paths = [
:dir_path_config,
@ -226,11 +218,46 @@ Total tasks to run: <%= task_list.count %>
:dir_path_reports,
]
max_length = paths.map { |p| p.to_s.length }.max
paths.each do |path|
directory = Noop::Config.send path
output "#{directory_check_status_string directory} #{path.to_s.ljust max_length} #{directory}"
end
end
# Output a list of tasks without a spec file
# and a list of specs without a task file.
def list_missing_tasks_and_specs
tasks_without_specs = find_tasks_without_specs.to_a
specs_without_tasks = find_specs_without_tasks.to_a
if tasks_without_specs.any?
Noop::Utils.output 'There are tasks without specs:'.colorize :red
tasks_without_specs.each do |task|
Noop::Utils.output "#{'*'.colorize :yellow} #{task}"
end
end
if specs_without_tasks.any?
Noop::Utils.output 'There are specs without tasks:'.colorize :red
specs_without_tasks.each do |spec|
Noop::Utils.output "#{'*'.colorize :yellow} #{spec}"
end
end
end
# Run all diagnostic procedures
def self_check
output Noop::Utils.separator 'Paths'
check_paths
if has_filters?
output Noop::Utils.separator 'Filters'
show_filters
end
output Noop::Utils.separator 'Missing'
list_missing_tasks_and_specs
output Noop::Utils.separator 'Library'
show_library
output Noop::Utils.separator 'End'
end
end
end

71
lib/noop/manager/setup.rb
View File

@ -1,59 +1,90 @@
module Noop
class Manager
# Get a GEM_HOME either from the environment (using RVM)
# or from the default value (using bundle)
# @return [Pathname]
def dir_path_gem_home
return Pathname.new ENV['GEM_HOME'] if ENV['GEM_HOME']
dir_name_bundle = Pathname.new 'bundled_gems'
Noop::Config.dir_path_workspace + dir_name_bundle
end
# Check if bundle command is installed
# @return [true,false]
def bundle_installed?
`bundle --version`
$?.exitstatus == 0
end
# Check if librarian-puppet command is installed
# @return [true,false]
def librarian_installed?
`librarian-puppet version`
$?.exitstatus == 0
end
# Setup bundle in the fixtures repo and bundle for puppet librarian
def setup_bundle
ENV['GEM_HOME'] = dir_path_gem_home.to_s
Dir.chdir Noop::Config.dir_path_root
bundle_install_and_update
Dir.chdir Noop::Config.dir_path_deployment
bundle_install_and_update
bundle_install_and_update Noop::Config.dir_path_root
bundle_install_and_update Noop::Config.dir_path_deployment
Dir.chdir Noop::Config.dir_path_root
end
# Run update script to setup external Puppet modules
def setup_library
ENV['GEM_HOME'] = dir_path_gem_home.to_s
Dir.chdir Noop::Config.dir_path_deployment
update_puppet_modules
update_puppet_modules Noop::Config.dir_path_deployment
Dir.chdir Noop::Config.dir_path_root
end
def bundle_install_and_update
Noop::Utils.error 'Bundle is not installed!' unless bundle_installed?
Noop::Utils.debug "Starting 'bundle install' in the Gem home: #{ENV['GEM_HOME']}"
Noop::Utils.run 'bundle install'
Noop::Utils.error 'Could not prepare bundle environment!' if $?.exitstatus != 0
Noop::Utils.debug "Starting 'bundle update' in the Gem home: #{ENV['GEM_HOME']}"
Noop::Utils.run 'bundle update'
Noop::Utils.error 'Could not update bundle environment!' if $?.exitstatus != 0
# @return [Pathname]
def file_name_gemfile_lock
Pathname.new 'Gemfile.lock'
end
# run librarian-puppet to fetch modules as necessary
def update_puppet_modules
Noop::Utils.error 'Puppet Librarian is not installed!' unless librarian_installed?
# Remove the Gem lock file at the given path
# @param root [String,Pathname]
def remove_gemfile_lock(root)
root = Noop::Utils.convert_to_path root
lock_file_path = root + file_name_gemfile_lock
if lock_file_path.file?
debug "Removing Gem lock file: '#{lock_file_path}'"
lock_file_path.unlink
end
end
# Run bundles install and update actions in the given folder
# @param root [String,Pathname]
def bundle_install_and_update(root)
error 'Bundle is not installed!' unless bundle_installed?
root = Noop::Utils.convert_to_path root
remove_gemfile_lock root
Dir.chdir root or error "Could not chdir to: #{root}"
debug "Starting 'bundle install' at: '#{root}' with the Gem home: '#{ENV['GEM_HOME']}'"
Noop::Utils.run 'bundle install'
error 'Could not prepare bundle environment!' if $?.exitstatus != 0
debug "Starting 'bundle update' at: '#{root}' with the Gem home: '#{ENV['GEM_HOME']}'"
Noop::Utils.run 'bundle update'
error 'Could not update bundle environment!' if $?.exitstatus != 0
end
# Run librarian-puppet to fetch modules as
# necessary modules at the given folder
# @param root [String,Pathname]
def update_puppet_modules(root)
error 'Puppet Librarian is not installed!' unless librarian_installed?
root = Noop::Utils.convert_to_path root
Dir.chdir root or error "Could not chdir to: #{root}"
command = './update_modules.sh -v'
command = command + ' -b' if options[:bundle_exec]
command = command + ' -r' if options[:reset_librarian_puppet]
Noop::Utils.debug 'Starting update_modules script'
debug 'Starting update_modules script'
Noop::Utils.run command
Noop::Utils.error 'Unable to update upstream puppet modules using librarian-puppet!' if $?.exitstatus != 0
Noop::Utils.debug 'Finished update_modules script'
error 'Unable to update upstream puppet modules using librarian-puppet!' if $?.exitstatus != 0
debug 'Finished update_modules script'
end
end

70
lib/noop/manager/xunit.rb
View File

@ -1,8 +1,60 @@
require 'rexml/document'
module Noop
class Manager
def xunit_report(tasks)
tasks_report = tasks_report_structure tasks
return unless tasks_report.is_a? Array
# Generate a data structure that will be used to create the xUnit report
# @return [Array]
def tasks_report_structure
tasks_report = []
task_list.each do |task|
task_hash = {}
task_hash[:status] = task.status
task_hash[:name] = task.to_s
task_hash[:description] = task.description
task_hash[:spec] = task.file_name_spec.to_s
task_hash[:hiera] = task.file_name_hiera.to_s
task_hash[:facts] = task.file_name_facts.to_s
task_hash[:task] = task.file_name_manifest.to_s
task_hash[:examples] = []
if task.report.is_a? Hash
examples = task.report['examples']
next unless examples.is_a? Array
examples.each do |example|
example_hash = {}
example_hash[:file_path] = example['file_path']
example_hash[:line_number] = example['line_number']
example_hash[:description] = example['description']
example_hash[:status] = example['status']
example_hash[:run_time] = example['run_time']
example_hash[:pending_message] = example['pending_message']
exception_class = example.fetch('exception', {}).fetch('class', nil)
exception_message = example.fetch('exception', {}).fetch('message', nil)
next unless example_hash[:description] and example_hash[:status]
if exception_class and exception_message
example_hash[:exception_class] = exception_class
example_hash[:exception_message] = exception_message
end
task_hash[:examples] << example_hash
end
summary = task.report['summary']
task_hash[:example_count] = summary['example_count']
task_hash[:failure_count] = summary['failure_count']
task_hash[:pending_count] = summary['pending_count']
task_hash[:duration] = summary['duration']
end
tasks_report << task_hash
end
tasks_report
end
# Generate xUnit XML report text
# @return [String]
def xunit_report
document = REXML::Document.new
declaration = REXML::XMLDecl.new
declaration.encoding = 'UTF-8'
@ -13,7 +65,7 @@ module Noop
failures = 0
task_id = 0
tasks_report.each do |task|
tasks_report_structure.each do |task|
testsuite = testsuites.add_element 'testsuite'
testsuite.add_attribute 'id', task_id
task_id += 1
@ -67,19 +119,25 @@ module Noop
document.to_s
end
# xUnit report file name
# @return [Pathname]
def file_name_xunit_report
Pathname.new 'report.xml'
end
# Full path to the xUnit report file
# @return [Pathname]
def file_path_xunit_report
Noop::Config.dir_path_reports + file_name_xunit_report
end
# Write the xUnit report to the file
# @return [void]
def save_xunit_report
debug "Saving xUnit XML report file to: #{file_path_xunit_report.to_s}"
File.open(file_path_xunit_report.to_s, 'w') do |file|
file.puts xunit_report task_list
file.puts xunit_report
end
Noop::Utils.debug "xUnit XML report was saved to: #{file_path_xunit_report.to_s}"
end
end

2
lib/noop/task.rb
View File

@ -8,3 +8,5 @@ require_relative 'task/run'
require_relative 'task/overrides'
require_relative 'task/catalog'
require_relative 'task/helpers'
require_relative 'task/files'
require_relative 'task/report'

81
lib/noop/task/base.rb
View File

@ -8,50 +8,96 @@ module Noop
self.pid = Process.pid
self.thread = Thread.current.object_id
@parallel = false
Noop::Utils.warning "#{self}: Validation is failed!" unless valid?
end
attr_accessor :parallel
attr_accessor :pid
attr_accessor :thread
attr_accessor :status
attr_accessor :valid
# Check if this task's configuration is valid
# @return [true,false]
def valid?
validate unless valid.is_a? TrueClass or valid.is_a? FalseClass
valid
end
# @return [true,false]
def success?
status == :success
end
# @return [true,false]
def failed?
status == :failed
end
# @return [true,false]
def pending?
status == :pending
end
# Write a debug message to the logger
# @return [void]
def debug(message)
Noop::Config.log.debug message
end
# Output a message to the console
# @return [void]
def output(message)
Noop::Utils.output message
end
# Write an error message to the log
# and raise the exception
# @return [void]
def error(message)
Noop::Utils.error message
end
# Write a warning message to the log
# @return [void]
def warning(message)
Noop::Utils.warning message
end
# @return [true,false]
def parallel_run?
parallel
end
# @return [true,false]
def valid?
unless file_path_spec.exist?
Noop::Utils.warning "No spec file: #{file_path_spec}!"
return false
def validate
if file_name_spec_set?
unless file_present_spec?
warning "No spec file: #{file_path_spec}!"
self.valid = false
return valid
end
else
warning 'Spec file is not set for this task!'
self.valid = false
return valid
end
unless file_path_manifest.exist?
Noop::Utils.warning "No task file: #{file_path_manifest}!"
return false
unless file_present_manifest?
warning "No task file: #{file_path_manifest}!"
self.valid = false
return valid
end
unless file_path_hiera.exist?
Noop::Utils.warning "No hiera file: #{file_path_hiera}!"
return false
unless file_present_hiera?
warning "No hiera file: #{file_path_hiera}!"
self.valid = false
return valid
end
unless file_path_facts.exist?
Noop::Utils.error "No facts file: #{file_path_hiera}!"
return false
unless file_present_facts?
warning "No facts file: #{file_path_facts}!"
self.valid = false
return valid
end
true
self.valid = true
end
# @return [String]
@ -59,9 +105,10 @@ module Noop
"Task[#{file_base_spec}]"
end
# @return [String]
def description
message = ''
message += "Task: #{file_name_manifest}"
message += "Manifest: #{file_name_manifest}"
message += " Spec: #{file_name_spec}"
message += " Hiera: #{file_name_hiera}"
message += " Facts: #{file_name_facts}"
@ -69,6 +116,7 @@ module Noop
message
end
# @return [String]
def process_info
message = ''
message + "Object: #{object_id}"
@ -81,5 +129,6 @@ module Noop
def inspect
"Task[#{description}]"
end
end
end

82
lib/noop/task/catalog.rb
View File

@ -2,35 +2,8 @@ require 'erb'
module Noop
class Task
# @return [String]
def status_report(context)
task = context.task
template = <<-'eof'
Facts: <%= task.file_path_facts %>
Hiera: <%= task.file_path_hiera %>
Spec: <%= task.file_path_spec %>
Manifest: <%= task.file_path_manifest %>
Node: <%= task.hiera_lookup 'fqdn' or '?' %>
Role: <%= task.hiera_lookup 'role' or '?' %>
Puppet: <%= Puppet.version %>
Ruby: <%= RUBY_VERSION %>
Hiera hierarchy:
<% task.hiera_hierarchy.each do |element| -%>
* <%= element %>
<% end -%>
Facts hierarchy:
<% task.facts_hierarchy.reverse.each do |element| -%>
* <%= element %>
<% end -%>
eof
ERB.new(template, nil, '-').result(binding)
end
# dumps the entire catalog structure to the text
# Dumps the entire catalog structure to the text
# representation in the Puppet language
# @param context [Object] the context from the rspec test
# @param resources_filter [Array] the list of resources to dump. Dump all resources if not given
@ -56,7 +29,7 @@ Facts hierarchy:
text
end
# takes a parameter value and formats it to the literal value
# Takes a parameter value and formats it to the literal value
# that could be placed in the Puppet manifest
# @param value [String, Array, Hash, true, false, nil]
# @return [String]
@ -88,7 +61,7 @@ Facts hierarchy:
end
end
# take a resource object and generate a manifest representation of it
# Take a resource object and generate a manifest representation of it
# in the Puppet language. Replaces "to_manifest" Puppet function which
# is not working correctly.
# @param resource [Puppet::Resource]
@ -131,7 +104,7 @@ Facts hierarchy:
clear_data.join "\n"
end
# check if two resources have same type and title
# Check if two resources have same type and title
# @param res1 [Puppet::Resource]
# @param res2 [Puppet::Resource]
# @return [TrueClass, False,Class]
@ -141,5 +114,52 @@ Facts hierarchy:
res1 == res2
end
# @return [Pathname]
def dir_name_catalogs
Pathname.new 'catalogs'
end
# @return [Pathname]
def dir_path_catalogs
Noop::Config.dir_path_root + dir_name_catalogs
end
# @return [Pathname]
def file_name_task_catalog
Noop::Utils.convert_to_path "#{file_name_base_task_report}.pp"
end
# @return [Pathname]
def file_path_task_catalog
dir_path_catalogs + file_name_task_catalog
end
# Write the catalog file of this task
# using the data from RSpec context
# @param context [Object] the context from the rspec test
# @return [void]
def file_write_task_catalog(context)
dir_path_catalogs.mkpath
error "Catalog directory '#{dir_path_catalogs}' doesn't exist!" unless dir_path_catalogs.directory?
debug "Writing catalog file: #{file_path_task_catalog}"
File.open(file_path_task_catalog.to_s, 'w') do |file|
file.puts catalog_dump context
end
end
# Check if the catalog file exists for this task
# @return [true,false]
def file_present_task_catalog?
file_path_task_catalog.file?
end
# Read the catalog file of this task
# @return [String]
def file_read_task_catalog
return unless file_present_task_catalog?
debug "Reading catalog file: #{file_path_task_catalog}"
file_path_task_catalog.read
end
end
end

3
lib/noop/task/facts.rb
View File

@ -60,6 +60,7 @@ module Noop
file_paths
end
# @return [void]
def add_host_names(facts_data)
hostname = hiera_lookup 'node_name'
fqdn = hiera_lookup 'fqdn'
@ -86,10 +87,12 @@ module Noop
alias :ubuntu_facts :facts_data
alias :centos_facts :facts_data
# @return [String,nil]
def hostname
facts_data[:hostname]
end
# @return [String,nil]
def fqdn
facts_data[:fqdn]
end

100
lib/noop/task/files.rb Normal file
View File

@ -0,0 +1,100 @@
module Noop
class Task
# @return [Pathname]
def dir_name_file_reports
Pathname.new 'files'
end
# @return [Pathname]
def dir_path_file_reports
Noop::Config.dir_path_reports + dir_name_file_reports
end
# @return [Pathname]
def file_name_file_report
Noop::Utils.convert_to_path "#{file_name_base_task_report}.yaml"
end
# @return [Pathname]
def file_path_file_report
dir_path_file_reports + file_name_file_report
end
# @return [Array<Puppet::Type>]
def find_file_resources(context)
catalog = context.subject
catalog = catalog.call if catalog.is_a? Proc
catalog.resources.select do |resource|
resource.type == 'File'
end
end
# @return [Hash]
def catalog_file_report_structure(context)
files = {}
find_file_resources(context).each do |resource|
next unless %w(present file directory).include? resource[:ensure] or not resource[:ensure]
if resource[:source]
content = resource[:source]
elsif resource[:content]
content = 'TEMPLATE'
else
content = nil
end
next unless content
files[resource[:path]] = content
end
files
end
# @return [String]
def catalog_file_report_template(binding)
template = <<-'eos'
<% if binary_files.any? -%>
You have <%= binary_files.length -%> files that are either binary or init.d scripts:
<% binary_files.each do |file| -%>
* <%= file %>
<% end -%>
<% end -%>
<% if downloaded_files.any? -%>
You are downloading <%= downloaded_files.length -%> files using File resource's source property:
<% downloaded_files.each do |file| -%>
* <%= file %>
<% end -%>
<% end -%>
eos
ERB.new(template, nil, '-').result(binding)
end
# @return [void]
def catalog_file_resources_check(context)
binary_files_regexp = %r{^/bin|^/usr/bin|^/usr/local/bin|^/usr/sbin|^/sbin|^/usr/lib|^/usr/share|^/etc/init.d|^/usr/local/sbin|^/etc/rc\S\.d}
binary_files = []
downloaded_files = []
find_file_resources(context).each do |resource|
next unless %w(present file directory).include? resource[:ensure] or not resource[:ensure]
file_path = resource[:path] or resource[:title]
file_source = resource[:source]
binary_files << file_path if file_path =~ binary_files_regexp
downloaded_files << file_path if file_source
end
if binary_files.any? or downloaded_files.any?
output Noop::Utils.separator
output catalog_file_report_template(binding)
output Noop::Utils.separator
error 'Puppet is installing files that should be packed to the Fuel package!'
end
end
# @return [void]
def catalog_file_report_write(context)
dir_path_file_reports.mkpath
error "File report directory '#{dir_path_file_reports}' doesn't exist!" unless dir_path_file_reports.directory?
debug "Saving File resources list file to: #{file_path_file_report.to_s}"
File.open(file_path_file_report.to_s, 'w') do |file|
YAML.dump catalog_file_report_structure(context), file
end
end
end
end

2
lib/noop/task/globals.rb
View File

@ -13,10 +13,10 @@ module Noop
end
def write_file_globals(content)
debug "Saving Globals YAML file to: '#{file_path_globals.to_s}'"
File.open(file_path_globals.to_s, 'w') do |file|
file.write content
end
Noop::Utils.debug "Globals YAML saved to: '#{file_path_globals.to_s}'"
end
# @return [Pathname]

45
lib/noop/task/helpers.rb
View File

@ -1,46 +1,77 @@
module Noop
class Task
# Extract the parameter or property of a Puppet resource in the catalog
# @param context [RSpec::ExampleGroup] The 'self' of the RSpec example group
# @param resource_type [String] Name of the resource type
# @param resource_name [String] Title of the resource
# @param parameter [String] Parameter name
# @return [Object]
def resource_parameter_value(context, resource_type, resource_name, parameter)
catalog = context.subject
catalog = catalog.call if catalog.is_a? Proc
resource = catalog.resource resource_type, resource_name
fail "No resource type: '#{resource_type}' name: '#{resource_name}' in the catalog!" unless resource
error "No resource type: '#{resource_type}' name: '#{resource_name}' in the catalog!" unless resource
resource[parameter.to_sym]
end
# save the current puppet scope
# Save the current puppet scope
# @param value [Puppet::Scope]
def puppet_scope=(value)
@puppet_scope = value
end
# The saved Puppet scope to run functions in
# Or the newly generated scope.
# @return [Puppet::Scope]
def puppet_scope
return @puppet_scope if @puppet_scope
PuppetlabsSpec::PuppetInternals.scope
end
# load a puppet function if it's not already loaded
# Load a puppet function if it's not already loaded
# @param name [String] Function name
def puppet_function_load(name)
name = name.to_sym unless name.is_a? Symbol
Puppet::Parser::Functions.autoloader.load name
end
# call a puppet function and return it's value
# Call a puppet function and return it's value
# @param name [String] Function name
# @param *args [Object] Function parameters
# @return [Object]
def puppet_function(name, *args)
name = name.to_sym unless name.is_a? Symbol
puppet_function_load name
fail "Could not load Puppet function '#{name}'!" unless puppet_scope.respond_to? "function_#{name}".to_sym
error "Could not load Puppet function '#{name}'!" unless puppet_scope.respond_to? "function_#{name}".to_sym
puppet_scope.send "function_#{name}".to_sym, args
end
# take a variable value from the saved puppet scope
# Take a variable value from the saved puppet scope
# @param name [String] variable name
def lookupvar(name)
puppet_scope.lookupvar name
end
alias :variable :lookupvar
# convert resource catalog to a RAL catalog
# Load a class from the Puppet modules into the current scope
# It can be used to extract values from 'params' classes like this:
# Noop.load_class 'nova::params'
# Noop.variable 'nova::params::common_package_name'
# => 'openstack-nova-common'
# These values can be later used in the spec examples.
# Note, that the loaded class will not be found in the spec's catalog
# object, but can be found here: Noop.puppet_scope.catalog
# @param class_name [String]
def puppet_class_include(class_name)
class_name = class_name.to_s
puppet_function 'include', class_name unless Noop.puppet_scope.catalog.classes.include? class_name
end
# Convert resource catalog to a RAL catalog
# and run both "generate" functions for each resource
# that has it and then add results to the catalog
# @param context [RSpec::ExampleGroup] The 'self' of the RSpec example group
# @return <Lambda>
def create_ral_catalog(context)
catalog = context.subject

2
lib/noop/task/hiera.rb
View File

@ -9,6 +9,7 @@ module Noop
@file_name_hiera
end
# @return [Pathname]
def file_name_hiera=(value)
return if value.nil?
@file_name_hiera = Noop::Utils.convert_to_path value
@ -141,6 +142,7 @@ module Noop
data = hiera key, nil, resolution_type
path_lookup.call data, path, default
end
alias :hiera_dir :hiera_structure
end
end

11
lib/noop/task/overrides.rb
View File

@ -1,19 +1,24 @@
module Noop
class Task
# Setup all needed override functions
def setup_overrides
puppet_default_settings
hiera_config_override
puppet_debug_override if ENV['SPEC_PUPPET_DEBUG']
setup_manifest
puppet_resource_scope_override
return unless file_name_spec_set?
hiera_config_override
setup_manifest
end
# Set the current module path and the manifest file
# to run in this RSpec session
def setup_manifest
RSpec.configuration.manifest = file_path_manifest.to_s
RSpec.configuration.module_path = Noop::Config.dir_path_modules_local.to_s
RSpec.configuration.manifest_dir = Noop::Config.dir_path_tasks_local.to_s
end
# Override Hiera configuration in the Puppet objects
def hiera_config_override
class << HieraPuppet
def hiera
@ -48,6 +53,7 @@ module Noop
Hiera::Config.config = hiera_config
end
# Ask Puppet to save the current scope reference to the task instance
def puppet_resource_scope_override
Puppet::Parser::Resource.module_eval do
def initialize(*args)
@ -60,6 +66,7 @@ module Noop
end
end
# Divert Puppet logs to the console
def puppet_debug_override
Puppet::Util::Log.level = :debug
Puppet::Util::Log.newdestination(:console)

100
lib/noop/task/report.rb Normal file
View File

@ -0,0 +1,100 @@
module Noop
class Task
attr_accessor :report
# Generate the report of the currently using files in this spec
# @return [String]
def status_report(context)
task = context.task
template = <<-'eof'
Facts: <%= task.file_path_facts %>
Hiera: <%= task.file_path_hiera %>
Spec: <%= task.file_path_spec %>
Manifest: <%= task.file_path_manifest %>
Node: <%= task.hiera_lookup 'fqdn' or '?' %>
Role: <%= task.hiera_lookup 'role' or '?' %>
Hiera hierarchy:
<% task.hiera_hierarchy.each do |element| -%>
* <%= element %>
<% end -%>
Facts hierarchy:
<% task.facts_hierarchy.reverse.each do |element| -%>
* <%= element %>
<% end -%>
eof
ERB.new(template, nil, '-').result(binding)
end
# Get a loaded gem version
# @return [String,nil]
def gem_version(gem)
gem = gem.to_s
return unless Object.const_defined? 'Gem'
return unless Gem.loaded_specs.is_a? Hash
return unless Gem.loaded_specs[gem].is_a? Gem::Specification
Gem.loaded_specs[gem].version
end
# Gem a report about RSpec gems versions
# @return [String]
def gem_versions_report
versions = "Ruby version: #{RUBY_VERSION}"
%w(puppet rspec rspec-puppet rspec-puppet-utils puppetlabs_spec_helper).each do |gem|
version = gem_version gem
versions += "\n'#{gem}' gem version: #{version}"if version
end
versions
end
# Load a report file of this task if it's present
def file_load_report_json
self.report = file_data_report_json
end
# Check if this task has report loaded
# @return [true,false]
def has_report?
report.is_a? Hash and report['examples'].is_a? Array
end
# @return [Pathname]
def file_name_report_json
Noop::Utils.convert_to_path "#{file_name_base_task_report}.json"
end
# @return [Pathname]
def file_path_report_json
Noop::Config.dir_path_reports + file_name_report_json
end
# @return [Hash]
def file_data_report_json
return unless file_present_report_json?
file_data = nil
begin
# debug "Reading report file: #{file_path_report_json}"
file_content = File.read file_path_report_json.to_s
file_data = JSON.load file_content
return unless file_data.is_a? Hash
rescue
debug "Error parsing report file: #{file_path_report_json}"
nil
end
file_data
end
# Remove the report file
def file_remove_report_json
#debug "Removing report file: #{file_path_report_json}"
file_path_report_json.unlink if file_present_report_json?
end
# @return [true,false]
def file_present_report_json?
file_path_report_json.exist?
end
end
end

57
lib/noop/task/run.rb
View File

@ -2,23 +2,24 @@ require 'json'
module Noop
class Task
# Run the actual spec of this task.
# It will execute the rspec command and will manage report files.
def run
validate
error 'Validation of this task have failed!' unless valid?
return unless pending?
self.pid = Process.pid
self.thread = Thread.current.object_id
Noop::Utils.debug "RUN: #{self.inspect}"
debug "RUN: #{self.inspect}"
file_remove_report_json
rspec_command_run
file_load_report_json
determine_task_status
Noop::Utils.debug "FINISH: #{self.inspect}"
debug "FINISH: #{self.inspect}"
status
end
def file_load_report_json
self.report = file_data_report_json
end
# Set the status string of the task according to run results
def set_status_value(value)
if value.is_a? TrueClass
self.status = :success
@ -29,6 +30,7 @@ module Noop
end
end
# Try to determine the task status based on the report data
def determine_task_status
if report.is_a? Hash
failures = report.fetch('summary', {}).fetch('failure_count', nil)
@ -39,45 +41,17 @@ module Noop
status
end
# @return [Pathname]
def file_name_report_json
Noop::Utils.convert_to_path "#{file_name_task_extension.sub_ext ''}_#{file_base_hiera}_#{file_base_facts}.json"
end
# @return [Pathname]
def file_path_report_json
Noop::Config.dir_path_reports + file_name_report_json
end
# @return [Hash]
def file_data_report_json
return unless file_present_report_json?
file_data = nil
begin
file_content = File.read file_path_report_json.to_s
file_data = JSON.load file_content
return unless file_data.is_a? Hash
rescue
nil
end
file_data
end
def file_remove_report_json
file_path_report_json.unlink if file_present_report_json?
end
# @return [true,false]
def file_present_report_json?
file_path_report_json.exist?
end
# Additional RSpec options
def rspec_options
options = '--color --tty'
options += ' --format documentation' unless parallel_run?
options
end
# Run the RSpec command and pass Hiera, facts and spec files names
# using the environment variables.
# Use bundler if it's enabled.
# Set the task status according to the RSpec exit code.
# @return [true,false]
def rspec_command_run
environment = {
@ -89,10 +63,13 @@ module Noop
command = "bundle exec #{command}" if ENV['SPEC_BUNDLE_EXEC']
Dir.chdir Noop::Config.dir_path_root
success = Noop::Utils.run environment, command
if success.nil?
debug 'RSpec command is not found!'
success = false
end
set_status_value success
success
end
attr_accessor :report
end
end

27
lib/noop/task/spec.rb
View File

@ -4,9 +4,24 @@ module Noop
def file_name_spec
return @file_name_spec if @file_name_spec
self.file_name_spec = Noop::Utils.path_from_env 'SPEC_FILE_NAME'
error 'The spec file name is not set for this task!' unless file_name_spec_set?
@file_name_spec
end
# Check if the spec name for this spec is set
# @return [true,false]
def file_name_spec_set?
not @file_name_spec.nil?
end
# Check if the currently running spec is the same as the given one
# @return [true,false]
def current_spec_is?(spec)
return false unless file_name_spec_set?
spec = Noop::Utils.convert_to_spec spec
file_name_spec == spec
end
# @return [Pathname]
def file_base_spec
Noop::Utils.convert_to_path(file_name_spec.to_s.gsub /_spec\.rb$/, '')
@ -35,14 +50,24 @@ module Noop
end
# @return [true,false]
def file_present_spec
def file_present_spec?
file_path_spec.readable?
end
# @return [true,false]
def file_present_manifest?
file_path_manifest.readable?
end
# @return [Pathname]
def file_name_task_extension
Noop::Utils.convert_to_path(file_base_spec.to_s.gsub('/', '-') + '.yaml')
end
# @return [Pathname]
def file_name_base_task_report
Noop::Utils.convert_to_path("#{file_name_task_extension.sub_ext ''}_#{file_base_hiera}_#{file_base_facts}")
end
end
end

16
lib/noop/utils.rb
View File

@ -57,7 +57,7 @@ module Noop
end
def self.run(*args)
debug "CMD: #{args.inspect} PWD: #{Dir.pwd}"
# debug "CMD: #{args.inspect} PWD: #{Dir.pwd}"
system *args
end
@ -75,7 +75,19 @@ module Noop
def self.error(message)
Noop::Config.log.fatal message
exit(1)
fail message
end
def self.output(message)
puts message
end
def self.separator(title=nil)
if title
"=< #{title} >=".ljust 70, '='
else
'=' * 70
end
end
end
end

1
reports/files/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
*.yaml

1
requirements.txt
View File

@ -1,3 +1,4 @@
sphinx
cloud_sptheme
pbr
pip

53
spec/lib/task_spec.rb
View File

@ -47,7 +47,7 @@ describe Noop::Task do
end
it 'should have valid?' do
is_expected.not_to be_valid
is_expected.to respond_to(:valid?)
end
it 'should have to_s' do
@ -55,7 +55,7 @@ describe Noop::Task do
end
it 'should have inspect' do
expect(subject.inspect).to eq 'Task[Task: my/test.pp Spec: my/test_spec.rb Hiera: novanet-primary-controller.yaml Facts: ubuntu.yaml Status: pending]'
expect(subject.inspect).to eq 'Task[Manifest: my/test.pp Spec: my/test_spec.rb Hiera: novanet-primary-controller.yaml Facts: ubuntu.yaml Status: pending]'
end
end
@ -199,4 +199,53 @@ describe Noop::Task do
end
end
context 'validation' do
context 'valid task' do
before(:each) do
allow(subject).to receive(:file_present_spec?).and_return true
allow(subject).to receive(:file_present_manifest?).and_return true
allow(subject).to receive(:file_present_hiera?).and_return true
allow(subject).to receive(:file_present_facts?).and_return true
end
it 'should be valid' do
subject.validate
is_expected.to be_valid
end
end
context 'spec is not set' do
subject do
Noop::Task.new
end
before(:each) do
allow(Noop::Utils).to receive(:warning)
end
it 'should report unset spec' do
is_expected.not_to be_file_name_spec_set
end
it 'should not be valid' do
subject.validate
is_expected.not_to be_valid
end
end
context 'spec is set but missing' do
it 'should NOT report unset spec' do
is_expected.to be_file_name_spec_set
end
it 'should not be valid' do
subject.validate
is_expected.not_to be_valid
end
end
end
end

53
spec/shared-examples.rb
View File

@ -1,20 +1,51 @@
begin
require_relative 'hosts/common.rb'
rescue LoadError
nil
end
shared_examples 'compile' do
it { is_expected.to compile }
end
shared_examples 'show_catalog' do
it 'shows catalog contents' do
puts '=' * 80
puts Noop.task.catalog_dump self
puts '=' * 80
Noop::Utils.output Noop::Utils.separator
Noop::Utils.output Noop.task.catalog_dump self
Noop::Utils.output Noop::Utils.separator
end
end
shared_examples 'status' do
it 'shows status' do
puts '=' * 80
puts Noop.task.status_report self
puts '=' * 80
Noop::Utils.output Noop::Utils.separator
Noop::Utils.output Noop.task.status_report self
Noop::Utils.output Noop::Utils.separator
Noop::Utils.output Noop.task.gem_versions_report
Noop::Utils.output Noop::Utils.separator
end
end
shared_examples 'files_installed_by_puppet' do
it 'should check that binary files are not installed by this task' do
Noop.catalog_file_resources_check self
end
end
shared_examples 'save_files_list' do
it 'should save the list of File resources to the file' do
Noop.catalog_file_report_write self
end
end
shared_examples 'saved_catalog' do
it 'should save the current task catalog to the file', :if => (ENV['SPEC_CATALOG_CHECK'] == 'save') do
Noop.file_write_task_catalog self
end
it 'should check the current task catalog against the saved one', :if => (ENV['SPEC_CATALOG_CHECK'] == 'check') do
saved_catalog = Noop.preprocess_catalog_data Noop.file_read_task_catalog
current_catalog = Noop.preprocess_catalog_data Noop.catalog_dump self
expect(saved_catalog).to eq current_catalog
end
end
@ -32,6 +63,7 @@ def run_test(manifest_file, *args)
Noop::Config.log.progname = 'noop_spec'
Noop::Utils.debug "RSPEC: #{Noop.task.inspect}"
Noop.setup_overrides
include FuelRelationshipGraphMatchers
@ -68,6 +100,9 @@ def run_test(manifest_file, *args)
include_examples 'status' if ENV['SPEC_SHOW_STATUS']
include_examples 'show_catalog' if ENV['SPEC_CATALOG_SHOW']
include_examples 'console' if ENV['SPEC_RSPEC_CONSOLE']
include_examples 'files_installed_by_puppet' if ENV['SPEC_PUPPET_BINARY_FILES']
include_examples 'save_files_list' if ENV['SPEC_SAVE_FILE_RESOURCES']
include_examples 'saved_catalog' if ENV['SPEC_CATALOG_CHECK']
begin
include_examples 'catalog'
@ -75,6 +110,12 @@ def run_test(manifest_file, *args)
true
end
begin
it_behaves_like 'common'
rescue ArgumentError
true
end
yield self if block_given?
end