16 KiB
Validation tool for Murano Application Packages
https://blueprints.launchpad.net/murano/+spec/murano-app-validation
Murano doesn't have validation mechanisms for MuranoPL language and application packages.
This spec proposes a validation toolkit that improves workflow by making error detection possible at the early stages of application development lifecycle.
Problem description
Early error detection simplifies application development, troubleshooting and support.
Currently Murano does not provide mechanisms to perform validation of the Murano application package before running deployment process. Thus Murano cannot prevent uploading of invalid package and developer cannot verify that application package contains valid structure and code before deployment process.
Many problems with application packages (such as invalid package structure, errors in code, typos, etc.) can be detected and solved by using static validation tool.
Proposed change
Introduce a new toolkit for validation MuranoPL language and Murano application packages.
- Validation toolkit SHOULD be implemented as a standalone Python package (library) with CLI interface.
- Validation library will be used to validate Murano packages during loading process.
- CLI tool can be used by application developer to perform early checks before package is assembled.
- Toolkit should be extensible with custom validators. It will provide API for extension purposes.
- Toolkit should be configurable to run list of specified inspections or to ignore specified inspections.
Toolkit should implement the following validation suites: * Manifest validation * Classes validation * UI definition validation Other validators will be implemented later as part of this tool or as external packages
Manifest validation
manifest.yaml
file should exist and should be a valid YAML document.manifest.yaml
structure should respond to manifest structure described inmanifest-structure-section
.- All files referenced in
Classes
section should exist inClasses
subdirectory and should be valid YAML documents. - Warning should be reported in case when additional files exist in
Classes
subdirectory but not referenced in the manifest file. - UI definition file should exist for Application package (MuranoPL =< 1.4)
Class validation
- Valid classes and methods should check if properties are defined correctly and there are no extra keys that are not supported.
- Check if class is in correct namespaces according to manifest and class name matches.
- Contract fields should be valid YAQL expressions.
- Methods blocks should be checked for MuranoPL code structure correctness. Method blocks may consist only of Expressions, Assignments and Block constructs. See full terms description for details.
- (Optional) Detect code that cannot be parsed as YAQL expression but is considered to be a valid expression.
UI validation
- UI definition should contain
Application
andForms
sections - YAQL expressions in UI definition should be valid
Form
section should contain only valid forms http://docs.openstack.org/developer/murano/draft/appdev-guide/murano_packages.html#forms
Errors
Error code consists of prefix and error number. Prefix is an upper
case alphabetical sequence. It can be empty for general errors. Errors
specific for some validator should use validator short name as prefix.
Error number is an upper case alphanumerical sequence (one single letter
E
or W
following by three digits). Examples:
E007
, MPL:E042
for errors and
W777
, UI:W123
for warnings. Validation tool
should have ability to produce documentation of supported errors and
warnings.
Package structure reference
http://docs.openstack.org/developer/murano/draft/appdev-guide/murano_packages.html#package-structure
Package consists of:
- Manifest - File in package root directory named
manifest.yaml
. - MuranoPL classes - YAML files located in
Classes
directory or its subdirectories (optional). - UI definition - YAML file in
UI
directory (optional). - Resources - files in
Resources
directory (optional). - Logo (optional)
- images.lst file (optional)
Manifest structure
Format
Application package format. Can be specified as
<FORMAT>/<VERSION>
, where format is a format
name string (can be MuranoPL
or Heat.HOT
). If
format is omitted, MuranoPL
is used by default.
- Required: no
- Default:
MuranoPL/1.0
- Format:
<FORMAT>/<VERSION>
Type
- Required: yes
- Type: Enum(Application, Library)
Name
Package human readable name
- Required: no
- Type: String
FullName
Package fully qualified name. Should be in lowerCamelCase notation separated by dots.
- Required: yes
- Type: String
Version
Version of package
- Required: no
- Default:
0.0.0
- Type: String, should have valid SemVer format
Description
- Required: no
- Type: String
Author
- Required: no
- Type: String
Tags
- Required: no
- Type: List(String)
Classes
- Required: no
- Type: Dict
- Keys: String, Fully qualified class name
- Values: String, Filename relative to
Classes
directory
Require
Package external requirements.
- Required: no
- Type: Dict
- Keys: Dependent package
FullName
- Values:
null
or string with exact version ('1', '2.3', '4.5.6') or -
valid spec_strings according to python-semanticversion docs
- Values:
- Keys: Dependent package
UI
File with package UI definition
- Required: yes (for package type
Application
) - Type: String, Filename relative to root directory
- Default:
logo.png
Logo
Package logo in png format.
- Required: no
- Type: String, Filename relative to
UI
directory - Default:
ui.yaml
Meta
- Required: no
- Type: Dict or List(Dict) with metadata information
Class structure
Name
- Required: yes, if more than one class defined in the file
- Type: String
- Format:
- Matches Python class name format, except leading double underscore
(
__ClassName
). - Matches CamelCase naming style.
- Matches Python class name format, except leading double underscore
(
Properties
- Required: no
- Type: Dict
For more details see class-properties-section
.
Methods
- Required: no
- Type: Dict
For more details see class-methods-section
.
Extends
Single or multiple class names.
- Required: no
- Type: String | YAML List
Namespaces
- Required: no
- Type: Dict
- Keys: String; Namespace alias
- Values: String; Namespace full qualified name
- Format:
- Namespace alias is a valid YAQL identifier or
=
. - Namespace FQN is a string divided by dots (
.
). - Recommendation is to use lowerCamelCase naming style.
- Namespace alias is a valid YAQL identifier or
Import
- Required: no
- Type: String, Class name, List(Class names)
Usage
- Required: no
- Type: Enum(Class, Meta)
- Default:
Class
Meta
- Required: no
- Type: Dict or List(Dict) with metadata information
MetaClass structure
In addition to regular Class structure, MetaClass (Class with Usage: Meta) can have additional properties:
Cardinality
- Required: no
- Type: Enum(One, Many)
Applies
- Required: no
- Type: Enum(Package, Type, Method, Property, Argument, All) or List(Enum)
Inherited
- Required: no
- Type: boolean
Class properties
Contract
- Required: yes
- Type: YAQL Expression
Usage
- Required: no
- Type: Enum
Usage | |
---|---|
In Out InOut Const Runtime |
Default |
Config | Since version 1.1 |
Static | Since version 1.2 |
Default
- Required: no
- Type: Any value
Meta
- Required: no
- Type: Dict or List(Dict) with metadata information
Class methods
Scope
1.4
- Required: no
- Type: Enum
- Values: Public, Session
Arguments
- Required: no
- Type: List(Dict)
For more details see methods-argument-section
.
Body
- Required: no
- Type: YAQL Expression | List(YAQL Expression)
Usage
- Required: no
- Type: Enum
Usage | |
---|---|
Runtime | Default |
Static | Since 1.3 |
Extension | Since 1.3 |
Action | Deprecated since 1.4 |
Meta
- Required: no
- Type: Dict or List(Dict) with metadata information
Method's Argument
- Type: Dict(String, Dict)
- Key: Argument name
Values:
- Contract: YAQL Expression
- Default: YAQL Expression
- Usage: Enum added since 1.4
Usage | |
---|---|
Standard VarArgs KwArgs |
Default |
Meta
- Required: no
- Type: List with metadata information
Extensions
Validation tool should be extensible and provide unified mechanism for builtin checks and extensions.
Extensions should be implemented as a python package and registered using entry points.
Extensions are loaded using stevedore library.
CLI Usage
Usage: mpl-check [options] <PACKAGE | DIRECTORY>
Arguments:
DIRECTORY Application working directory (e.g. contains
manifest.yaml, Classes/, etc.)
PACKAGE Compressed application (ZIP-archive)
Options:
--ignore=errors Skip errors and warnings.
--select=errors Check only for selected errors and warnings.
--scope=validators List of validators to execute
--strict Consider warnings as an errors.
Alternatives
Implement validation as part of Murano and validate the package when it is uploaded to Murano.
Pros:
- Easier to implement.
Cons:
- Suitable for Murano itself, but not for developers.
- Integration with CI which requires execution of functional tests and is slower than static validation.
- Cannot be integrated with text editors or IDEs.
Data model impact
None
REST API impact
Murano API for upload packages should use this tool to do package validation before store it. If package is not valid, HTTP error with code 400 should be returned to user with list of check errors in description.
Versioning impact
None
Other end user impact
- Murano application package developer will be able to validate package during development process.
Deployer impact
None
Developer impact
Application developer will be able to validate application code during development process without importing package to Murano and running deployment process or test suite.
Murano-dashboard / Horizon impact
Error message is displayed in Horizon when trying to upload malformed package.
Implementation
Application class
Application class provides an entry point to the checker. It is responsible for configuration, extensions loading and validation execution.
Application options are registered using oslo.config
Methods:
- __init__(self, options)
load_plugins(self)
- Finds validators from installed pluginsprepare_check_suit(self, loaded_package, validator_list)
- Collect prepared checks from all validators in validator_list against specified package. Returns list with prepared checks.list_errors(self, filters)
- Lists errors/warnings exposed by all available validators
Validator classes
Validator class represents set for specific checks and logic to register them.
Classes
BaseValidator
+-- PackageValidator
+-- ManifestValidator
+-- UiValidator
+-- ClassValidator
Attributes
short_name
short name for validator. Used as prefix for validator specific errors
Methods
__init__(loaded_package)
- Init validator for specific formatrun(self)
- Runs validator against specified suit of files or whole package if file_suit=None (for Package validator)
Checkers
Checker is a function that performs checking logic. It takes data from args and yields error if it is not valid. Checkers should not contain logic for data preparation. Checker can be reused multiple times with different options in a single suit of checks. Checker can be wrapped with custom filters (i.e. manifest version), so it is called only for matching conditions.
Exceptions
Class CheckerError
contains information including error
code, message, file name and position in that file. Additionally it can
include code snippet, if it's available. For YAML data it's available
using yaml.error.Mark
class. Otherwise code snippet can be
generated during error formatting.
Attributes
- code - Error code (Usually single letter and three digits)
- message - Human readable error reason
- filename - File name relative to package root
- line - Line number (starts from 0, optional)
- column - Column number (starts from 0, optional)
- source - One line code snippet with highlighted column (optional)
Package loaders
Package loaders provide interface to Murano packages in multiple formats. Currently packages can be loaded from directory, single file or zip archive package.
PackageLoader
+-- DirectoryLoader
+-- FileLoader
+-- ZipLoader
Methods
try_load(cls, path, *options)
- [classmethod] Tries to load package, if it can be loaded returns instance of loaded package, otherwise None.
Loaded package
Loaded package class introduces a general interface for access to packages of different format.
Methods
get_file(self, name)
- File wrapper if file exists, None otherwiseformat(self)
- Ruexists(self, name)
- Returns true if file exists in the package.list(self, subdir=None)
- Returns list of all files in the package (or in it's subdirectory).search_for(self, regex)
- Return iterator of files fit to regex
File wrapper allows to access to raw file context with method
raw()
and to parsed dict with method
parsed()
Formatters
Formatter classes help to represent error report in various ways.
Errors can be printed to stdout
/stderr
using
text representation, which suits developer or written to file in
JSON
/YAML
format.
Assignee(s)
- Primary assignee:
-
Alexander Saprykin <cutwater>
- Other contributors:
-
Krzysztof Szukielojc <kszukielojc> Sergey Slipushenko <sslypushenko>
Work Items
- Develop core framework with support of extensions.
- Implement manifest and package structure checks.
- Implement Murano PL classes checks.
- Implement UI definition file validation.
- Implement YAQL checks.
- Integrate package validation with murano API.
- Develop CLI tool based on core framework
- Create documentation.
Dependencies
Testing
Package should include unit and functional tests.
Documentation Impact
Documentation for the library should include:
- Usage information that describes how to use the application.
- API reference.
- Plugin developer documentation.
References
None