openstack
/
syntribos
Code Issues Proposed changes

Added meta variable support to runner 

The runner will now respect inheritence when passing meta variables
to the parser.

Change-Id: I84ae827fcb396a1cb701d212601a8dcd56a37d9e
This commit is contained in:
Michael Dong 2017-02-15 17:09:47 -06:00
parent 6ded748d09
commit 5d7771fb8e
6 changed files with 217 additions and 69 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

57
README.rst
View File

@ -872,9 +872,7 @@ Syntribos allows for templates to read in variables from a user-specified
meta variable file. These files contain JSON objects that define variables
to be used in one or more request templates.
The file must be named `meta.json` and must be placed in the same
directory as the template files that reference it. Meta variable files take
the form:
The file must be named `meta.json`, and they take the form:
::
{
@ -884,11 +882,13 @@ the form:
"user_name": {
"type": config,
"val": "user.username"
"fuzz_types": ["ascii"]
},
"user_token": {
"type": "function",
"val": "syntribos.extensions.identity:get_scoped_token_v3",
"args": ["user"]
"args": ["user"],
"fuzz": false
}
}
@ -899,8 +899,6 @@ variables is as follows:
POST /user HTTP/1.1
X-Auth-Token: |user_token|
Accept: */*
Content-type: application/json
{
"user": {
@ -909,6 +907,53 @@ variables is as follows:
}
}
Note: Meta-variable usage in templates should take the form `|user_name|`, not
`user_|name|` or `|user|_|name|`. This is to avoid ambiguous behavior when the
value is fuzzed.
Meta Variable Attributes
------------------------
* val - All meta variable objects must define a value, which can be of any json
DataType. Unlike the other attributes, this attribute is not optional.
* type - Defining a type instructs syntribos to interpret the variable in a
certain way. Any variables without a type defined will be read in directly
from the value. The following types are allowed:
* config - syntribos reads the config value specified by the "val"
attribute and returns that value.
* function - syntribos calls the function named in the "val" attribute
with any arguments given in the optional "args" attribute, and returns the
value from calling the function. This value is cached, and will be returned
on subsequent calls.
* generator - Works the same way as the function type, but its results are
not cached and the function will be called every time.
* args - A list of function arguments (if any) which can be defined here if the
variable is a generator or a function
* fuzz - A boolean value that, if set to false, instructs syntribos to
ignore this variable for any fuzz tests
* fuzz_types - A list of strings which instructs syntribos to only use certain
fuzz strings when fuzzing this variable. More than one fuzz type can be
defined. The following fuzz types are allowed:
* ascii - strings that can be encoded as ascii
* url - strings that contain only url safe characters
* min_length/max_length - An integer that instructs syntribos to only use fuzz
strings that meet certain length requirements
Inheritence
-----------
Meta variable files inherit based on the directory it's in. That is, if you
have `foo/meta.json` and `foo/bar/meta.json`, templates in `foo/bar/` will take
their meta variable values from `foo/bar/meta.json`, but they can also
reference meta variables that are defined only in `foo/meta.json`. This also
means that templates in `foo/baz/` cannot reference variables defined only in
`foo/bar/meta.json`.
Each directory can have no more than one file named `meta.json`.
Running a specific test
~~~~~~~~~~~~~~~~~~~~~~~

57
doc/source/test-anatomy.rst
View File

@ -152,9 +152,7 @@ Syntribos allows for templates to read in variables from a user-specified
meta variable file. These files contain JSON objects that define variables
to be used in one or more request templates.
The file must be named `meta.json` and must be placed in the same
directory as the template files that reference it. Meta variable files take
the form:
The file must be named `meta.json`, and they take the form:
::
{
@ -164,11 +162,13 @@ the form:
"user_name": {
"type": config,
"val": "user.username"
"fuzz_types": ["ascii"]
},
"user_token": {
"type": "function",
"val": "syntribos.extensions.identity:get_scoped_token_v3",
"args": ["user"]
"args": ["user"],
"fuzz": false
}
}
@ -179,8 +179,6 @@ variables is as follows:
POST /user HTTP/1.1
X-Auth-Token: |user_token|
Accept: */*
Content-type: application/json
{
"user": {
@ -189,6 +187,53 @@ variables is as follows:
}
}
Note: Meta-variable usage in templates should take the form `|user_name|`, not
`user_|name|` or `|user|_|name|`. This is to avoid ambiguous behavior when the
value is fuzzed.
Meta Variable Attributes
------------------------
* val - All meta variable objects must define a value, which can be of any json
DataType. Unlike the other attributes, this attribute is not optional.
* type - Defining a type instructs syntribos to interpret the variable in a
certain way. Any variables without a type defined will be read in directly
from the value. The following types are allowed:
* config - syntribos reads the config value specified by the "val"
attribute and returns that value.
* function - syntribos calls the function named in the "val" attribute
with any arguments given in the optional "args" attribute, and returns the
value from calling the function. This value is cached, and will be returned
on subsequent calls.
* generator - Works the same way as the function type, but its results are
not cached and the function will be called every time.
* args - A list of function arguments (if any) which can be defined here if the
variable is a generator or a function
* fuzz - A boolean value that, if set to false, instructs syntribos to
ignore this variable for any fuzz tests
* fuzz_types - A list of strings which instructs syntribos to only use certain
fuzz strings when fuzzing this variable. More than one fuzz type can be
defined. The following fuzz types are allowed:
* ascii - strings that can be encoded as ascii
* url - strings that contain only url safe characters
* min_length/max_length - An integer that instructs syntribos to only use fuzz
strings that meet certain length requirements
Inheritence
-----------
Meta variable files inherit based on the directory it's in. That is, if you
have `foo/meta.json` and `foo/bar/meta.json`, templates in `foo/bar/` will take
their meta variable values from `foo/bar/meta.json`, but they can also
reference meta variables that are defined only in `foo/meta.json`. This also
means that templates in `foo/baz/` cannot reference variables defined only in
`foo/bar/meta.json`.
Each directory can have no more than one file named `meta.json`.
Running a specific test
~~~~~~~~~~~~~~~~~~~~~~~

122
syntribos/clients/http/parser.py
View File

@ -63,7 +63,7 @@ class RequestCreator(object):
break
if lines[index] != "":
index = index + 1
method, url, params, _ = cls._parse_url_line(lines[0], endpoint)
method, url, params, version = cls._parse_url_line(lines[0], endpoint)
headers = cls._parse_headers(lines[1:index])
data = cls._parse_data(lines[index + 1:])
return RequestObject(
@ -80,9 +80,9 @@ class RequestCreator(object):
object read in from meta.json
"""
if var not in cls.meta_vars:
print("Expected to find {0} in meta.json, but didn't. "
"Check your templates".format(var))
return
msg = _("Expected to find %s in meta.json, but didn't. "
"Check your templates") % var
raise TemplateParseException(msg)
var_dict = cls.meta_vars[var]
if "type" in var_dict:
var_dict["var_type"] = var_dict.pop("type")
@ -116,17 +116,18 @@ class RequestCreator(object):
try:
return reduce(getattr, var_obj.val.split("."), CONF)
except AttributeError:
print(_(
"Meta json file contains reference to the config option "
"%s, which does not appear to exist.") % var_obj.val)
msg = _("Meta json file contains reference to the config "
"option %s, which does not appear to"
"exist.") % var_obj.val
raise TemplateParseException(msg)
elif var_obj.var_type == 'function':
if var_obj.function_return_value:
return var_obj.function_return_value
if not var_obj.val:
print(_(
"The type of variable is function, but there is no "
"reference to the function."))
return
msg = _("The type of variable %s is function, but there is no "
"reference to the function.") % var_obj.name
raise TemplateParseException(msg)
else:
var_obj.function_return_value = cls.call_one_external_function(
var_obj.val, var_obj.args)
@ -134,10 +135,10 @@ class RequestCreator(object):
elif var_obj.var_type == 'generator':
if not var_obj.val:
print(_(
"The type of variable {0} is generator, but there is no "
"reference to the function."))
return
msg = _("The type of variable %s is generator, but there is no"
" reference to the function.") % var_obj.name
raise TemplateParseException(msg)
return cls.call_one_external_function(var_obj.val, var_obj.args)
else:
return str(var_obj.val)
@ -147,17 +148,24 @@ class RequestCreator(object):
"""Recursively evaluates all meta variables in a given dict."""
for (key, value) in dic.items():
# Keys dont get fuzzed, so can handle them here
new_key = key.strip("|%s" % string.whitespace)
if re.search(cls.METAVAR, key):
key_obj = cls._create_var_obj(new_key)
new_key = cls.replace_one_variable(key_obj)
match = re.search(cls.METAVAR, key)
if match:
replaced_key = match.group(0).strip("|")
key_obj = cls._create_var_obj(replaced_key)
replaced_key = cls.replace_one_variable(key_obj)
new_key = re.sub(cls.METAVAR, replaced_key, key)
del dic[key]
dic[new_key] = value
# Vals are fuzzed so they need to be passed to datagen as an object
if isinstance(value, six.string_types):
if re.search(cls.METAVAR, value):
value = value.strip("|%s" % string.whitespace)
val_obj = cls._create_var_obj(value)
match = re.search(cls.METAVAR, value)
if match:
var_str = match.group(0).strip("|")
if var_str != value.strip("|%s" % string.whitespace):
msg = _("Meta-variable references cannot come in the "
"middle of the value %s") % value
raise TemplateParseException(msg)
val_obj = cls._create_var_obj(var_str)
if key in dic:
dic[key] = val_obj
elif new_key in dic:
@ -214,7 +222,7 @@ class RequestCreator(object):
url = url[0]
url = urlparse.urljoin(endpoint, url)
if method not in valid_methods:
raise ValueError("Invalid HTTP method: {0}".format(method))
raise ValueError(_("Invalid HTTP method: %s") % method)
return (method, cls._replace_str_variables(url),
cls._replace_dict_variables(params), version)
@ -253,12 +261,16 @@ class RequestCreator(object):
return cls._replace_dict_variables(data)
else:
return cls._replace_str_variables(data)
except Exception:
except TemplateParseException:
raise
except (TypeError, ValueError):
try:
data = ElementTree.fromstring(data)
except Exception:
if not re.match(postdat_regex, data):
raise TypeError(_("Unknown data format"))
except Exception:
raise
return data
@classmethod
@ -301,19 +313,44 @@ class RequestCreator(object):
if not match:
match = re.search(cls.FUNC_WITH_ARGS, string)
func_string_has_args = True
if not match:
print(_("The reference to the function %s failed to parse "
"correctly") % string)
return
dot_path = match.group(1)
func_name = match.group(2)
mod = importlib.import_module(dot_path)
func = getattr(mod, func_name)
if func_string_has_args and not args:
arg_list = match.group(3)
args = json.loads(arg_list)
val = func(*args)
if match:
try:
dot_path = match.group(1)
func_name = match.group(2)
mod = importlib.import_module(dot_path)
func = getattr(mod, func_name)
if func_string_has_args and not args:
arg_list = match.group(3)
args = json.loads(arg_list)
val = func(*args)
except Exception:
msg = _("The reference to the function %s failed to parse "
"correctly, please check the documentation to ensure "
"your function import string adheres to the proper "
"format") % string
raise TemplateParseException(msg)
else:
try:
func_lst = string.split(":")
if len(func_lst) == 2:
args = func_lst[1]
func_str = func_lst[0]
dot_path = ".".join(func_str.split(".")[:-1])
func_name = func_str.split(".")[-1]
mod = importlib.import_module(dot_path)
func = getattr(mod, func_name)
val = func(*args)
except Exception:
msg = _("The reference to the function %s failed to parse "
"correctly, please check the documentation to ensure "
"your function import string adheres to the proper "
"format") % string
raise TemplateParseException(msg)
if isinstance(val, types.GeneratorType):
return str(six.next(val))
else:
@ -328,10 +365,11 @@ class VariableObject(object):
fuzz_types=[], min_length=0, max_length=sys.maxsize,
url_encode=False, **kwargs):
if var_type and var_type.lower() not in self.VAR_TYPES:
print(_(
"The variable %(name)s has a type of %(var)s which"
" syntribos does not"
" recognize") % {'name': name, 'var': var_type})
msg = _("The meta variable %(name)s has a type of %(var)s which "
"syntribos does not"
"recognize") % {'name': name, 'var': var_type}
raise TemplateParseException(msg)
self.name = name
self.var_type = var_type.lower()
self.val = val
@ -347,6 +385,10 @@ class VariableObject(object):
return str(vars(self))
class TemplateParseException(Exception):
pass
class RequestHelperMixin(object):
"""Class that helps with fuzzing requests."""

44
syntribos/runner.py
View File

@ -159,6 +159,26 @@ class Runner(object):
else:
cls.output = sys.stdout
@classmethod
def get_meta_vars(cls, file_path):
"""Creates the appropriate meta_var dict for the given file path
Meta variables are inherited according to directory. This function
builds a meta variable dict from the top down.
:param file_path: the path of the current template
:returns: `dict` of meta variables
"""
path_segments = [""] + os.path.dirname(file_path).split(os.sep)
meta_vars = {}
current_path = ""
for seg in path_segments:
current_path = os.path.join(current_path, seg)
if current_path in cls.meta_dir_dict:
for k, v in cls.meta_dir_dict[current_path].items():
meta_vars[k] = v
return meta_vars
@classmethod
def run(cls):
"""Method sets up logger and decides on Syntribos control flow
@ -228,21 +248,21 @@ class Runner(object):
exit(1)
print(_("\nPress Ctrl-C to pause or exit...\n"))
# TODO(mdong): Make this handle inheritence and all that. For now, just
# pass the first meta file it sees to the parser. Also, find a better
# way to pass meta_vars
meta_vars = None
templates_dir = list(templates_dir)
for file_path, req_str in templates_dir:
if "meta.json" not in file_path:
continue
else:
meta_vars = json.loads(req_str)
break
cls.meta_dir_dict = {}
for file_path, file_content in templates_dir:
if os.path.basename(file_path) == "meta.json":
meta_path = os.path.dirname(file_path)
try:
cls.meta_dir_dict[meta_path] = json.loads(file_content)
except Exception:
print("Unable to parse %s, skipping..." % file_path)
for file_path, req_str in templates_dir:
if "meta.json" in file_path:
continue
meta_vars = cls.get_meta_vars(file_path)
LOG = cls.get_logger(file_path)
CONF.log_opt_values(LOG, logging.DEBUG)
if not file_path.endswith(".template"):
@ -294,10 +314,10 @@ class Runner(object):
"""
for k, test_class in list_of_tests: # noqa
try:
print("\nParsing template file...")
print("\nParsing template file...\n")
test_class.create_init_request(file_path, req_str, meta_vars)
except Exception as e:
print("Error in parsing template:\n \t{0}\n".format(
print("\nError in parsing template:\n \t{0}\n".format(
traceback.format_exc()))
LOG.error(_LE("Error in parsing template:"))
output["failures"].append({
@ -305,7 +325,7 @@ class Runner(object):
"error": e.__str__()
})
else:
print(_("Request sucessfully generated!\n"))
print(_("\nRequest sucessfully generated!\n"))
output["successes"].append(file_path)
test_cases = list(test_class.get_test_cases(file_path, req_str))

2
syntribos/tests/base.py
View File

@ -246,8 +246,6 @@ class BaseTestCase(unittest.TestCase):
confidence=confidence,
description=description)
# Still associating request and response objects with issue in event of
# debug log
issue.request = self.test_req
issue.response = self.test_resp

4
syntribos/tests/headers/xst.py
View File

@ -70,6 +70,4 @@ class XstHeader(base.BaseTestCase):
confidence=syntribos.HIGH,
description=(_("XST vulnerability found.\n"
"Make sure that response to a "
"TRACE request is filtered."
)
))
"TRACE request is filtered.")))