Ignore heredoc contents

I noticed that we end up treating a heredoc as one big logical line,
then passing it through things like the line-length check.  So

cat <<EOF
this is normal lines
that get picked up as a long line!
EOF

end up triggering the long-line exception ... clearly not what is
intended.

In fact, there's not much we can do with heredocs.  They are usually
some weird syntax for a config file, or something else you're writing
out in bulk.  Often they have offsets that just don't match what we
want, etc.

Thus most logical thing to do seems to be to just ignore whatever is
in the heredoc.

I also noticed we don't handle <<'EOF', which is another common idiom,
so add that to the regex.  Also add a test-case to ensure we're
ignoring the heredoc contents.

Change-Id: I2e3d66f27ff4c0d1f542511ca3e19c29a596896f

bashate/bashate.py

@@ -87,7 +87,9 @@ def check_function_decl(line, report):
 
 
 def starts_multiline(line):
-    m = re.search("[^<]<<\s*(?P<token>\w+)", line)
+    # note, watch out for <<EOF and <<'EOF' ; quotes in the
+    # deliminator are part of syntax
+    m = re.search("[^<]<<\s*([\'\"]?)(?P<token>\w+)([\'\"]?)", line)
     return m.group('token') if m else False
 
 
@@ -257,7 +259,10 @@ class BashateRun(object):
                     if verbose:
                         print("Running bashate on %s" % fileinput.filename())
 
-                # NOTE(sdague): multiline processing of heredocs is interesting
+                # strip out heredocs, and don't run any checks on
+                # their contents.  These are usually things like yaml
+                # files or other bits and pieces that don't obey our
+                # syntax such as indenting or line-length.
                 if not in_multiline:
                     logical_line = line
                     token = starts_multiline(line)
@@ -270,6 +275,11 @@ class BashateRun(object):
                         continue
                     else:
                         in_multiline = False
+                        # XXX: if we want to do something with
+                        # heredocs in the future, then the whole thing
+                        # is now stored in logical_line.  for now,
+                        # skip
+                        continue
 
                 # Don't run any tests on comment lines
                 if logical_line.lstrip().startswith('#'):

bashate/tests/samples/heredoc_ignore.sh

@@ -0,0 +1,42 @@
+#!/bin/bash
+
+FOO=<<EOF
+
+ this is a file
+  that does not obey our indenting
+ or our line length -------------------------------------------------
+
+EOF
+
+cat <<EOF > /tmp/tofile
+
+ this is a file
+  that does not obey our indenting
+ or our line length -------------------------------------------------
+
+EOF
+
+cat << 'EOF' | sed 's/foo/bar'
+
+ this is a file
+  that does not obey our indenting
+ or our line length -------------------------------------------------
+
+EOF
+
+cat <<"EOF"
+
+ this is a file
+  that does not obey our indenting
+ or our line length -------------------------------------------------
+
+EOF
+
+cat > foo <<BLAH
+
+ this is a file
+  that does not obey our indenting
+ or our line length -------------------------------------------------
+
+
+BLAH

bashate/tests/test_bashate.py

@@ -249,6 +249,11 @@ class TestBashateSamples(base.TestCase):
         self.assertEqual(0, self.run.error_count)
         self.assertEqual(4, self.run.warning_count)
 
+    def test_ignore_heredoc(self):
+        test_files = ['bashate/tests/samples/heredoc_ignore.sh']
+        self.run.check_files(test_files, False)
+        self.assertEqual(0, self.run.error_count)
+
     def test_pre_zero_dot_one_sample_file(self):
         """Test the sample file with all pre 0.1.0 release checks.