update pylint to 1.9.2

The older version of pylint being used does not work correctly under
python 3. In order to be able to update the pylint job to run under
python 3, we need to update the tool.

This patch updates to the latest version at this time. It also updates
and pins astroid, which was previously capped. Using a pin instead of
a cap should let us avoid issues with new releases while being
specific about which version to actually use.

The import-error linter rule is disabled because it appears in
Windows-related code that looks for modules we don't expect to be able
to find on the CI system under Linux.

The non-iterator-returned rule is disabled because it is erroring on a
class that doesn't make sense (freezer.utils.utils.ReSizeStream does
appear to honor the iterator protocol). Further investigation is
probably warranted on this one.

The public entry point for multiprocessing.SimpleQueue is the
top-level multiprocessing module, not multiprocessing.queues. The
linter correctly caught this error because the ctx argument was not
being passed to the class. The fix is to use the top-level entry point
to access the class.

basestring is no longer defined under python 3. The fix is to use
six.string_types, which correctly sets the alias to str or
basestring. By using the library, we can avoid having to tell the
linter to ignore the local customization.

The linter also caught a base raise, without an exception specified
and not in the context of an exception handler. I added an exception
using the pattern from the one on an earlier line.

Change-Id: I349de35c9ee52e2946e661f777308444b61ba4e0
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Co-Authored-By: Nguyen Hai <nguyentrihai93@gmail.com>

.pylintrc

@@ -44,7 +44,7 @@ symbols=no
 # --enable=similarities". If you want to run only the classes checker, but have
 # no Warning level messages displayed, use"--disable=all --enable=classes
 # --disable=W"
-disable=W,C,R,E1002,no-member
+disable=W,C,R,E1002,no-member,import-error,non-iterator-returned
 
 
 [REPORTS]

freezer/engine/engine.py

@@ -249,7 +249,10 @@ class BackupEngine(object):
         max_level = max(backups.keys())
 
         # Use SimpleQueue because Queue does not work on Mac OS X.
-        read_except_queue = queues.SimpleQueue()
+        if six.PY2:
+            read_except_queue = queues.SimpleQueue()
+        else:
+            read_except_queue = multiprocessing.SimpleQueue()
         LOG.info("Restoring backup {0}".format(hostname_backup_name))
         for level in range(0, max_level + 1):
             LOG.info("Restoring from level {0}".format(level))
@@ -266,7 +269,10 @@ class BackupEngine(object):
             # Start the tar pipe consumer process
 
             # Use SimpleQueue because Queue does not work on Mac OS X.
-            write_except_queue = queues.SimpleQueue()
+            if six.PY2:
+                write_except_queue = queues.SimpleQueue()
+            else:
+                write_except_queue = multiprocessing.SimpleQueue()
 
             engine_stream = multiprocessing.Process(
                 target=self.restore_level,

freezer/lib/pep3143daemon/daemon.py

@@ -32,12 +32,7 @@ import signal
 import socket
 import sys
 
-# PY2 / PY3 gap
-PY3 = sys.version_info[0] == 3
-if PY3:
-    string_types = str,
-else:
-    string_types = basestring,
+import six
 
 
 class DaemonError(Exception):
@@ -176,7 +171,7 @@ class DaemonContext(object):
         """
         if not handler:
             result = signal.SIG_IGN
-        elif isinstance(handler, string_types):
+        elif isinstance(handler, six.string_types):
             result = getattr(self, handler)
         else:
             result = handler

freezer/openstack/admin.py

@@ -69,7 +69,10 @@ class AdminOs(object):
                                                          start_time > timeout):
                 LOG.error("Delete backup %s failed, In a state of"
                           "deleting over 120s")
-                raise
+                raise Exception(
+                    "Delete backup %s failed due to timeout over 120s, "
+                    "the status of backup is %s."
+                    % (backup_id, del_backup[0].status))
         timer = loopingcall.FixedIntervalLoopingCall(wait_del_backup)
         timer.start(interval=0.5).wait()
 

test-requirements.txt

@@ -6,13 +6,13 @@ hacking!=0.13.0,<0.14,>=0.12.0 # Apache-2.0
 coverage!=4.4,>=4.0 # Apache-2.0
 ddt>=1.0.1 # MIT
 mock>=2.0.0 # BSD
-pylint==1.4.5 # GPLv2
+pylint==1.9.2 # GPLv2
 sphinx!=1.6.6,!=1.6.7,>=1.6.2 # BSD
 openstackdocstheme>=1.18.1 # Apache-2.0
 stestr>=2.0.0 # Apache-2.0
 testtools>=2.2.0 # MIT
 reno>=2.5.0 # Apache-2.0
-astroid<1.4.0 # LGPLv2.1 # breaks pylint 1.4.4
+astroid==1.6.5 # LGPLv2.1
 
 # Tempest Plugin
 tempest>=17.1.0 # Apache-2.0