The Py2 version of the csv module cannot accept unicode text. In Py2
we replaced the csv module with the unicodecsv module which encodes
unicode text prior to calling the standard Py2 csv module. Thus when
the cvs formatted output is emitted on stdout it is presented as a
encoded byte stream. But stdout has been replaced with a StreamWriter
which encodes to the desired encoding. The problem is the StreamWriter
attempts to encode all objects passed to it's write function,
including str objects. Instead it should only encode unicode text objects
and allow bytes to pass through unmodified.
This patch adds an override of the codecs.getwriter function which
only encodes unicode text objects. In addtion we pass the encoding
value obtained from the stream to the unicodecsv writer.
The patch fixes the codec error when outputing csv formated text that
contains a non-ASCII character. The unicodecsv implmentation will emit
byte encoded str objects to the stream. When the core StreamWriter
attempts to encode a str object Python will first promote the str
object to a unicode object. The promotion of str to unicode requires
the str bytes to be decoded. However the encoding associated with the
str object is not known therefore Python applies the default-encoding
which is ASCII. In the case where the str object contains utf-8
encoded non-ASCII characters a decoding error is raised. By not
attempting to encode a byte stream we avoid this error.
A more complete discussion of the above issues can be found here:
https://github.com/fedora-infra/kitchen/blob/develop/kitchen2/docs/unicode-frustrations.rst#frustration-4-now-it-doesnt-take-byte-strings
Change-Id: I22b5ad8bf0e227ec75a2a36986f0487191f7cbc2
Closes-Bug: 1720115
Signed-off-by: John Dennis <jdennis@redhat.com>
(cherry picked from commit c61cc30060)
35f3820855