Unicode docstrings are not handled correctly on Python 2.x

[mdboom]

If a docstring is Unicode on Python 2.x, Numpydoc will crash with a UnicodeEncodeError. This is because it calls str on the SphinxDocString object, calling SphinxDocString.__str__ which returns unicode when the docstring itself is unicode. Python then fails converting that unicode object to a bytes (str) object using the default encoding.

# -*- coding: utf-8 -*-

docstring = u"""
numpydoc הוא מדהים
"""

from numpydoc import docscrape_sphinx

d = docscrape_sphinx.SphinxDocString(docstring)

x = str(d)  # fails
x = unicode(d)  # works, whether docstring is unicode or bytes

This PR restores the behavior in the numpydoc that ships with Numpy 1.7.1, which was to keep unicode as-is, and convert bytes to unicode using the default encoding.

[stefanv]

+1

[jseabold]

+1

[mdboom]

Any chance we can have this merged? As matplotlib has doctsrings with unicode, there have been a few reports over there about this.