DataFrame.__repr__ raises TypeError after pd.show_versions() was run

[xflr6]

Maybe one of the imports in show_versions has unwanted side effects?

>>> import pandas as pd
>>> pd.DataFrame({'spam': range(10)})
   spam
0     0
1     1
2     2
3     3
4     4
5     5
6     6
7     7
8     8
9     9
>>> pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.12.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 42 Stepping 7, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.18.1
nose: 1.3.7
pip: 8.1.2
setuptools: 24.0.3
Cython: 0.24.1
numpy: 1.11.1
scipy: 0.18.0rc2
statsmodels: None
xarray: None
IPython: 5.0.0
sphinx: 1.4.5
patsy: 0.4.1
dateutil: 2.5.3
pytz: 2016.6.1
blosc: None
bottleneck: 1.1.0
tables: None
numexpr: 2.6.0
matplotlib: 1.5.1
openpyxl: 2.3.5
xlrd: 1.0.0
xlwt: None
xlsxwriter: None
lxml: 3.6.0
bs4: None
html5lib: 0.999999999
httplib2: 0.9.2
apiclient: None
sqlalchemy: 1.0.14
pymysql: None
psycopg2: 2.6.2 (dt dec pq3 ext lo64)
jinja2: 2.8
boto: None
pandas_datareader: None
>>> pd.DataFrame({'spam': range(10)})

Traceback (most recent call last):
  File "<pyshell#3>", line 1, in <module>
    pd.DataFrame({'spam': range(10)})
  File "C:\Program Files\Python27\lib\site-packages\pandas\core\base.py", line 67, in __repr__
    return str(self)
  File "C:\Program Files\Python27\lib\site-packages\pandas\core\base.py", line 47, in __str__
    return self.__bytes__()
  File "C:\Program Files\Python27\lib\site-packages\pandas\core\base.py", line 59, in __bytes__
    return self.__unicode__().encode(encoding, 'replace')
  File "C:\Program Files\Python27\lib\site-packages\pandas\core\frame.py", line 535, in __unicode__
    line_width=width, show_dimensions=show_dimensions)
  File "C:\Program Files\Python27\lib\site-packages\pandas\core\frame.py", line 1488, in to_string
    formatter.to_string()
  File "C:\Program Files\Python27\lib\site-packages\pandas\formats\format.py", line 549, in to_string
    strcols = self._to_str_columns()
  File "C:\Program Files\Python27\lib\site-packages\pandas\formats\format.py", line 467, in _to_str_columns
    str_index = self._get_formatted_index(frame)
  File "C:\Program Files\Python27\lib\site-packages\pandas\formats\format.py", line 746, in _get_formatted_index
    fmt_index = [index.format(name=show_index_names, formatter=fmt)]
  File "C:\Program Files\Python27\lib\site-packages\pandas\indexes\base.py", line 1462, in format
    return self._format_with_header(header, **kwargs)
  File "C:\Program Files\Python27\lib\site-packages\pandas\indexes\base.py", line 1486, in _format_with_header
    result = _trim_front(format_array(values, None, justify='left'))
  File "C:\Program Files\Python27\lib\site-packages\pandas\formats\format.py", line 2007, in format_array
    return fmt_obj.get_result()
  File "C:\Program Files\Python27\lib\site-packages\pandas\formats\format.py", line 2027, in get_result
    return _make_fixed_width(fmt_values, self.justify)
  File "C:\Program Files\Python27\lib\site-packages\pandas\formats\format.py", line 2394, in _make_fixed_width
    max_len = np.max([adj.len(x) for x in strings])
  File "C:\Program Files\Python27\lib\site-packages\numpy\core\fromnumeric.py", line 2293, in amax
    out=out, **kwargs)
  File "C:\Program Files\Python27\lib\site-packages\numpy\core\_methods.py", line 26, in _amax
    return umr_maximum(a, axis, None, out, keepdims)
TypeError: an integer is required

[jorisvandenbossche]

I cannot reproduce this using Windows and python 2.7.

Could you try to debug this? To see where the error is coming from (seems there is something wrong with the np.max([adj.len(x) for x in strings])).
Or eg create an isolated environment with only required dependencies to see of the problem occurs there as well (using conda or virtualenv).

[xflr6]

Tracked it to this import of numpy, which according to the docs does a reload().

Indeed numpy (at least on my machines) seems to dislike being reloaded:

>>> import numpy as np
>>> np.max([42])
42
>>> reload(np)
<module 'numpy' from 'C:\Program Files\Python27\lib\site-packages\numpy\__init__.pyc'>
>>> np.max([42])

Traceback (most recent call last):
  File "<pyshell#3>", line 1, in <module>
    np.max([42])
  File "C:\Program Files\Python27\lib\site-packages\numpy\core\fromnumeric.py", line 2293, in amax
    out=out, **kwargs)
  File "C:\Program Files\Python27\lib\site-packages\numpy\core\_methods.py", line 26, in _amax
    return umr_maximum(a, axis, None, out, keepdims)
TypeError: an integer is required

The line was changed in b4e2d34, so I guess that should fix this for the next version (though I am still curious if others have this numpy issue with reload).

[sinhrks]

I can't reproduce it using NumPy 1.10.4 and 1.11.1. Can you report it to NumPy?

[xflr6]

Seems to be a problem with the binaries here (does not occur with the PyPI wheels): I'll report to the packager, closing.

[xflr6]

The numpy issue: numpy/numpy#7844

[jorisvandenbossche]

@xflr6 Thanks for tracking it down!

[njsmith]

I think there is a real pandas bug here. The bug is that show_versions calls importlib.import_module, and apparently -- this is not documented anywhere, and may vary between py2 and py3 -- import_module may reload modules. show_versions should not be reloading all these modules. I'd suggest replacing that line with something like

if modname in sys.modules:
    mod = sys.modules[modname]
else:
    mod = importlib.import_module(modname)

[jreback]

@njsmith but seems numpy is not robust to being reloaded.

[charris]

@jreback I suspect a lot of modules are not robust against reloading. For instance if you define a class in foo.py, use it in bar.py, instanciate a = foo.MyClass() in another module, then use isinstance(a, foo.MyClass) that statement will fail if foo is reloaded. I suspect what is wanted in many cases is a simple import, but I haven't checked. Numpy also uses load_module in a few places that should probably be audited.

@njsmith The reload property is documented in the imp module documentation. It always happens for existing modules. Note that nonexisting modules get created...