CI: Build failures

[TomAugspurger]

e.g. https://travis-ci.org/pandas-dev/pandas/jobs/326347971

[script_single]
pytest -s -m single --strict --cov=pandas --cov-report xml:/tmp/cov-single.xml --junitxml=/tmp/single.xml --skip-slow --skip-network pandas
Traceback (most recent call last):
  File "/home/travis/miniconda3/envs/pandas/lib/python3.6/site-packages/_pytest/config.py", line 328, in _getconftestmodules
    return self._path2confmods[path]
KeyError: local('/home/travis/build/pandas-dev/pandas/pandas')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/miniconda3/envs/pandas/lib/python3.6/site-packages/_pytest/config.py", line 359, in _importconftest
    return self._conftestpath2mod[conftestpath]
KeyError: local('/home/travis/build/pandas-dev/pandas/pandas/conftest.py')
During handling of the above exception, another exception occurred:
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
Traceback (most recent call last):
  File "/home/travis/miniconda3/envs/pandas/lib/python3.6/site-packages/_pytest/config.py", line 328, in _getconftestmodules
    return self._path2confmods[path]
KeyError: local('/home/travis/build/pandas-dev/pandas/pandas')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/miniconda3/envs/pandas/lib/python3.6/site-packages/_pytest/config.py", line 359, in _importconftest
    return self._conftestpath2mod[conftestpath]
KeyError: local('/home/travis/build/pandas-dev/pandas/pandas/conftest.py')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/build/pandas-dev/pandas/pandas/__init__.py", line 26, in <module>
    from pandas._libs import (hashtable as _hashtable,
  File "/home/travis/build/pandas-dev/pandas/pandas/_libs/__init__.py", line 4, in <module>
    from .tslib import iNaT, NaT, Timestamp, Timedelta, OutOfBoundsDatetime
  File "pandas/_libs/tslibs/conversion.pxd", line 11, in init pandas._libs.tslib
    cdef class _TSObject:
  File "pandas/_libs/tslibs/conversion.pyx", line 1, in init pandas._libs.tslibs.conversion
    # -*- coding: utf-8 -*-
ImportError: numpy.core.multiarray failed to import
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/miniconda3/envs/pandas/lib/python3.6/site-packages/_pytest/config.py", line 365, in _importconftest
    mod = conftestpath.pyimport()
  File "/home/travis/miniconda3/envs/pandas/lib/python3.6/site-packages/py/_path/local.py", line 668, in pyimport
    __import__(modname)
  File "/home/travis/build/pandas-dev/pandas/pandas/__init__.py", line 35, in <module>
    "the C extensions first.".format(module))
ImportError: C extension: numpy.core.multiarray failed to import not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
ERROR: could not load /home/travis/build/pandas-dev/pandas/pandas/conftest.py

Looking into it now.

[TomAugspurger]

Looks like we compiled against numpy 1.14 but were downgraded to 1.11.

[TomAugspurger]

https://travis-ci.org/pandas-dev/pandas/jobs/326153864#L2059

The following packages will be DOWNGRADED:
    numpy:                         1.14.0-py36_blas_openblas_200 conda-forge [blas_openblas] --> 1.13.3-py36_blas_openblas_201 conda-forge [blas_openblas]

[TomAugspurger]

Looks like it's numba:

 numba-0.36.2               |np113py36h7a10136_0

[TomAugspurger]

Which is presumably being pulled in through fastparquet.

[earthgecko]

I got this error on 2018-01-05 as well with Python-2.7 pandas==0.22.0 and numpy==1.13.3 breaking with the same error

  File "/opt/python_virtualenv/projects/skyline-luminosity-py2714/lib/python2.7/site-packages/pandas/__init__.py", line 35, in <module>
    "the C extensions first.".format(module))
ImportError: C extension: numpy.core.multiarray failed to import not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.

As per:

• ImportError: C extension: numpy.core.multiarray failed to import not built. #18281 - ImportError: C extension: numpy.core.multiarray failed to import not built
• 0.21.0 wheels broken for older (<1.13) numpy version #18530 (comment)

I rebuilt the virtualenv and explicitly installed numpy first and it was still broken and in my app log I found gunicorn report

RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
...
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
Traceback (most recent call last):
  File "/opt/skyline/github/skyline/skyline/crucible/agent.py", line 16, in <module>
    from crucible import Crucible
  File "/opt/skyline/github/skyline/skyline/crucible/crucible.py", line 41, in <module>
    from crucible_algorithms import run_algorithms
  File "/opt/skyline/github/skyline/skyline/crucible/crucible_algorithms.py", line 2, in <module>
    import pandas
  File "/opt/python_virtualenv/projects/skyline-luminosity-py2714/lib/python2.7/site-packages/pandas/__init__.py", line 35, in <module>
    "the C extensions first.".format(module))
ImportError: C extension: numpy.core.multiarray failed to import not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
error - failed to start crucible

I thought seeing as it was said to have been fixed in 0.21 it must be me :)
I figured that I would roll back to 0.20.3 and keep an eye on "numpy.core.multiarray" - https://github.com/pandas-dev/pandas/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+numpy.core.multiarray
And I found this, so I thought I would share as this inflicted 2.7 as well and this suggests it was not me just me or my env :) but possibly something deeper.

[TomAugspurger]

@earthgecko this is just for our CI.

what version of python are you on, and how did you install numpy and pandas?

[TomAugspurger]

Oh, I see 2.7 in your virtualenv name. You installed numpy=1.13 before installing pandas?

[TomAugspurger]

Can you paste the output of pip install numpy and pip install pandas on your machine?

[TomAugspurger]

e.g.

$ docker run -it python bash                                                                                                                                                                                                                                                              root@63b72e7e2cf8:/# python
root@63b72e7e2cf8:/# python3 -m pip install numpy==1.13.3
Collecting numpy==1.13.3
  Downloading numpy-1.13.3-cp36-cp36m-manylinux1_x86_64.whl (17.0MB)
    100% |████████████████████████████████| 17.0MB 99kB/s
Installing collected packages: numpy
Successfully installed numpy-1.13.3
root@63b72e7e2cf8:/# python3 -m pip install pandas===0.22.0
Collecting pandas===0.22.0
  Downloading pandas-0.22.0-cp36-cp36m-manylinux1_x86_64.whl (26.2MB)
    100% |████████████████████████████████| 26.3MB 66kB/s
Collecting pytz>=2011k (from pandas===0.22.0)
  Downloading pytz-2017.3-py2.py3-none-any.whl (511kB)
    100% |████████████████████████████████| 512kB 2.0MB/s
Requirement already satisfied: numpy>=1.9.0 in /usr/local/lib/python3.6/site-packages (from pandas===0.22.0)
Collecting python-dateutil>=2 (from pandas===0.22.0)
  Downloading python_dateutil-2.6.1-py2.py3-none-any.whl (194kB)
    100% |████████████████████████████████| 194kB 2.5MB/s
Collecting six>=1.5 (from python-dateutil>=2->pandas===0.22.0)
  Downloading six-1.11.0-py2.py3-none-any.whl
Installing collected packages: pytz, six, python-dateutil, pandas
Successfully installed pandas-0.22.0 python-dateutil-2.6.1 pytz-2017.3 six-1.11.0
root@63b72e7e2cf8:/# python
Python 3.6.4 (default, Dec 21 2017, 01:35:12)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas

[earthgecko]

Yes thanks now 0.22.0 works for me now, why did it not work the other day?

I think I had done some change in my requirements and gone through 2 numpy version and not reinstalled pandas or seems used --no-cache-dir with pip which reflecting on this info and some more debugging is probably the case.