Cannot install pandas 0.24.2 from source with Python 2.7 into an environment without NumPy

[didip]

Code Sample, a copy-pastable example if possible

RuntimeError: Python version >= 3.5 required.

Problem description

Under this docs: https://pandas.pydata.org/pandas-docs/stable/whatsnew/v0.24.0.html

0.24.x is supposed to be backward compatible to Python 2.

The 0.24.x series of releases will be the last to support Python 2. Future feature releases will support Python 3 only. See Plan for dropping Python 2.7 for more details

Expected Output

0.24.2 should still work on Python 2.

[jreback]

are you sure you actually tried to install 0.24.x? and not the 0.25rc1

[didip]

Looks like it is trying to download numpy that is really new.

Our specified version for numpy is numpy==1.16.2

      File "/tmp/pip-install-E4jz9w/pandas/setup.py", line 746, in <module>
        **setuptools_kwargs)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/__init__.py", line 144, in setup
        _install_setup_requires(attrs)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/dist.py", line 724, in fetch_build_eggs
        replace_conflicting=True,
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
        replace_conflicting=replace_conflicting
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
        return self.obtain(req, installer)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
        return installer(requirement)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/dist.py", line 791, in fetch_build_egg
        return cmd.easy_install(req)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 679, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 705, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 890, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1158, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1144, in run_setup
        run_setup(setup_script, args)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
        raise
      File "/code/.cicd/runtime/python/.python/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/code/.cicd/runtime/python/.python/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
        saved_exc.resume()
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 141, in resume
        six.reraise(type, exc, self._tb)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
        yield saved
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
        _execfile(setup_script, ns)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-dglvOb/numpy-1.17.0rc2/setup.py", line 31, in <module>
    
    RuntimeError: Python version >= 3.5 required.

[jreback]

right so numpy 1.17 is also python 3 only. guess would be that you have a too old pip, or are doing some odd type of installation.

[didip]

Inside setup.py, should this block of logic check for Python version first and set max version of numpy for Python 2?

min_numpy_ver = "1.13.3"
setuptools_kwargs = {
    "install_requires": [
        "python-dateutil >= 2.6.1",
        "pytz >= 2017.2",
        "numpy >= {numpy_ver}".format(numpy_ver=min_numpy_ver),
    ],
    "setup_requires": ["numpy >= {numpy_ver}".format(numpy_ver=min_numpy_ver)],
    "zip_safe": False,
}

[TomAugspurger]

Can you show the command you're running? You shouldn't be executing setup.py, pip would be doing it for you if needed.

[didip]

Fairly standard procedure: pip install -r requirements.txt.

...
numpy==1.16.2
pandas==0.24.2
...

During pip install, when it's collecting pandas...

Collecting pandas==0.24.2... 

pandas ended up downloading much newer numpy: numpy-1.17.0rc2 instead of what I specified in requirements.txt: numpy==1.16.2. numpy==1.16.2 is downloaded correctly.

[TomAugspurger]

What platform? Are you passing --pre or --no-binary anywhere? I'm confused why you aren't getting a wheel for NumPy and pandas.

[TomAugspurger]

You might also post the full output.

[didip]

Collecting pandas==0.24.2 (from redacted==0.0.0)
  Downloading https://pypi.redacted.com/packages/b2/4c/b6f966ac91c5670ba4ef0b0b5613b5379e3c7abdfad4e7b89a87d73bae13/pandas-0.24.2.tar.gz (11.8MB)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-E4jz9w/pandas/setup.py", line 746, in <module>
        **setuptools_kwargs)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/__init__.py", line 144, in setup
        _install_setup_requires(attrs)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/dist.py", line 724, in fetch_build_eggs
        replace_conflicting=True,
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
        replace_conflicting=replace_conflicting
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
        return self.obtain(req, installer)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
        return installer(requirement)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/dist.py", line 791, in fetch_build_egg
        return cmd.easy_install(req)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 679, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 705, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 890, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1158, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1144, in run_setup
        run_setup(setup_script, args)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
        raise
      File "/code/.cicd/runtime/python/.python/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/code/.cicd/runtime/python/.python/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
        saved_exc.resume()
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 141, in resume
        six.reraise(type, exc, self._tb)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
        yield saved
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
        _execfile(setup_script, ns)
      File "/code/.cicd/runtime/python/.python/lib/python2.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-dglvOb/numpy-1.17.0rc2/setup.py", line 31, in <module>
    
    RuntimeError: Python version >= 3.5 required.

requirements.txt

blinker==1.4
Flask==0.10.1
Flask-KVSession==0.4
Flask-Statsd-Ext==0.0.3
gevent==1.1.1
greenlet==0.4.9
gunicorn==19.4.5
itsdangerous==0.24
Jinja2==2.8
MarkupSafe==0.23
mysql-connector-python-rf==2.0.4
numpy==1.16.2
pandas==0.24.2
psycopg2==2.7.1
pyjwt==1.6.4
pylru==1.0.9
python-dateutil==2.5.0
python-ldap==2.3.13
pytest==3.3.0
pytest-cov==2.5.1
pytz==2016.3
raven==6.6.0
redis==2.10.5
requests==2.10.0
simplekv==0.7.1.dev1
six==1.10.0
SQLAlchemy==1.3.3
Werkzeug==0.11.9
Unidecode==0.4.21

setup.py

from setuptools import setup, find_packages


import os

here = os.path.abspath(os.path.dirname(__file__))
README = open(os.path.join(here, 'README.md')).read()

exclude_dirs = ['venv', 'scripts']

with open('requirements.txt') as f:
    install_requires = f.read().splitlines()


setup(name='redacted',
    long_description=README,
    packages=find_packages(exclude=exclude_dirs),
    include_package_data=True,
    scripts=[],

    install_requires=install_requires,
    dependency_links=[],
    tests_require=[],
    zip_safe=False
)

And we call python setup.py test to build our app, installing dependencies, etc.

Note that I am redacting a lot of business logic related stuff, but the flow looks like that.

[TomAugspurger]

So pip install -r requirements.txt works?

[didip]

pip install -r requirements.txt failed because pandas tried to download numpy-1.17.0rc2

[didip]

Should the setup.py be updated like this? Essentially check Python 2 version?

import sys

min_numpy_ver = "1.13.3"
max_numpy_ver = ""

if (sys.version_info < (3, 0)):
    max_numpy_ver = "1.16.4"

numpy_install_rule = "numpy >= {numpy_ver}".format(numpy_ver=min_numpy_ver)
if max_numpy_ver:
    numpy_install_rule += ", <= {numpy_ver}".format(numpy_ver=max_numpy_ver)

setuptools_kwargs = {
    "install_requires": [
        "python-dateutil >= 2.6.1",
        "pytz >= 2017.2",
        numpy_install_rule,
    ],
    "setup_requires": [numpy_install_rule],
    "zip_safe": False,
}

[didip]

The above snippet works on our internal fork of pandas. All of our Py 2 apps have correct dependencies now.
What do you guys think of updating the setup.py?

[jreback]

there are no more updates planned to the 0.24.x branch

closing this issue as won't fix.