RuntimeError: Python version >= 3.5 required.

[tysonclugg]

Environment

• pip version: 19.1.1
• Python version: 2.7.12
• OS: Ubuntu 16.04

Description
Fails to install packages from source where install_requires includes versions have incompatible python_requires.

Expected behavior
Source package should be installed with compatible versions of packages listed in install_requires from setup.py. Eg: Installing pandas in Python 2.7 should result in numpy version 1.16.4 being installed (the last 2.7 compatible release before 1.17.0rc1).

How to Reproduce

1. Run pip install --no-binary=pandas pandas in a Python 2.7 environment.
2. An error occurs.

Output

$ mkvirtualenv -ppython2.7 pip27
Running virtualenv with interpreter /usr/bin/python2.7
New python executable in /home/tclugg/.virtualenvs/pip27/bin/python2.7
Also creating executable in /home/tclugg/.virtualenvs/pip27/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /home/tclugg/.virtualenvs/pip27/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/tclugg/.virtualenvs/pip27/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/tclugg/.virtualenvs/pip27/bin/preactivate
virtualenvwrapper.user_scripts creating /home/tclugg/.virtualenvs/pip27/bin/postactivate
virtualenvwrapper.user_scripts creating /home/tclugg/.virtualenvs/pip27/bin/get_env_details
Python 2.7.12
(pip27) $ pip install -U pip
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Requirement already up-to-date: pip in /home/tclugg/.virtualenvs/pip27/lib/python2.7/site-packages (19.1.1)
(pip27) $ pip install --no-binary=pandas pandas
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Collecting pandas
  Using cached https://files.pythonhosted.org/packages/b2/4c/b6f966ac91c5670ba4ef0b0b5613b5379e3c7abdfad4e7b89a87d73bae13/pandas-0.24.2.tar.gz
    ERROR: Complete output from command python setup.py egg_info:
    ERROR: Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-rC8dFo/pandas/setup.py", line 746, in <module>
        **setuptools_kwargs)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/__init__.py", line 144, in setup
        _install_setup_requires(attrs)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/dist.py", line 717, in fetch_build_eggs
        replace_conflicting=True,
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
        replace_conflicting=replace_conflicting
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
        return self.obtain(req, installer)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
        return installer(requirement)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/dist.py", line 784, in fetch_build_egg
        return cmd.easy_install(req)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 679, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 705, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 890, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1158, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1144, in run_setup
        run_setup(setup_script, args)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
        raise
      File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
        saved_exc.resume()
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 141, in resume
        six.reraise(type, exc, self._tb)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
        yield saved
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
        _execfile(setup_script, ns)
      File "/home/tclugg/.virtualenvs/pip27/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-sfO_WV/numpy-1.17.0rc1/setup.py", line 31, in <module>
    
    RuntimeError: Python version >= 3.5 required.
    ----------------------------------------
ERROR: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-rC8dFo/pandas/

Paste the output of the steps above, including the commands themselves and
pip's output/traceback etc.

[xavfernandez]

pandas is still using setup_requires mechanism ( cf https://github.com/pandas-dev/pandas/blob/v0.24.2/setup.py#L40) which invokes easy_install behind pip's back, and apparently easy_install does not honor python_requires declarations.

You should install numpy first with pip that will honor python_requires, then install pandas.
And to definitely fix this kind of situation, pandas should start using PEP-518 build-system.requires to declare its build dependencies.

[tysonclugg]

@xavfernandez I understand what you're saying in that easy_install is effectively an escape hatch out from pip into easy_install.

I invoked pip, and was presented with a cryptic traceback. I am fortunate enough that I could understand enough to file this bug report. You've been kind enough to give me both a valid work-around, and made a good suggestion as to what the pandas team can do for future releases. Thanks!

However, you've failed to convince me this is someone else's problem. At some point, it was pip that delegated the setup_requires task to easy_install (or perhaps to some other intermediary). I didn't invoke easy_install, and nor did pandas.

I feel that pip should do one of the following:

1. Stop delegating to external tools (ie: the buck stops here).
2. Present a big warning before and/or after delegating to external tools, so that package developers (eg: the pandas team) are aware of issues with their packages (ie: pass the buck).

Not doing either of the above will mean more pip users are left staring at cryptic tracebacks, and further bug reports being logged.

[tysonclugg]

You should install numpy first with pip that will honor python_requires, then install pandas.

This works as expected. Thanks. :-)

And to definitely fix this kind of situation, pandas should start using PEP-518 build-system.requires to declare its build dependencies.

FYI: A pull request to introduce (or reintroduce?) pyproject.toml in pandas was closed only 4 days ago. The comments cited various issues with pip, and the final verdict was that they didn't feel using pyproject.toml was necessary. Given the issues pandas experienced resulting from pip bugs, I'm not surprised they're holding back for the moment.

[TomAugspurger]

the final verdict was that they didn't feel using pyproject.toml was necessary.

Just to clarify, pandas wants to adopt pyproject.toml, but it's not a blocker for our next release.

[xavfernandez]

However, you've failed to convince me this is someone else's problem. At some point, it was pip that delegated the setup_requires task to easy_install (or perhaps to some other intermediary). I didn't invoke easy_install, and nor did pandas.

Well, pip has no concept of setup_requires (that's a big part of PEP-518 rationale ).

Since you are specifying --no-binary=pandas, you're basically saying to pip: "go ahead, execute pandas' setup.py".
That's where setuptools (pandas' build tool) checks setup_requires, sees that numpy isn't available and decides to use easy_install to provide it.
So I'd say that, technically, pandas did invoke easy_install.

[pradyunsg]

Yea, there's nothing actionable here from pip's end -- in an ideal world, pandas would to switch over to PEP 518.

Until that happens, it'd be best for pandas to improve error messaging that's printed when there's some sort of incompatibility.

[tysonclugg]

So, let me get this straight.

The --no-binary option is supported, but yields undefined behaviour. The details shown by pip install --help don't mention that this option has been disavowed, and errors generated aren't meant to give users any clues as to why pip has thrown a long traceback - they should have known better, right?

Installing source only packages may also trigger this behaviour (ie: when wheels aren't available, perhaps because they pre-date wheels), and I'm guessing you're about to tell me that isn't really supported through some convoluted logic about an obscure technical detail that attributes blame to the package author (who was probably following the advice of the day, or trying to avoid one of the previously mentioned pip bugs).

Oh the joys of installing and maintaining Python packages - and installers!

[TomAugspurger]

@tysonclugg speaking as a maintainer (not a pip maintainer), those kinds of comments are disheartening to read :/