On systems with older SSL, packages that trigger easy_install fail

[ilovezfs]

bash-3.2$ /usr/local/Cellar/gitfs/0.4.5.1_1/libexec/bin/pip --version
pip 9.0.3 from /usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages (python 2.7)
bash-3.2$  

but

iMac-TMP:homebrew-core joe$ brew instal -dvs gitfs
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/gitfs.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/libgit2.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/pkg-config.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/cmake.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/sphinx-doc.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/libssh2.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/openssl.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/makedepend.rb
/usr/bin/sandbox-exec -f /tmp/homebrew20180330-16347-uu47na.sb nice /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3/bin/ruby -W0 -I /usr/local/Homebrew/Library/Homebrew -- /usr/local/Homebrew/Library/Homebrew/build.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/gitfs.rb --verbose --debug
/usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FromPathLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/gitfs.rb
/usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/libgit2.rb
/usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/libssh2.rb
/usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/openssl.rb
/usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/pkg-config.rb
/usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/cmake.rb
/usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/sphinx-doc.rb
/usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/makedepend.rb
==> Downloading https://github.com/PressLabs/gitfs/archive/0.4.5.1.tar.gz
Already downloaded: /Users/joe/Library/Caches/Homebrew/gitfs-0.4.5.1.tar.gz
==> Verifying gitfs-0.4.5.1.tar.gz checksum
tar xzf /Users/joe/Library/Caches/Homebrew/gitfs-0.4.5.1.tar.gz
==> Downloading https://files.pythonhosted.org/packages/b1/72/2d70c5a1de409ceb3a27ff2ec007ecdd5cc52239e7c74990e32af57affe9/virtualenv-15.2.0.tar.gz
Already downloaded: /Users/joe/Library/Caches/Homebrew/gitfs--homebrew-virtualenv-15.2.0.tar.gz
==> Verifying gitfs--homebrew-virtualenv-15.2.0.tar.gz checksum
tar xzf /Users/joe/Library/Caches/Homebrew/gitfs--homebrew-virtualenv-15.2.0.tar.gz
==> python2.7 -c import setuptools... --no-user-cfg install --prefix=/tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target --single-version-externally-managed --record=installed.txt
running install
running build
running build_py
creating build
creating build/lib
copying virtualenv.py -> build/lib
creating build/lib/virtualenv_support
copying virtualenv_support/__init__.py -> build/lib/virtualenv_support
copying virtualenv_support/argparse-1.4.0-py2.py3-none-any.whl -> build/lib/virtualenv_support
copying virtualenv_support/pip-9.0.3-py2.py3-none-any.whl -> build/lib/virtualenv_support
copying virtualenv_support/setuptools-36.8.0-py2.py3-none-any.whl -> build/lib/virtualenv_support
copying virtualenv_support/setuptools-39.0.1-py2.py3-none-any.whl -> build/lib/virtualenv_support
copying virtualenv_support/wheel-0.29.0-py2.py3-none-any.whl -> build/lib/virtualenv_support
copying virtualenv_support/wheel-0.30.0-py2.py3-none-any.whl -> build/lib/virtualenv_support
running install_lib
copying build/lib/virtualenv.py -> /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/lib/python2.7/site-packages
creating /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/lib/python2.7/site-packages/virtualenv_support
copying build/lib/virtualenv_support/__init__.py -> /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/lib/python2.7/site-packages/virtualenv_support
copying build/lib/virtualenv_support/argparse-1.4.0-py2.py3-none-any.whl -> /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/lib/python2.7/site-packages/virtualenv_support
copying build/lib/virtualenv_support/pip-9.0.3-py2.py3-none-any.whl -> /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/lib/python2.7/site-packages/virtualenv_support
copying build/lib/virtualenv_support/setuptools-36.8.0-py2.py3-none-any.whl -> /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/lib/python2.7/site-packages/virtualenv_support
copying build/lib/virtualenv_support/setuptools-39.0.1-py2.py3-none-any.whl -> /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/lib/python2.7/site-packages/virtualenv_support
copying build/lib/virtualenv_support/wheel-0.29.0-py2.py3-none-any.whl -> /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/lib/python2.7/site-packages/virtualenv_support
copying build/lib/virtualenv_support/wheel-0.30.0-py2.py3-none-any.whl -> /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/lib/python2.7/site-packages/virtualenv_support
byte-compiling /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/lib/python2.7/site-packages/virtualenv.py to virtualenv.pyc
byte-compiling /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/lib/python2.7/site-packages/virtualenv_support/__init__.py to __init__.pyc
running install_egg_info
running egg_info
writing virtualenv.egg-info/PKG-INFO
writing top-level names to virtualenv.egg-info/top_level.txt
writing dependency_links to virtualenv.egg-info/dependency_links.txt
writing entry points to virtualenv.egg-info/entry_points.txt
reading manifest file 'virtualenv.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*' found under directory 'docs/_templates'
warning: no previously-included files matching '*' found under directory 'docs/_build'
writing manifest file 'virtualenv.egg-info/SOURCES.txt'
Copying virtualenv.egg-info to /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/lib/python2.7/site-packages/virtualenv-15.2.0-py2.7.egg-info
running install_scripts
Installing virtualenv script to /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/bin
writing list of installed files to 'installed.txt'
==> python2.7 -s /tmp/gitfs--homebrew-virtualenv-20180330-16349-1dpywqn/target/bin/virtualenv -p python2.7 /usr/local/Cellar/gitfs/0.4.5.1_1/libexec
New python executable in /usr/local/Cellar/gitfs/0.4.5.1_1/libexec/bin/python
Installing setuptools, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2.7
/usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/python.rb
==> Downloading https://files.pythonhosted.org/packages/86/8c/70aea8215c6ab990f2d91e7ec171787a41b7fbc83df32a067ba5d7f3324f/atomiclong-0.1.1.tar.gz
Already downloaded: /Users/joe/Library/Caches/Homebrew/gitfs--atomiclong-0.1.1.tar.gz
==> Verifying gitfs--atomiclong-0.1.1.tar.gz checksum
tar xzf /Users/joe/Library/Caches/Homebrew/gitfs--atomiclong-0.1.1.tar.gz
==> /usr/local/Cellar/gitfs/0.4.5.1_1/libexec/bin/pip install -v --no-deps --no-binary :all: --ignore-installed /private/tmp/gitfs--atomiclong-20180330-16349-1dq3o08/atomiclong-0.1.1
Processing /private/tmp/gitfs--atomiclong-20180330-16349-1dq3o08/atomiclong-0.1.1
  Running setup.py (path:/private/tmp/pip-jc5alv-build/setup.py) egg_info for package from file:///private/tmp/gitfs--atomiclong-20180330-16349-1dq3o08/atomiclong-0.1.1
    Running command python setup.py egg_info
    Couldn't find index page for 'cffi' (maybe misspelled?)
    No local packages or working download links found for cffi
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/tmp/pip-jc5alv-build/setup.py", line 42, in <module>
        cmdclass={"test": PyTest},
      File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/setuptools/__init__.py", line 128, in setup
        _install_setup_requires(attrs)
      File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/setuptools/__init__.py", line 123, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/setuptools/dist.py", line 513, in fetch_build_eggs
        replace_conflicting=True,
      File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/pkg_resources/__init__.py", line 774, in resolve
        replace_conflicting=replace_conflicting
      File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1057, in best_match
        return self.obtain(req, installer)
      File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1069, in obtain
        return installer(requirement)
      File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/setuptools/dist.py", line 580, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 667, in easy_install
        raise DistutilsError(msg)
    distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('cffi')
Cleaning up...
Command "python setup.py egg_info" failed with error code 1 in /private/tmp/pip-jc5alv-build/
Exception information:
Traceback (most recent call last):
  File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/pip/req/req_set.py", line 634, in _prepare_file
    abstract_dist.prep_for_dist()
  File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/pip/req/req_set.py", line 129, in prep_for_dist
    self.req_to_install.run_egg_info()
  File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/pip/req/req_install.py", line 439, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/usr/local/Cellar/gitfs/0.4.5.1_1/libexec/lib/python2.7/site-packages/pip/utils/__init__.py", line 707, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command "python setup.py egg_info" failed with error code 1 in /private/tmp/pip-jc5alv-build/
/usr/local/Homebrew/Library/Homebrew/debrew.rb:11:in `raise'
BuildError: Failed executing: /usr/local/Cellar/gitfs/0.4.5.1_1/libexec/bin/pip install -v --no-deps --no-binary :all: --ignore-installed /private/tmp/gitfs--atomiclong-20180330-16349-1dq3o08/atomiclong-0.1.1
1. raise
2. ignore
3. backtrace
4. irb
5. shell
Choose an action: 

CC @alex

This is running with

iMac-TMP:Homebrew joe$ git diff
diff --git a/Library/Homebrew/language/python_virtualenv_constants.rb b/Library/Homebrew/language/python_virtualenv_constants.rb
index 4efd2ce..64d2416 100644
--- a/Library/Homebrew/language/python_virtualenv_constants.rb
+++ b/Library/Homebrew/language/python_virtualenv_constants.rb
@@ -1,2 +1,2 @@
-PYTHON_VIRTUALENV_URL = "https://files.pythonhosted.org/packages/d4/0c/9840c08189e030873387a73b90ada981885010dd9aea134d6de30cd24cb8/virtualenv-15.1.0.tar.gz".freeze
-PYTHON_VIRTUALENV_SHA256 = "02f8102c2436bb03b3ee6dede1919d1dac8a427541652e5ec95171ec8adbc93a".freeze
+PYTHON_VIRTUALENV_URL = "https://files.pythonhosted.org/packages/b1/72/2d70c5a1de409ceb3a27ff2ec007ecdd5cc52239e7c74990e32af57affe9/virtualenv-15.2.0.tar.gz".freeze
+PYTHON_VIRTUALENV_SHA256 = "1d7e241b431e7afce47e77f8843a276f652699d1fa4f93b9d8ce0076fd7b0b54".freeze
iMac-TMP:Homebrew joe$ 

[dstufft]

It's not obvious, but that's actually easy_install causing this error. There is likely a setup_requires in this package that's triggering setuptools to fetch some build time dependencies outside of what pip has control over.

[ilovezfs]

@dstufft yet another example: https://gist.github.com/ilovezfs/ad434ccb5849584f9234f33791486468

Same problem?

[ilovezfs]

That's running with this diff

iMac-TMP:homebrew-core joe$ git diff -- Formula/conan.rb 
diff --git a/Formula/conan.rb b/Formula/conan.rb
index 79b421f..a272efd 100644
--- a/Formula/conan.rb
+++ b/Formula/conan.rb
@@ -17,10 +17,10 @@ class Conan < Formula
   depends_on "pkg-config" => :build
   depends_on "libffi"
   depends_on "openssl"
-  depends_on "python"
+#  depends_on "python"
 
   def install
-    venv = virtualenv_create(libexec, "python3")
+    venv = virtualenv_create(libexec) #, "python3")
     system libexec/"bin/pip", "install", "-v", "--no-binary", ":all:",
                               "--ignore-installed", buildpath
     system libexec/"bin/pip", "uninstall", "-y", name

(I switched it to Python 3 the other day because I was running into the brownout errors and thought it was conan specific)

[dstufft]

Yes, this is the relevant bit of the output:

  Running setup.py (path:/private/tmp/pip-build-D0rZQn/pylint/setup.py) egg_info for package pylint
    Running command python setup.py egg_info
    Couldn't find index page for 'pytest-runner' (maybe misspelled?)
    No local packages or working download links found for pytest-runner
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/tmp/pip-build-D0rZQn/pylint/setup.py", line 178, in <module>
        install()
      File "/private/tmp/pip-build-D0rZQn/pylint/setup.py", line 175, in install
        **kwargs)
      File "/usr/local/Cellar/conan/1.2.0/libexec/lib/python2.7/site-packages/setuptools/__init__.py", line 128, in setup
        _install_setup_requires(attrs)
      File "/usr/local/Cellar/conan/1.2.0/libexec/lib/python2.7/site-packages/setuptools/__init__.py", line 123, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/usr/local/Cellar/conan/1.2.0/libexec/lib/python2.7/site-packages/setuptools/dist.py", line 513, in fetch_build_eggs
        replace_conflicting=True,
      File "/usr/local/Cellar/conan/1.2.0/libexec/lib/python2.7/site-packages/pkg_resources/__init__.py", line 774, in resolve
        replace_conflicting=replace_conflicting
      File "/usr/local/Cellar/conan/1.2.0/libexec/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1057, in best_match
        return self.obtain(req, installer)
      File "/usr/local/Cellar/conan/1.2.0/libexec/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1069, in obtain
        return installer(requirement)
      File "/usr/local/Cellar/conan/1.2.0/libexec/lib/python2.7/site-packages/setuptools/dist.py", line 580, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/local/Cellar/conan/1.2.0/libexec/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 667, in easy_install
        raise DistutilsError(msg)
    distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pytest-runner')

You can see pip is executing the setup.py, then it starts dumping the output from setup.py (indented). The traceback you see comes from setup.py, and the error traceback is rooted in setuptools.

[ilovezfs]

@dstufft any suggested workarounds for these other than the use-non-Apple-Python sledgehammer?

[dstufft]

You can pre-install whatever is in setup_requires so that when setuptools hits that code path, it sees they're already available and doesn't try to fetch from PyPI. You might also consider opening an issue with setuptools and ask for a SecureTransport fallback like pip has or an option that would fallback to pyOpenSSL, that would let people pre-install pyOpenSSL to get a newer OpenSSL version.

[RichardProsser]

I have macOS "High Sierra" (10.13.3) and we use Python 3.4.4 with a 'venv' for each project; the rather obscure error message "Could not find a version that satisfies the requirement ..." occurs within the first 15 minutes of each hour, due to the TLS 'brownouts' apparently: https://status.python.org/

When the 'pip install' command succeeds, the usual upgrade prompt appears (with a new VE):
"You are using pip version 7.1.2, however version 9.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command."

Upgrading to pip 9.0.3 resolves the 'brownout' issue I believe but I cannot perform the upgrade until after the 15 minute 'closed window' is over!

Given that the Mac itself relies on Python 2.7, is there a system-wide way to upgrade pip without having to call upon PyPi? Better, could the Python distribution include the latest version of pip? I really don't see why we should be expected to upgrade pip each time, especially when this 'brownout' policy is in place.

[brainwane]

We recommend that people upgrade their local versions of pip using:

curl https://bootstrap.pypa.io/get-pip.py | python

I sent out a note recommending this to people on pre-High Sierra versions of macOS; I think this should work for you as well. I defer to others' assessments on this.

[njsmith]

we use Python 3.4.4 [...] Better, could the Python distribution include the latest version of pip?

You're using a python release from 2015. If you upgrade to the latest 3.6 release, then it does indeed include the latest version of pip.

[benjaminjkraft]

I just filed this against setuptools as suggested. Is there a good workaround other than manually pre-installing the setup_requires packages?

[alex]

Either pre-install the packages, or use a Python linked against a newer OpenSSL. Those are pretty much the only two options ATM.

…

On Mon, Apr 9, 2018 at 9:00 PM, Ben Kraft ***@***.***> wrote: I just filed this against setuptools as suggested. Is there a good workaround other than manually pre-installing the setup_requires packages? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#134 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAADBPJvPgCgb0XZW-WYHDIJSuVIbXclks5tnAQ6gaJpZM4TCVg_> .

-- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: D1B3 ADC0 E023 8CA6