DOC: Force Removal of pandas From Dependencies

[WillAyd]

From a discussion on Gitter with @TomAugspurger when following the pandas contributing guide the step:

conda env create -f environment.yml

Causes pandas to be installed in the environment from conda as a result of dependency resolution with third party libraries (ex: statsmodels, pyarrow, seaborn).

To prevent undesired conflicts between the local development version and the installed version we should update the documentation to have an explicit removal of the bundled version as such:

conda uninstall --force pandas

[yehia67]

So you want to Update this section

# Create and activate the build environment
conda env create -f environment.yml
conda activate pandas-dev

to be like that?

# Create and activate the build environment
conda uninstall --force pandas
conda env create -f environment.yml
conda activate pandas-dev

[WillAyd]

Swap the first and second lines but yes

…

Sent from my iPhone

On Feb 28, 2019, at 2:45 PM, yehia67 ***@***.***> wrote: So you want to Update this section # Create and activate the build environment conda env create -f environment.yml conda activate pandas-dev to be like that? # Create and activate the build environment conda uninstall --force pandas conda env create -f environment.yml conda activate pandas-dev — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[yehia67]

okay

[jorisvandenbossche]

I am not sure this is actually needed to recommend. It's true that conda does not work very nice with -e installed dev packages, but by force removing it, you might also run into troubles when updating / installing other packages that depend on pandas (as they don't find pandas, and conda then might install pandas again shadowing the dev version. Not sure this is what happens though).

The latest conda release with better pip integration enabled also solves this, by actually recognizing the dev installed package. But I think this is not yet enabled by default I think.

[TomAugspurger]

I think the common case is "setup an env from scratch", in which case the old way would guarantee a double-install of pandas, right?

…

On Mon, Mar 4, 2019 at 7:54 AM Joris Van den Bossche < ***@***.***> wrote: I am not sure this is actually needed to recommend. It's true that conda does not work very nice with -e installed dev packages, but by force removing it, you might also run into troubles when updating / installing other packages that depend on pandas (as they don't find pandas, and conda then might install pandas again shadowing the dev version. Not sure this is what happens though). The latest conda release with better pip integration enabled also solves this, by actually recognizing the dev installed package. But I think this is not yet enabled by default I think. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#25487 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIpt4iINt-v63E_T7valTRTxXFKQQks5vTSWLgaJpZM4bX3FA> .

[jorisvandenbossche]

Yes, but you typically don't keep this new env as is for the rest of your contributing lifetime no? (or at least, I update my dev environment from time to time in place, not each time re-creating it from scratch). What happens when I eg update seaborn that depends on pandas, if pandas was force-removed like this?

[TomAugspurger]

Yes I think so. Will conda overwrite a locally installed pandas? If so, then yes, we should maybe revert this.

…

On Mon, Mar 4, 2019 at 8:04 AM Joris Van den Bossche < ***@***.***> wrote: Yes, but you typically don't keep this new env as is for the rest of your contributing lifetime no? (or at least, I update my dev environment from time to time in place, not each time re-creating it from scratch). What happens when I eg update seaborn that depends on pandas, if pandas was force-removed like this? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#25487 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIm1DfO4hkkmt93nyHT0ir0NuRdmmks5vTSfRgaJpZM4bX3FA> .

[jorisvandenbossche]

It seems so. Eg doing conda install seaborn in the example below would have installed pandas 0.24.1 again.
Doing a conda config --set pip_interop_enabled True seems to prevent that however (but with that option, you would probably also not have to force remove pandas in the first place). Maybe we should recommend doing that instead.

(base) joris@joris-XPS-13-9350:~/scipy$ conda create -n test-pandas python=3.7 pandas cython
Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /home/joris/miniconda3/envs/test-pandas

  added / updated specs:
    - cython
    - pandas
    - python=3.7


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    cython-0.29.6              |   py37hf484d3e_0         2.2 MB  conda-forge
    numpy-1.16.2               |py37_blas_openblash1522bff_0         4.3 MB  conda-forge
    openssl-1.1.1b             |       h14c3975_0         4.0 MB  conda-forge
    python-3.7.1               |    h381d211_1002        36.4 MB  conda-forge
    ------------------------------------------------------------
                                           Total:        46.8 MB

The following NEW packages will be INSTALLED:

  blas               conda-forge/linux-64::blas-1.1-openblas
  bzip2              conda-forge/linux-64::bzip2-1.0.6-h14c3975_1002
  ca-certificates    conda-forge/linux-64::ca-certificates-2018.11.29-ha4d7672_0
  certifi            conda-forge/linux-64::certifi-2018.11.29-py37_1000
  cython             conda-forge/linux-64::cython-0.29.6-py37hf484d3e_0
  libffi             conda-forge/linux-64::libffi-3.2.1-hf484d3e_1005
  libgcc-ng          conda-forge/linux-64::libgcc-ng-7.3.0-hdf63c60_0
  libgfortran-ng     conda-forge/linux-64::libgfortran-ng-7.2.0-hdf63c60_3
  libstdcxx-ng       conda-forge/linux-64::libstdcxx-ng-7.3.0-hdf63c60_0
  ncurses            conda-forge/linux-64::ncurses-6.1-hf484d3e_1002
  numpy              conda-forge/linux-64::numpy-1.16.2-py37_blas_openblash1522bff_0
  openblas           conda-forge/linux-64::openblas-0.3.3-h9ac9557_1001
  openssl            conda-forge/linux-64::openssl-1.1.1b-h14c3975_0
  pandas             conda-forge/linux-64::pandas-0.24.1-py37hf484d3e_0
  pip                conda-forge/linux-64::pip-19.0.3-py37_0
  python             conda-forge/linux-64::python-3.7.1-h381d211_1002
  python-dateutil    conda-forge/noarch::python-dateutil-2.8.0-py_0
  pytz               conda-forge/noarch::pytz-2018.9-py_0
  readline           conda-forge/linux-64::readline-7.0-hf8c457e_1001
  setuptools         conda-forge/linux-64::setuptools-40.8.0-py37_0
  six                conda-forge/linux-64::six-1.12.0-py37_1000
  sqlite             conda-forge/linux-64::sqlite-3.26.0-h67949de_1000
  tk                 conda-forge/linux-64::tk-8.6.9-h84994c4_1000
  wheel              conda-forge/linux-64::wheel-0.33.1-py37_0
  xz                 conda-forge/linux-64::xz-5.2.4-h14c3975_1001
  zlib               conda-forge/linux-64::zlib-1.2.11-h14c3975_1004


Proceed ([y]/n)? 


Downloading and Extracting Packages
python-3.7.1         | 36.4 MB   | ##################################################################################################################################### | 100% 
numpy-1.16.2         | 4.3 MB    | ##################################################################################################################################### | 100% 
openssl-1.1.1b       | 4.0 MB    | ##################################################################################################################################### | 100% 
cython-0.29.6        | 2.2 MB    | ##################################################################################################################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate test-pandas
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(base) joris@joris-XPS-13-9350:~/scipy$ act test-pandas
(test-pandas) joris@joris-XPS-13-9350:~/scipy$ conda uninstall --force pandas

## Package Plan ##

  environment location: /home/joris/miniconda3/envs/test-pandas

  removed specs:
    - pandas


The following packages will be REMOVED:

  pandas-0.24.1-py37hf484d3e_0


Proceed ([y]/n)? 

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

(test-pandas) joris@joris-XPS-13-9350:~/scipy/pandas$ pip install -e .
Obtaining file:///home/joris/scipy/pandas
Requirement already satisfied: python-dateutil>=2.5.0 in /home/joris/miniconda3/envs/test-pandas/lib/python3.7/site-packages (from pandas==0.25.0.dev0+192.gbd49d2f2a) (2.8.0)
Requirement already satisfied: pytz>=2011k in /home/joris/miniconda3/envs/test-pandas/lib/python3.7/site-packages (from pandas==0.25.0.dev0+192.gbd49d2f2a) (2018.9)
Requirement already satisfied: numpy>=1.12.0 in /home/joris/miniconda3/envs/test-pandas/lib/python3.7/site-packages (from pandas==0.25.0.dev0+192.gbd49d2f2a) (1.16.2)
Requirement already satisfied: six>=1.5 in /home/joris/miniconda3/envs/test-pandas/lib/python3.7/site-packages (from python-dateutil>=2.5.0->pandas==0.25.0.dev0+192.gbd49d2f2a) (1.12.0)
Installing collected packages: pandas
  Running setup.py develop for pandas
Successfully installed pandas
(test-pandas) joris@joris-XPS-13-9350:~/scipy/pandas$ conda list
# packages in environment at /home/joris/miniconda3/envs/test-pandas:
#
# Name                    Version                   Build  Channel
blas                      1.1                    openblas    conda-forge
bzip2                     1.0.6             h14c3975_1002    conda-forge
ca-certificates           2018.11.29           ha4d7672_0    conda-forge
certifi                   2018.11.29            py37_1000    conda-forge
cython                    0.29.6           py37hf484d3e_0    conda-forge
libffi                    3.2.1             hf484d3e_1005    conda-forge
libgcc-ng                 7.3.0                hdf63c60_0    conda-forge
libgfortran-ng            7.2.0                hdf63c60_3    conda-forge
libstdcxx-ng              7.3.0                hdf63c60_0    conda-forge
ncurses                   6.1               hf484d3e_1002    conda-forge
numpy                     1.16.2          py37_blas_openblash1522bff_0  [blas_openblas]  conda-forge
openblas                  0.3.3             h9ac9557_1001    conda-forge
openssl                   1.1.1b               h14c3975_0    conda-forge
pandas                    0.25.0.dev0+192.gbd49d2f2a           dev_0    <develop>
pip                       19.0.3                   py37_0    conda-forge
python                    3.7.1             h381d211_1002    conda-forge
python-dateutil           2.8.0                      py_0    conda-forge
pytz                      2018.9                     py_0    conda-forge
readline                  7.0               hf8c457e_1001    conda-forge
setuptools                40.8.0                   py37_0    conda-forge
six                       1.12.0                py37_1000    conda-forge
sqlite                    3.26.0            h67949de_1000    conda-forge
tk                        8.6.9             h84994c4_1000    conda-forge
wheel                     0.33.1                   py37_0    conda-forge
xz                        5.2.4             h14c3975_1001    conda-forge
zlib                      1.2.11            h14c3975_1004    conda-forge
(test-pandas) joris@joris-XPS-13-9350:~/scipy/pandas$ conda install seaborn
Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /home/joris/miniconda3/envs/test-pandas

  added / updated specs:
    - seaborn


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    matplotlib-3.0.3           |           py37_0           6 KB  conda-forge
    matplotlib-base-3.0.3      |   py37h167e16e_0         6.6 MB  conda-forge
    tornado-6.0.1              |   py37h14c3975_0         635 KB  conda-forge
    ------------------------------------------------------------
                                           Total:         7.2 MB

The following NEW packages will be INSTALLED:

  atk                conda-forge/linux-64::atk-2.25.90-hb9dd440_1002
  cairo              conda-forge/linux-64::cairo-1.16.0-ha4e643d_1000
  cycler             conda-forge/noarch::cycler-0.10.0-py_1
  dbus               conda-forge/linux-64::dbus-1.13.0-h4e0c4b3_1000
  expat              conda-forge/linux-64::expat-2.2.5-hf484d3e_1002
  fontconfig         conda-forge/linux-64::fontconfig-2.13.1-h2176d3f_1000
  freetype           conda-forge/linux-64::freetype-2.9.1-h94bbf69_1005
  gdk-pixbuf         conda-forge/linux-64::gdk-pixbuf-2.36.12-h49783d7_1002
  gettext            conda-forge/linux-64::gettext-0.19.8.1-h9745a5d_1001
  glib               conda-forge/linux-64::glib-2.58.2-hf63aee3_1001
  gobject-introspec~ conda-forge/linux-64::gobject-introspection-1.58.2-py37h2da5eee_1000
  graphite2          conda-forge/linux-64::graphite2-1.3.13-hf484d3e_1000
  gstreamer          conda-forge/linux-64::gstreamer-1.14.4-h66beb1c_1001
  gtk2               conda-forge/linux-64::gtk2-2.24.31-hb68c50a_1001
  harfbuzz           conda-forge/linux-64::harfbuzz-2.3.1-h6824563_0
  icu                conda-forge/linux-64::icu-58.2-hf484d3e_1000
  jpeg               conda-forge/linux-64::jpeg-9c-h14c3975_1001
  kiwisolver         conda-forge/linux-64::kiwisolver-1.0.1-py37h6bb024c_1002
  libiconv           conda-forge/linux-64::libiconv-1.15-h14c3975_1004
  libpng             conda-forge/linux-64::libpng-1.6.36-h84994c4_1000
  libtiff            conda-forge/linux-64::libtiff-4.0.10-h9022e91_1002
  libuuid            conda-forge/linux-64::libuuid-2.32.1-h14c3975_1000
  libxcb             conda-forge/linux-64::libxcb-1.13-h14c3975_1002
  libxml2            conda-forge/linux-64::libxml2-2.9.8-h143f9aa_1005
  matplotlib         conda-forge/linux-64::matplotlib-3.0.3-py37_0
  matplotlib-base    conda-forge/linux-64::matplotlib-base-3.0.3-py37h167e16e_0
  pandas             conda-forge/linux-64::pandas-0.24.1-py37hf484d3e_0
  pango              conda-forge/linux-64::pango-1.40.14-h4ea9474_1004
  patsy              conda-forge/noarch::patsy-0.5.1-py_0
  pcre               conda-forge/linux-64::pcre-8.41-hf484d3e_1003
  pixman             conda-forge/linux-64::pixman-0.34.0-h14c3975_1003
  pthread-stubs      conda-forge/linux-64::pthread-stubs-0.4-h14c3975_1001
  pyparsing          conda-forge/noarch::pyparsing-2.3.1-py_0
  pyqt               conda-forge/linux-64::pyqt-5.6.0-py37h13b7fb3_1008
  qt                 conda-forge/linux-64::qt-5.6.2-hce4f676_1013
  scipy              conda-forge/linux-64::scipy-1.2.1-py37_blas_openblash1522bff_0
  seaborn            conda-forge/noarch::seaborn-0.9.0-py_0
  sip                conda-forge/linux-64::sip-4.18.1-py37hf484d3e_1000
  statsmodels        conda-forge/linux-64::statsmodels-0.9.0-py37h3010b51_1000
  tornado            conda-forge/linux-64::tornado-6.0.1-py37h14c3975_0
  xorg-kbproto       conda-forge/linux-64::xorg-kbproto-1.0.7-h14c3975_1002
  xorg-libice        conda-forge/linux-64::xorg-libice-1.0.9-h14c3975_1004
  xorg-libsm         conda-forge/linux-64::xorg-libsm-1.2.3-h4937e3b_1000
  xorg-libx11        conda-forge/linux-64::xorg-libx11-1.6.7-h14c3975_1000
  xorg-libxau        conda-forge/linux-64::xorg-libxau-1.0.9-h14c3975_0
  xorg-libxdmcp      conda-forge/linux-64::xorg-libxdmcp-1.1.2-h14c3975_1007
  xorg-libxext       conda-forge/linux-64::xorg-libxext-1.3.3-h14c3975_1004
  xorg-libxrender    conda-forge/linux-64::xorg-libxrender-0.9.10-h14c3975_1002
  xorg-libxt         conda-forge/linux-64::xorg-libxt-1.1.5-h14c3975_1002
  xorg-renderproto   conda-forge/linux-64::xorg-renderproto-0.11.1-h14c3975_1002
  xorg-xextproto     conda-forge/linux-64::xorg-xextproto-7.3.0-h14c3975_1002
  xorg-xproto        conda-forge/linux-64::xorg-xproto-7.0.31-h14c3975_1007
  zstd               conda-forge/linux-64::zstd-1.3.3-1

The following packages will be DOWNGRADED:

  openssl                                 1.1.1b-h14c3975_0 --> 1.0.2r-h14c3975_0
  python                                3.7.1-h381d211_1002 --> 3.7.1-hd21baee_1001


Proceed ([y]/n)? n


Exiting.