Allow plotting backend option context without matplotlib

[TomAugspurger]

Right now, using pd.option_context to set the backend fails on __exit__ when matplotlib isn't installed.

In [1]: import pandas as pd

In [2]: with pd.option_context('plotting.backend', 'test'):
   ...:     pass
   ...:
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
~/sandbox/pandas/pandas/core/config_init.py in register_plotting_backend_cb(key)
    593         try:
--> 594             import pandas.plotting._matplotlib  # noqa
    595         except ImportError:

~/sandbox/pandas/pandas/plotting/_matplotlib/__init__.py in <module>
      2
----> 3 from pandas.plotting._matplotlib.boxplot import (
      4     BoxPlot,

~/sandbox/pandas/pandas/plotting/_matplotlib/boxplot.py in <module>
      3
----> 4 from matplotlib.artist import setp
      5 import numpy as np

ModuleNotFoundError: No module named 'matplotlib'

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
<ipython-input-2-08ebc0f8d5ad> in <module>
      1 with pd.option_context('plotting.backend', 'test'):
----> 2     pass
      3

~/sandbox/pandas/pandas/_config/config.py in __exit__(self, *args)
    410         if self.undo:
    411             for pat, val in self.undo:
--> 412                 _set_option(pat, val, silent=True)
    413
    414

~/sandbox/pandas/pandas/_config/config.py in _set_option(*args, **kwargs)
    134             if silent:
    135                 with warnings.catch_warnings(record=True):
--> 136                     o.cb(key)
    137             else:
    138                 o.cb(key)

~/sandbox/pandas/pandas/core/config_init.py in register_plotting_backend_cb(key)
    595         except ImportError:
    596             raise ImportError(
--> 597                 "matplotlib is required for plotting when the "
    598                 'default backend "matplotlib" is selected.'
    599             )

ImportError: matplotlib is required for plotting when the default backend "matplotlib" is selected.

I don't believe that should raise an error. We may need to make the default option None, and then resole that to matplotlib later on.

[sameshl]

@TomAugspurger, if I understand you correctly, you mean, make default backend None and then resole it to matplotlib later on?

[TomAugspurger]

I'm not sure. That may be my first approach.

We may need to have a larger discussion around when backends are loaded / validated. Right now it's when used (_get_plot_backend()), but we might also do it when the option is set. @datapythonista do you have an opinion?

[datapythonista]

I think what we do now, load the backend when the option is set is the best option. If users make a typo in the name of the backend, I don't want to raise an exception at a later time, when they run a proper call to .plot().

In this case, the problem seems to be something else. You're setting the backend to test, but if cf.get_option(key)== "matplotlib": seems to be true. Or I'm missing something, or the bug is that with the context manager the value of the option that is being obtained is not the one set, but the previous.

[TomAugspurger]

Here's a better example

import sys
import types
import pandas as pd


foo = types.ModuleType('foo')
foo.plot = lambda *args, **kwargs: None
sys.modules['foo'] = foo

with pd.option_context('plotting.backend', 'foo'):
    pass

which raises with the following with no mpl:

Traceback (most recent call last):
  File "/Users/taugspurger/sandbox/pandas/pandas/core/config_init.py", line 594, in register_plotting_backend_cb
    import pandas.plotting._matplotlib  # noqa
  File "/Users/taugspurger/sandbox/pandas/pandas/plotting/_matplotlib/__init__.py", line 3, in <module>
    from pandas.plotting._matplotlib.boxplot import (
  File "/Users/taugspurger/sandbox/pandas/pandas/plotting/_matplotlib/boxplot.py", line 4, in <module>
    from matplotlib.artist import setp
ModuleNotFoundError: No module named 'matplotlib'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tmp.py", line 11, in <module>
    pass
  File "/Users/taugspurger/sandbox/pandas/pandas/_config/config.py", line 412, in __exit__
    _set_option(pat, val, silent=True)
  File "/Users/taugspurger/sandbox/pandas/pandas/_config/config.py", line 136, in _set_option
    o.cb(key)
  File "/Users/taugspurger/sandbox/pandas/pandas/core/config_init.py", line 597, in register_plotting_backend_cb
    "matplotlib is required for plotting when the "
ImportError: matplotlib is required for plotting when the default backend "matplotlib" is selected.

That's inside the context managers __exit__ where we essentially re-does `pd.set_options('plotting.backend', 'matplotlib'), since it's the default.

[datapythonista]

Ah, thanks for clarifying, I think I understand now.

I think my preferred option is to not import the backend if it's matplotlib in register_plotting_backend_cb. I think that makes sense even if we don't consider this bug. We only import when changing the option to see if it the module exists, and we know it does.

I think the only thing that may be somehow inconsistent, is that if the backend is set to the default value pandas.options.plotting.backend = 'matplotlib' it won't raise an exception if matplotlib is not installed. But I think that makes sense, is how pandas behaved until now too.

Does it make sense to you?

[vladu]

This appears to have been fixed.

#41503

@TLouf @jreback

[TLouf]

Indeed, the first example now raises

ValueError: Could not find plotting backend 'test'. Ensure that you've installed the package providing the 'test' entrypoint, or that the package has a top-level .plot method.

while the second runs without raising any exception. I guess this issue can be closed.

[jbrockmendel]

There's a comment in test_register_entrypoint

    # TODO: https://github.com/pandas-dev/pandas/issues/27517
    # Remove the td.skip_if_no_mpl

Since this is closed, does that mean we can remove that decoration?