FutureWarnings should specify a stacklevel

[njsmith]

Running my test suite, I see:

/home/njs/.user-python2.7-64bit-3/local/lib/python2.7/site-packages/pandas/core/categorical.py:443: FutureWarning: Accessing 'levels' is deprecated, use 'categories'
  warn("Accessing 'levels' is deprecated, use 'categories'", FutureWarning)
/home/njs/.user-python2.7-64bit-3/local/lib/python2.7/site-packages/pandas/core/categorical.py:291: RuntimeWarning: Values and categories have different dtypes. Did you mean to use
'Categorical.from_codes(codes, categories)'?
  "'Categorical.from_codes(codes, categories)'?", RuntimeWarning)
/home/njs/.user-python2.7-64bit-3/local/lib/python2.7/site-packages/pandas/core/categorical.py:295: RuntimeWarning: None of the categories were found in values. Did you mean to use
'Categorical.from_codes(codes, categories)'?
  "'Categorical.from_codes(codes, categories)'?", RuntimeWarning)
./home/njs/.user-python2.7-64bit-3/local/lib/python2.7/site-packages/pandas/core/categorical.py:395: FutureWarning: 'labels' is deprecated. Use 'codes' instead
  warnings.warn("'labels' is deprecated. Use 'codes' instead", FutureWarning)

Okay, fair enough. But the problem is that all of the listed lines of code are inside pandas, so I have no idea which parts of my code are actually doing the wrong thing. warnings.warn provides a stacklevel argument, where the idea is that you should pass a non-zero value so that warnings are attributed to the code that actually needs to be changed. E.g., stacklevel=1 means that the code which is doing something wrong is the code which called the function containing the call to warnings.warn.

[jorisvandenbossche]

@njsmith I changed the title, as this is not specific to the categorical example

This was brought to my attention again by the discussion in ipython about changing the default visibility of DeprecationWarnings (ipython/ipython#8478)

Basically, in plain warning calls, we should use stacklevel=2

[njsmith]

Rule of thumb: Calling warnings.warn without specifying a stacklevel= is always a bug.

Here's a summary of what's at stake, since as @jorisvandenbossche pointed out in the ipython discussion the docs are somewhat confusing. It's actually pretty simple. Say I have this code:

def f():
    warnings.warn("this is deprecated", category=DeprecationWarning, stacklevel=X)

def g():
    f()

If X is 1 (the default), then Python will print a warning saying "f is doing something wrong -- it's using deprecated functionality". If X is 2 (what you usually want, esp. for deprecation warnings), then Python will print a warning saying "g is doing something wrong -- it's using deprecated functionality".

[jorisvandenbossche]

Thanks @njsmith for the overview!

For pandas it means that

• most direct uses of FutureWarning (inside functions) need a stacklevel=2
• we have a deprecate(..) function: here stacklevel=2 works as well (has only to be changed in utils/decorators.py
• we have a deprecate_kwarg decorator: I tested this one, and apparantly a stacklevel=2 is also the correct one
• only for the ones on import (eg rpy: https://github.com/pydata/pandas/blob/master/pandas/rpy/__init__.py) we don't have to change anything (as you cannot get this right for all python versions -> @njsmith correct?)

[njsmith]

For warn-on-import for deprecated modules/packages, you should also use stacklevel=2. This works correctly on Python 2.7 (so... like 80%+ of your users), and will hopefully get fixed to work correctly on python 3 as well (see: https://bugs.python.org/issue24305)