Support axis=None in all reductions

[TomAugspurger]

#21486 is implementing support for axis=None for compatibility with NumPy 1.15's all / any.

The basic idea is to have df.op(axis=None) be the same as np.op(df, axis=None) for reductions like sum, mean, etc.

Getting there poses a backwards-compatibility challenge. Currently for some reduction functions like np.sum we interpret axis=None as

In [11]: df = pd.DataFrame({"A": [1, 2]})

In [12]: np.sum(df)
Out[12]:
A    3
dtype: int64

The best I option I see is to just warn that the behavior will change in the future, without providing a way to achieve the behavior today. Then users can update things like np.sum(df) to np.sum(df, axis=0). In a later version we'll make the actual change to interpret axis=None like NumPy.

[uds5501]

@TomAugspurger @gfyoung , I am interested in this issue and would like to help. But could you guys guide me on how to display these deprecate messages?

[gfyoung]

@uds5501 : Perhaps this will help?

import pandas.util.testing as tm
import warnings

def f():
   warnings.warn("This is a deprecation warning", DeprecationWarning)

def test_f():
   # Might need to also pass in `check_stacklevel=False`.
   with tm.assert_produces_warning(DeprecationWarning):
      f()

[uds5501]

@gfyoung Thanks for the help!
Now regarding this issue, will the warning in the functions like df.sum and df.mean etc : "The behavior 'df.op(axis = None)` is going to change in the future versions " suffice for this issue?

[gfyoung]

@uds5501 : Describe (briefly) how the behavior would change in the message.

[uds5501]

@gfyoung Alright sure, would submit a PR soon 😄

[uds5501]

@gfyoung I had one more query regarding this. If I am not wrong, we call Dataframe.sum from generic.py at core . So, am I supposed to place this error message in the function given below

pandas/pandas/core/generic.py

Lines 9551 to 9571 in 1a23779

	def _make_min_count_stat_function(cls, name, name1, name2, axis_descr, desc,
	f, examples):
	@Substitution(outname=name, desc=desc, name1=name1, name2=name2,
	axis_descr=axis_descr, min_count=_min_count_stub,
	examples=examples)
	@Appender(_num_doc)
	def stat_func(self, axis=None, skipna=None, level=None, numeric_only=None,
	min_count=0,
	**kwargs):
	nv.validate_stat_func(tuple(), kwargs, fname=name)
	if skipna is None:
	skipna = True
	if axis is None:
	axis = self._stat_axis_number
	if level is not None:
	return self._agg_by_level(name, axis=axis, level=level,
	skipna=skipna, min_count=min_count)
	return self._reduce(f, name, axis=axis, skipna=skipna,
	numeric_only=numeric_only, min_count=min_count)
	return set_function_name(stat_func, name, cls)

or make separate files for the same like these?

pandas/pandas/core/categorical.py

Lines 1 to 8 in 83e95e0

	import warnings
	# TODO: Remove after 0.23.x
	warnings.warn("'pandas.core' is private. Use 'pandas.Categorical'",
	FutureWarning, stacklevel=2)
	from pandas.core.arrays import Categorical # noqa
	from pandas.core.dtypes.dtypes import CategoricalDtype # noqa

A little guidance would be appreciated!

[gfyoung]

IINM, you should add it to generic.py.

cc @jreback

[TomAugspurger]

FYI @uds5501, this is a rather large change that needs discussion first. If you're interested in working on it, then by all means, but you might want to wait for feedback from other maintainers before diving into it.

[uds5501]

@TomAugspurger Sure sir! I have not yet begun either and conclusive discussions from maintainers who have better context regarding this are always welcome!