RLS: 0.25.0

[jreback]

Tracking issue for 0.25.0

[jreback]

@pandas-dev/pandas-core

as we briefly discussed before, we need to be pretty circumspect about merging even to master changes that are not backportable to 0.24.x, prob for a little while. Small changes ok, but sweeping reorgs just make backporting much more difficult (as may need to go thru potentially a lot of effort to backport). So am going to suggest that we hold off on 'bigger' things until 0.24.1 is out.

• this means don't remove any PY2 removal stuff quite yet.
• we may need to keep merging master on soon to be merged PRs, so the backlog may grow.
• bug fixes, by all means, sometimes this may mean narrowing the scope of a bug fix though (meaning that if the 'right' way to do it involves a fairly big reorg).

[jbrockmendel]

Do we have any major goals for 0.25.0?

[jreback]

• deprecate remaining items on the list
• remove PY2

[h-vetinari]

I think one of the key points is still to decide on a policy for dealing with breaking changes, both for 0.25/1.0 and after.

There's still a bunch of inconsistencies that we'd be locking in for 1.0, and even more so if we'd be doing actual SemVer afterwards - changing a None to a np.nan somewhere would then already count as breaking).

• groupby: API/DOC: clean up DataFrame.groupby.apply #22545 (+ Inconsistent groupby-apply output shape and random values returned. #20420, BUG: index of group not returned correctly in groupby.apply #22541, API: groupby aggregation with apply does not drop groupby-column #22542, BUG: weird behaviour for returning group in groupby.apply #22546, ...)
• consistent IO-API: API: formalize the pandas IO API #15862, API: Unify compression-kwarg for IO-methods #21640, maybe CLN: spin msgpack support to [new] pandas-msgpack #15841
• boolean Series: Mismatch in Series/DataFrame boolean ops behavior #22724, NaN values impact binary or operations asymmetrically #6528, "Or" does not have commutative property (possibly because it catches exceptions silently?) #13806

This list is obviously not exhaustive. Pandas is certainly very mature by now, but IMO not consistent enough to do SemVer (otherwise every release would be major).

Maybe numpy-style rolling deprecations (as we've been doing as well) would be less restrictive to actually fixing some of these issues (it's worth noting that on the numpy side, it will still take monumental determination and effort to actually fix some very longstanding issues for an eventual 2.0)

There's much smaller fish I'd like to fry as well, but all of which are breaking in some way.

• inferring types for .str-accessor; moving bytes to separate accessor; Disable .str-accessor for byte data #23011 API: Series.str-accessor infers dtype (and Index.str does not raise on all-NA) #23167+ several fixes that are blocked on that refactor
• fixing maybe_promote and maybe_upcast_putmask BUG/Internals: maybe_upcast_putmask #23823 BUG/Internals: maybe_promote #23833 to then fix .update etc. API/BUG: unnecessary upcasting for df.update #23606
• adding the long-open inverses to unique API: provide a better way of doing np.unique(return_inverses=True) #4087, API/ENH: overhaul/unify/improve .unique #22824, ENH: adding .unique() to DF (or return_inverse for duplicated) #21357, which needs that Series.unique stays a Series (which would also fit very well with the current drive from 0.24 that one needs .to_numpy() to get an ndarray)

TL;DR: most bigger PRs are breaking from the point of SemVer. A policy of zero breaking changes until 1.0 will probably mean little development.

[stonebig]

What is the likely pandas-0.25.0 release date ?

[jorisvandenbossche]

@stonebig we didn't really discuss in detail recently, but I think we still target to release somewhere in June (but no guarantees!)

[TomAugspurger]

Are we on track for cutting an RC today? (my gut says "not quite").

I'm fixing up a few CI issues across pandas & dask right now. Will do a run through outstanding issues after that (I know I have the __array_ufunc__ PR at least).

[jreback]

yeah let’s get ci fixed and merge what we can
tomorrow

[jreback]

@jorisvandenbossche @TomAugspurger I am super busy today / tomorrow, so don't block on me for things and merge where you guys think appropriate. if you want me to specifically review something, pls ping.

[jorisvandenbossche]

@jreback I would like to have your go for #17753

[TomAugspurger]

@pandas-dev/pandas-core I think #27095 is the last one for the RC.

I'll try to tag 0.25.0rc0 tonight (~5 hours from now).

LMK if there are any others that should go in.