DEPR: set_value / get_value

[jreback]

we already have too many public indexers.....

[bluss]

Is there a fast replacement for set_value? Using loc in its place seems to be slow.

In [328]: columns = list("abcdef")
In [329]: dx = 0.01; xs = np.arange(0, 1, step=dx);
In [330]: df = pd.DataFrame(index=xs)
In [331]: %%timeit
     ...: for x in xs:
     ...:   for c in columns:
     ...:     df.set_value(x, c, 1)
     ...: 
The slowest run took 8.40 times longer than the fastest. This could mean that an intermediate result is being cached.
100 loops, best of 3: 2.24 ms per loop

In [333]: df = pd.DataFrame(index=xs)
In [334]: %%timeit
     ...: for x in xs:
     ...:   for c in columns:
     ...:     df.loc[x, c] = 1
     ...: 
10 loops, best of 3: 96.4 ms per loop

[jorisvandenbossche]

In your example, df.at[x, c] = 1 is also faster than loc

[jorisvandenbossche]

Sparked by #15268

Personally, I never use those functions, so would also not regret them being gone (and would welcome the cleaning up of namespace).

But, are there genuine cases where these methods can be useful? (compared to the other indexing methods) Why were they added in the first place?

On StackOverflow, it seems mainly mentioned for its speed (eg http://stackoverflow.com/questions/13842088/set-value-for-particular-cell-in-pandas-dataframe/24517695#24517695)

cc @pandas-dev/pandas-core

[jreback]

IIRC these were always there :> (e.g. way before .iloc/.loc even).

replacements are well-supported .iat/.at. (which internally do almost what these functions do, they should be performant, yet support all of the pandas types). These routines are 'raw' and don't do any validation / checking.

These are not idiomatic (e.g. calling functions to set/get values). And confusing to beginners.

[wesm]

Once upon a time, I spent a lot of time making get_value and set_value extremely fast. Over time the performance has degraded significantly.

In the course of rewriting the scalar value access code paths for pandas 2, things will get fast again, so I'm not sure how to proceed given that will occur at some point in the future

[bluss]

.at is indeed fast, but still slows down that loop example by 2-3x. .at is barely documented, too.

[jreback]

@bluss

you know you can just do this right:

In [7]: %timeit pd.DataFrame(1.0, index=xs, columns=list('abcde'))
10000 loops, best of 3: 165 µs per loop

[bluss]

Yes, the actual use case does not write just the same value, but a value specific to the current index and column. I've noticed that if I do it wrong, building the DataFrame overshadows all other computation, but with .set_value it is fine (and .at too, slower but not dominating).