ENH: Reimplement and undeprecate DataFrame.lookup

[impredicative]

Is your feature request related to a problem?

I seriously get the impression that Pandas is actively being sabotaged. As a case in point, DataFrame.lookup was deprecated in v1.2. The problems with this are:

1. The doc for DataFrame.lookup says to see itself for an example. Well, there is no example in the doc page. The only example I'm aware of is here instead which is a different page.
2. The changelog for this deprecation points to GH18682 which looks to be wholly irrelevant to this change. A lot has gone wrong here.
3. There is significant user code that already uses lookup. Why break it? If the current implementation of lookup is suboptimal, shouldn't it be optimized instead?
4. It is extremely complicated to use melt when lookup works quite simply. For example, compare this simple answer using lookup with this complicated answer using melt.

Does nobody review changes, docs, and release notes anymore prior to the release? It looks this way.

Describe the solution you'd like

1. Optimize lookup if attainable using melt or otherwise.
2. Undeprecate lookup.

[jreback]

Does nobody review changes, docs, and release notes anymore prior to the release? It looks this way.

this is not a friendly way to ask for things.

DataFrame.lookup has seen NO love at all since its initial implemention. It is a duplicative and unmaintained function. This is also polluting the namespace and is not in any way integrated to all of the other indexers. We do not need N ways to do the same thing.

[jreback]

#35224 is the PR and #18262 is the issue

[jreback]

@impredicative you are welcome to contribute patches, e.g. doc fixes or other things.

[erfannariman]

@impredicative out of curiosity, what is exactly difficult about the new proposed method, which is also significantly faster since DataFrame.lookup was a for-loop on the background.

I agree that we can fix the documentation in the deprecated message and in the whatsnew though.

@jreback Would you be open to un-deprecate DataFrame.lookup and implement the melt + loc method on the background? I don't say it was an extremely popular method, but on SO I saw it being used sometimes.

[impredicative]

@erfannariman What's difficult, for example, is how complicated this answer using melt is relative to how simple this answer using lookup is. The answer using melt requires three complicated method calls relative to the answer using lookup which is a single simple method call. In summary, just as you're saying, DataFrame.lookup could've been reimplemented using melt, thereby satisfying everyone.

[erfannariman]

@erfannariman What's difficult, for example, is how complicated this answer using melt is relative to how simple this answer using lookup is. The answer using melt requires three complicated method calls relative to the answer using lookup which is a single simple method call. In summary, just as you're saying, DataFrame.lookup could've been reimplemented using melt, thereby satisfying everyone.

I don't disagree with your point, although I don't think the proposed method is persé difficult, but I might be biased. I would not be against un-deprecating DataFrame.lookup. I will await @jreback response.

[impredicative]

Pandas is used because it is believed to have a wealth of friendly methods. If this belief erodes, so will its user base.

[jreback]

@impredicative there is almost no usage of lookup AFAICT. very very few issues / tests / SO entries. If you have a valid, real work very common then am happy to show a doc example / recipe. Having a method must be a high bar.

[impredicative]

@jreback A google.com search for "df.lookup" site:stackoverflow.com lists 462 hits. That's more than for df.bfill (208) and df.ffill (375). Will you now be deprecating these two methods as well? What is your cutoff?

[jreback]

@impredicative you are not comparing apples to apples here. try with indexing operators, we already have .loc, .iloc, .at, and .iat, not to mention partial slicing and of course []. which one shall we cut to keep .lookup????

I in fact have an issue to remove .iat and .at. We simply do not need this much choice.