Remove duplicate is_lexsorted function

[jbrockmendel]

_libs.lib and _libs.algos have near-identical is_lexsorted functions. The only differences appear to be small optimizations/modernizations in the algos version. AFAICT the algos version is only used in tests.test_algos ATM. This PR removes the libs._lib version and changes the one usage (in indexes.multi) to use the algos version.

[jreback]

no problem removing can you post a timeit with the perf comparison? (of the is_lexsorted of each version), do we have an asv to cover this?

[jbrockmendel]

Weird results from timeit, using one of the examples in asv_bench.multiindex_object and mimicing the usage in MultiIndex.lexsort_depth:

import string
import numpy as np
import pandas as pd
from pandas.core.dtypes.common import _ensure_int64
from pandas._libs import lib, algos

mi_large = pd.MultiIndex.from_product(
            [np.arange(1000), np.arange(20), list(string.ascii_letters)],
            names=['one', 'two', 'three'])

self = mi_large
int64_labels = [_ensure_int64(lab) for lab in self.labels]

k = 3
%timeit lib.is_lexsorted(int64_labels[:k])
%timeit algos.is_lexsorted(int64_labels[:k])

For k=1, 2, the perf is effectively indistinguishable between the two versions. For k=3 the libalgos version (the one this PR prefers) is 5000 times slower:

In [48]: %timeit lib.is_lexsorted(int64_labels[:k])
The slowest run took 5.17 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 1.75 µs per loop

In [51]: %timeit algos.is_lexsorted(int64_labels[:k])
100 loops, best of 3: 9.36 ms per loop

AFAICT the difference is in the fact that in the k=3 case, lib version returns False, while the algos version calls free and then returns False. In the k=1,2 cases both versions call free.

I'd guess that calling free is the right thing to do here, emphasis on "guess".

[codecov]

Codecov Report

Merging #19305 into master will decrease coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #19305      +/-   ##
==========================================
- Coverage   91.52%    91.5%   -0.03%     
==========================================
  Files         150      150              
  Lines       48875    48875              
==========================================
- Hits        44733    44721      -12     
- Misses       4142     4154      +12

Flag	Coverage Δ
#multiple	89.87% <100%> (-0.03%)	⬇️
#single	41.66% <100%> (ø)	⬆️

Impacted Files	Coverage Δ
pandas/core/indexes/multi.py	96.22% <100%> (ø)	⬆️
pandas/plotting/_converter.py	65.22% <0%> (-1.74%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca2d261...b4d3e8f. Read the comment docs.

[jreback]

hmm, that's a memory leak in lib.pyx, ok let's go with the algos one.