ENH: Adding support for calamine as Excel reader engine

[abielr]

Feature Type

• Adding new functionality to pandas

• Changing existing functionality in pandas

• Removing existing functionality in pandas

Problem Description

Reading Excel files in Pandas is considerably slower than in some alternative data frame tools, for example the readxl package in R can read Excel files much faster. The Rust calamine library can read Excel files much faster than other engines supported by Pandas, and there is an existing Python binding to it, python-calamine. I would like to request that Pandas add official support for calamine, so that users can read an Excel file like:

pd.read_excel("test.xlsx", engine="calamine")

Feature Description

The python-calamine package already implements code that enables the calamine engine in Pandas, see the examples using pandas_monkeypatch() at the bottom of their Github README. The code to enable this is here

Although python-calamine already implements the necessary features to use the library with Pandas, I am unclear on how similar the behavior is between calamine and other engines that Pandas supports like openpyxl. I am hoping that by bringing calamine in as an officially supported engine that Pandas unit tests will confirm consistent behavior across calamine and other engines.

Alternative Solutions

None

Additional Context

No response

[lithomas1]

This looks pretty cool, especially since it seems to be able to read all sorts of excel formats(we might be able to kill off the other engines, like odfpy and pyxlsb).

I think a few weeks back, I profiled the Excel code, and most of the time seemed to be spent in openpyxl, as opposed to the parsing, so this'll definetely be an improvement.

PRs are welcome.

[kostyafarber]

I'd be happy to explore implementing this. Just to be sure are we happy to include introduce another dependency for this engine?

[lithomas1]

Yes, ideally we'd be able to deprecate of some of the other engines(e.g. pyxlsb, odfpy, xlrd), since calamine seems to support a lot more formats.

[kostyafarber]

Okay cool I'll have a look at 👍🏻