ENH: Expose integer formatting also for default display, not only styler

[enryH]

Feature Type

• Adding new functionality to pandas

• Changing existing functionality in pandas

• Removing existing functionality in pandas

Problem Description

I noticed that in Pandas version 2.2 the internal implementation of IntArrayFormatter was changed (or at least renamed), marking the classes as private using one underscore. I used the implementation as described here to format integers in a DataFrame on each display.

See changes between 2.1 and 2.2 to make the linked patch work: link

I see that the styler supports formatting thousands:

import pandas as pd
pd.options.styler.format.thousands = ','
s = pd.Series([1_000_000]).to_frame()
s.style # does change display of s to 1,000,000

however

s # will display 1000000

Would it be possible to add the same to the display options?

Feature Description

pd.options.format.thousands = ','

would need to pass:

if pd.options.format.thousands:
  format_str = f':{pd.options.format.thousands}d'.format
else:
  format_str = ':d'.format

Alternative Solutions

Change the default here

class IntArrayFormatter(pf._GenericArrayFormatter):

    def _format_strings(self):
        formatter = self.formatter or '{:,d}'.format
        fmt_values = [formatter(x) for x in self.values]
        return fmt_values

pf._IntArrayFormatter = IntArrayFormatter

Using the above patch I get the display I would prefer:

>>> pd.Series([1_000_000])
0    1,000,000
dtype: int64

Additional Context

Working with large integers in tables is made easier if thousands are visually distinguished. Maybe there is already a way to specify that formatter function (str.format) for display, which would be perfect.

[rhshadrach]

Thanks for the request. From your example:

s.style # does not change display of s

I'm seeing that this does display the comma separator, are you not? Or perhaps you meant that s without .style still does not show the commas?

[attack68]

i dont think this feature is about Styler. It is about adding functionality to the default DataFrame display that is demonstrated by Styler, i.e. which Styler already possesses.

[enryH]

Yes exactly, currently I have to use the internals to make s already represent with the commas.

>>> import pandas as pd
>>> pd.options.styler.format.thousands = ','  # here it would be great to have the option pd.options.display.thousands 
>>> s = pd.Series([1_000_000])

>>> s
0    1000000
dtype: int64
>>>s.to_frame().style
         0
0	1,000,000

I see that I put a "not" at the wrong place intially... Sorry about that. I'll clarify my example above by modifying it.

[rhshadrach]

I would personally find this feature convenient, I'm positive on it. @attack68 do you have any thoughts?

If we do implement this, I think it should only be implemented for integer/float dtypes, and documented as such. Users should not expect pandas to infer whether to insert commas in object dtype values.

[Meadiocre]

Hello @enryH,

My project team is looking for Pandas enhancement features for our grad school semester long project. We saw this task and would like to contribute if possible!

[attack68]

i think its generally quite handy to have configurable options for precision, thousands and decimal separator. Styler avoided using the locale package, if I recall correctly, but it was a bit tricky in the end to make sure the european decimal-comma was compatible with the us comma-decimal style.

To make it easier this should definitely only be applied to float/int dtypes.