Skip to content

Unified common dtype discovery #13947

Closed
Closed
@sstanovnik

Description

@sstanovnik

Common dtype discovery (like np.find_common_type) should be unified into an internal function, with proper handling of pandas dtypes. As of #13917, there are two such implementations:

  1. pandas/types/common.py:_lcd_dtypes, which works on numpy dtypes only, but differently than np.find_common_type.
  2. pandas/types/cast.py:_find_common_type, which just uses np.find_common_type internally.

Exchanging the first one with the second (which is, judging by the code, more proper), breaks some tests.

Proposed tasks:

  • Convert usages of _lcd_dtypes into _find_common_types.
  • Extend _find_common_types to properly evaluate pandas dtypes.

Occurences of _lcd_dtypes, the one in internals.py is removed by #13917:

[jreback-~/pandas] grin _lcd_dtype pandas
pandas/core/frame.py:
   47 :                                  _lcd_dtypes,
 3705 :                 new_dtype = _lcd_dtypes(this_dtype, other_dtype)
pandas/core/internals.py:
 4438 :     def _lcd_dtype(l):
 4473 :         lcd = _lcd_dtype(counts[IntBlock])
 4489 :         return _lcd_dtype(counts[FloatBlock] + counts[SparseBlock])
pandas/types/common.py:
  389 : def _lcd_dtypes(a_dtype, b_dtype):

Metadata

Metadata

Assignees

No one assigned

    Labels

    Compatpandas objects compatability with Numpy or Python functionsDtype ConversionsUnexpected or buggy dtype conversions

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions