API: many custom errors / warnings are not exposed in pandas.errors

[jorisvandenbossche]

From #27553 and #27522, I quickly looked at how consistent we are with defining errors and warnings in pandas.errors.

So we have a pandas.errors module which is meant to publicly expose our custom exceptions and warnings.

Some are defined there such as PerformanceWarning, UnsortedIndexError, ParserError, ... (https://github.com/pandas-dev/pandas/blob/bb6135880e5e453d7701764b9f2e4ad3356a68d7/pandas/errors/__init__.py).
But many are not:

• tslibs.parsing.DateParseError
• tslibs.period.IncompatibleFrequency
• pandas/_config/config.py: OptionError
• pandas/core/base.py: DataError, GroupByError, SpeciicationError
• pandas/core/common.py: SettingWithCopyError/Warning
• pandas/core/computation/common/py: NameResolutionError
• pandas/core/computation/engine.py: NumExprClobberingError
• pandas/core/computation/ops.py: UndefinedVariableError
• pandas/core/indexes/base.py: InvalidIndexError
• pandas/coreindexing.py: IndexingError
• pandas/io/clipboard/exceptions.py: PyperclipException
• pandas/io/formats/css.py: CSSWarning
• pandas/io/msgpack: several exceptions
• pandas/io/pytables.py: lots of exceptions
• pandas/io/sql.py: SQLALchemyRequired, DatabaseError
• pandas/io/stata.py: several warnings

Do we want to move them all to the public pandas.errors module?
Are there reasons that some can / should not be moved? Eg should the cython ones be defined in the cython file for performance?
Are there certain custom exceptions/warnings that we rather convert in a built-in one instead of publicly exposing them?
Do we want to move all the io related ones? (those modules are somewhat public)

[jbrockmendel]

I think the _config and tslibs should be defined where they currently are and imported into pandas.errors. The others I don't have strong opinions on.

[jorisvandenbossche]

For the tslibs, do you think this will have a performance impact, or is it only for the "being self-contained" reason? (note that if all errors are defined in pandas.errors and does not import from anywhere else, this module is fully independent on pandas core as well)

[jbrockmendel]

My motivation here is the self-containment, yah. I think in the case of OutOfBoundsDatetime there was a structural reason why it couldn't be moved to pd.errors, but I don't remember it off the top of my head.

[jbrockmendel]

brainstorming how to write tests for this, we could do something like:

def get_subclasses_recursive(cls):
    result = []
    subs = cls.__subclasses__()
    result.extend(subs)
    for sub in subs:
        extra = get_subclasses_recursive(sub)
        result.extend(extra)
    return result

all_excs = get_subclasses_recursive(Exception)  # <-- locally after just importing pandas, I get 190 of these

ours = [x for x in all_excs if x.__module__.startswith("pandas.")]  # <-- i get 39 of these

We could use ours and blacklist internal-only classes for the parametrization in tests.test_errors.test_exception_importable

[dataxerik]

take

[dataxerik]

We could use ours and blacklist internal-only classes for the parametrization in tests.test_errors.test_exception_importable

@jbrockmendel

Hello, I just started to work on this and wanted to follow up on the this because I think I may be on the wrong page. Thank you for the code by the way.

I was initially thinking we would get the pandas exceptions that aren't in pandas.errors and fail them, unless they're intentionally not in pandas.errors. There could be a list for the exceptions that are allowed to not be in pandas.errors. But, I was also thinking that might be cumbersome to maintain and maybe the idea is not to enforce all exceptions in pandas.errors.

My idea doesn't seem to match yours. I'm not quite following how we use ours to blacklist the parametrized exceptions in test_exception_importable