BUG: Incorrect stack traceback when try except finally is used in jupyter notebooks.

[dvorst]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

# Example incorrect stack traceback
import pandas as pd

def g():
    pd.DataFrame().reshape(1, 1)

try:
    g()
except Exception as e:
    print("except")
    raise
finally:
    print("finally")

########################################################################
# Example expected stack traceback
def f():
    return {}["a"]


try:
    f()
except:
    print("except")
    raise
finally:
    print("finally")

Issue Description

The stack traceback of both examples are shown below, I showed these in both plain text as well as html. The incorrect stacktrace points to the last statement run in the finally block (i.e. print("finally")). If i remove the finally block it will point to the raise statement in the except block. It should however, point to the function call g(). The example with the correct stacktrace correctly points to the function call f().

Reason why i included the stack traceback in both plain text as well as html, is because I noticed the the correct example contains hyperlinks to the cell-lines. This also leads to my gut feeling telling me this is a bug with pandas rather than Jupyter notebook, maybe this is a hint to what causes the bug.

To exclude the possibility that these hyperlinks only exist if the code is fully contained inside a notebook, I moved the function f() to an external .py file. When doing so, the hyperlinks will still exist.

Since the stack trace does point to the statement pd.DataFrame().reshape(1, 1), I would consider this to be a minor bug.

Stack trace plain text

Output example incorrect stack traceback:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/var/folders/j4/hhybm8xx3pb7nnmwfptdqng00000gp/T/ipykernel_96700/1557906605.py in ?()
      8 except Exception as e:
      9     print("except")
     10     raise
     11 finally:
---> 12     print("finally")

/var/folders/j4/hhybm8xx3pb7nnmwfptdqng00000gp/T/ipykernel_96700/1557906605.py in ?()
      3 def g():
----> 4     pd.DataFrame().reshape(1, 1)

~/Documents/projects/tmp/.venv/lib/python3.11/site-packages/pandas/core/generic.py in ?(self, name)
   6200             and name not in self._accessors
   6201             and self._info_axis._can_hold_identifiers_and_holds_name(name)
   6202         ):
   6203             return self[name]
-> 6204         return object.__getattribute__(self, name)

AttributeError: 'DataFrame' object has no attribute 'reshape'

output example incorrect stack traceback:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
[/Users/user/Documents/projects/tmp/test.ipynb](https://file+.vscode-resource.vscode-cdn.net/Users/user/Documents/projects/tmp/test.ipynb) Cell 2 line 6
      2     return {}["a"]
      5 try:
----> 6     f()
      7 except:
      8     print("except")

/Users/user/Documents/projects/tmp/test.ipynb Cell 2 line 2
      1 def f():
----> 2     return {}["a"]

KeyError: 'a'

stack trace HTML

Output example incorrect stack traceback

notice how each code-line is NOT a hyperlink:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[/var/folders/j4/hhybm8xx3pb7nnmwfptdqng00000gp/T/ipykernel_96700/1557906605.py](https://file+.vscode-resource.vscode-cdn.net/var/folders/j4/hhybm8xx3pb7nnmwfptdqng00000gp/T/ipykernel_96700/1557906605.py) in ?()
      8 except Exception as e:
      9     print("except")
     10     raise
     11 finally:
---> 12     print("finally")

[/var/folders/j4/hhybm8xx3pb7nnmwfptdqng00000gp/T/ipykernel_96700/1557906605.py](https://file+.vscode-resource.vscode-cdn.net/var/folders/j4/hhybm8xx3pb7nnmwfptdqng00000gp/T/ipykernel_96700/1557906605.py) in ?()
      3 def g():
----> 4     pd.DataFrame().reshape(1, 1)

[~/Documents/projects/tmp/.venv/lib/python3.11/site-packages/pandas/core/generic.py](https://file+.vscode-resource.vscode-cdn.net/Users/user/Documents/projects/tmp/~/Documents/projects/tmp/.venv/lib/python3.11/site-packages/pandas/core/generic.py) in ?(self, name)
   6200             and name not in self._accessors
   6201             and self._info_axis._can_hold_identifiers_and_holds_name(name)
   6202         ):
   6203             return self[name]
-> 6204         return object.__getattribute__(self, name)

AttributeError: 'DataFrame' object has no attribute 'reshape'

output example incorrect stack traceback:

notice how each code-line IS a hyperlink:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
[/Users/user/Documents/projects/tmp/test.ipynb](https://file+.vscode-resource.vscode-cdn.net/Users/user/Documents/projects/tmp/test.ipynb) Cell 2 line 6
      [2](vscode-notebook-cell:/Users/user/Documents/projects/tmp/test.ipynb#W0sZmlsZQ%3D%3D?line=1)     return {}["a"]
      [5](vscode-notebook-cell:/Users/user/Documents/projects/tmp/test.ipynb#W0sZmlsZQ%3D%3D?line=4) try:
----> [6](vscode-notebook-cell:/Users/user/Documents/projects/tmp/test.ipynb#W0sZmlsZQ%3D%3D?line=5)     f()
      [7](vscode-notebook-cell:/Users/user/Documents/projects/tmp/test.ipynb#W0sZmlsZQ%3D%3D?line=6) except:
      [8](vscode-notebook-cell:/Users/user/Documents/projects/tmp/test.ipynb#W0sZmlsZQ%3D%3D?line=7)     print("except")

[/Users/user/Documents/projects/tmp/test.ipynb](https://file+.vscode-resource.vscode-cdn.net/Users/user/Documents/projects/tmp/test.ipynb) Cell 2 line 2
      [1](vscode-notebook-cell:/Users/user/Documents/projects/tmp/test.ipynb#W0sZmlsZQ%3D%3D?line=0) def f():
----> [2](vscode-notebook-cell:/Users/user/Documents/projects/tmp/test.ipynb#W0sZmlsZQ%3D%3D?line=1)     return {}["a"]

KeyError: 'a'

Expected Behavior

The stack trace should point to the correct statement in the try block.

Installed Versions

pd.show_versions()

INSTALLED VERSIONS

commit : 2a953cf
python : 3.11.6.final.0
python-bits : 64
OS : Darwin
OS-release : 23.1.0
Version : Darwin Kernel Version 23.1.0: Mon Oct 9 21:28:31 PDT 2023; root:xnu-10002.41.9~6/RELEASE_ARM64_T8112
machine : arm64
processor : arm
byteorder : little
LC_ALL : None
LANG : None
LOCALE : None.UTF-8

pandas : 2.1.3
numpy : 1.26.2
pytz : 2023.3.post1
dateutil : 2.8.2
setuptools : 68.2.2
pip : 23.2.1
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : 8.18.1
pandas_datareader : None
bs4 : None
bottleneck : None
dataframe-api-compat: None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2023.3
qtpy : None
pyqt5 : None

About Visual Studio Code

Version: 1.84.2
Commit: 1a5daa3a0231a0fbba4f14db7ec463cf99d7768e
Date: 2023-11-09T10:52:57.054Z
Electron: 25.9.2
ElectronBuildId: 24603566
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Darwin arm64 23.1.0

IPython.sys_info()

'commit_hash': '49914f938'
'commit_source': 'installation'
'default_encoding': 'utf-8'
'ipython_path': '/Users/user/Documents/projects/tmp/.venv/lib/python3.11/site-packages/IPython'
'ipython_version': '8.18.1'
'os_name': 'posix'
'platform': 'macOS-14.1.1-arm64-arm-64bit'
'sys_executable': '/Users/user/Documents/projects/tmp/.venv/bin/python'
'sys_platform': 'darwin'
'sys_version': '3.11.6 (main, Oct 2 2023, 20:46:14) [Clang 14.0.3 '
'(clang-1403.0.22.14.1)]'

[rhshadrach]

Thanks for the report! Reproduced on main with Python 3.10 and the following Jupyter packages

Selected Jupyter core packages...
IPython          : 8.14.0
ipykernel        : 6.23.3
ipywidgets       : 8.0.6
jupyter_client   : 8.3.0
jupyter_core     : 5.3.1
jupyter_server   : 2.7.0
jupyterlab       : 4.0.7
nbclient         : 0.8.0
nbconvert        : 7.6.0
nbformat         : 5.9.0
notebook         : 6.5.4
qtconsole        : 5.4.4
traitlets        : 5.9.0

Reason why i included the stack traceback in both plain text as well as html, is because I noticed the the correct example contains hyperlinks to the cell-lines. This also leads to my gut feeling telling me this is a bug with pandas rather than Jupyter notebook...

Can you elaborate on this? I don't see how this could indicate it's a problem with pandas.

This occurs with pandas Series and DataFrame but not Index. It also doesn't happen with e.g. NumPy ndarray. I don't see how pandas can have any influence on this and suspect it may be Jupyter / IPython not successfully handling some type of interrogation of Series / DataFrame instances.

[lithomas1]

Agreed with Richard.

Can you try reporting this issue to IPython/Jupyter? I don't think we really control how the exceptions are rendered.