test_index_file_diffing fails with git 2.30.0

[mgorny]

$ git --version
git version 2.30.0

Seems to pass with 2.29.2.

======================================================================
ERROR: test_index_file_diffing (test.test_index.TestIndex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/GitPython/test/lib/helper.py", line 140, in repo_creator
    return func(self, rw_repo)
  File "/tmp/GitPython/test/test_index.py", line 378, in test_index_file_diffing
    self.assertRaises(CheckoutError, index.checkout, "doesnt_exist_ever.txt.that")
  File "/usr/lib/python3.9/unittest/case.py", line 733, in assertRaises
    return context.handle('assertRaises', args, kwargs)
  File "/usr/lib/python3.9/unittest/case.py", line 201, in handle
    callable_obj(*args, **kwargs)
  File "/tmp/GitPython/git/index/util.py", line 76, in check_default_index
    return func(self, *args, **kwargs)
  File "/tmp/GitPython/git/index/base.py", line 1133, in checkout
    self._flush_stdin_and_wait(proc, ignore_stdout=True)
  File "/tmp/GitPython/git/index/base.py", line 980, in _flush_stdin_and_wait
    proc.wait()
  File "/tmp/GitPython/git/cmd.py", line 408, in wait
    raise GitCommandError(self.args, status, errstr)
git.exc.GitCommandError: Cmd('/usr/bin/git') failed due to: exit code(1)
  cmdline: /usr/bin/git checkout-index --index --stdin
  stderr: 'git checkout-index: doesnt_exist_ever.txt.that is not in the cache
'

[Byron]

Which gitpython version are you using?

[mgorny]

Cloned just before filing the report:

$ git rev-parse HEAD
3dd71d3edbf3930cce953736e026ac3c90dd2e59

[Byron]

Maybe it's related to the recent change in how diffs work: 69b75e1 (and the two commits before that one). Maybe you could try that with the previous gitpython release to test this idea.

It's strange though that there is a difference in git versions that seems to trigger this behaviour.

[mgorny]

I don't seem to get the previous tag to work but I've reverted 69b75e1 and the two commits before it, and the test fails the same.

[Byron]

Thanks, it's good to know the recent modifications didn't cause subtle breakage.

self.assertRaises(CheckoutError, index.checkout, "doesnt_exist_ever.txt.that")

An assertion to an exception being thrown if a file that doesn't exist is checked out. An exception is indeed thrown, yet it's unhappy. Is it related to Python not liking the exception that is thrown? Because to git it's all the same, it's a process which doesn't exit with 0 which in turn gets handled in Python.
It's a bit puzzling.

[mgorny]

Were you able to install git 2.30 and reproduce?

[Byron]

Yes, I can reproduce it, here is the logs:

➜  GitPython git:(master) git --version                                                            git version 2.24.3 (Apple Git-128)
➜  GitPython git:(master) nosetests test/test_index.py                                             .................
----------------------------------------------------------------------
Ran 17 tests in 4.633s

OK
➜  GitPython git:(master)
➜  GitPython git:(master) /Users/byron/dev/git/git --version
git version 2.30.0
➜  GitPython git:(master) GIT_PYTHON_GIT_EXECUTABLE=/Users/byron/dev/git/git nosetests test/test_index.py
........E........
======================================================================
ERROR: test_index_file_diffing (test.test_index.TestIndex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/byron/dev/GitPython/test/lib/helper.py", line 140, in repo_creator
    return func(self, rw_repo)
  File "/Users/byron/dev/GitPython/test/test_index.py", line 378, in test_index_file_diffing
    self.assertRaises(CheckoutError, index.checkout, "doesnt_exist_ever.txt.that")
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/unittest/case.py", line 816, in assertRaises
    return context.handle('assertRaises', args, kwargs)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/unittest/case.py", line 202, in handle
    callable_obj(*args, **kwargs)
  File "/Users/byron/dev/GitPython/git/index/util.py", line 76, in check_default_index
    return func(self, *args, **kwargs)
  File "/Users/byron/dev/GitPython/git/index/base.py", line 1133, in checkout
    self._flush_stdin_and_wait(proc, ignore_stdout=True)
  File "/Users/byron/dev/GitPython/git/index/base.py", line 980, in _flush_stdin_and_wait
    proc.wait()
  File "/Users/byron/dev/GitPython/git/cmd.py", line 408, in wait
    raise GitCommandError(self.args, status, errstr)
git.exc.GitCommandError: Cmd('/Users/byron/dev/git/git') failed due to: exit code(1)
  cmdline: /Users/byron/dev/git/git checkout-index --index --stdin
  stderr: 'git checkout-index: doesnt_exist_ever.txt.that is not in the cache
'
-------------------- >> begin captured logging << --------------------
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'clone', '-n', '--shared', '-v', '/Users/byron/dev/GitPython/.git', '/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6'], cwd=/Users/byron/dev/GitPython, universal_newlines=True, shell=None, istream=None)
git.repo.base: DEBUG: Cmd(['/Users/byron/dev/git/git', 'clone', '-n', '--shared', '-v', '/Users/byron/dev/GitPython/.git', '/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6'])'s unused stdout:
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'cat-file', '--batch-check'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=<valid stream>)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'checkout', 'master'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'reset', '--mixed', 'HEAD~6', '--'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'diff', '-R', '79cfe05054de8a31758eb3de74532ed422c8da27', '--cached', '--abbrev=40', '--full-index', '-M', '--raw', '-z', '--no-color'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'diff', '-R', '9a4b1d4d11eee3c5362a4152216376e634bd14cf', '--cached', '--abbrev=40', '--full-index', '-M', '--raw', '-z', '--no-color'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'cat-file', '--batch'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=<valid stream>)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'diff', '9a4b1d4d11eee3c5362a4152216376e634bd14cf', '--cached', '--abbrev=40', '--full-index', '-M', '--raw', '-z', '--no-color'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'diff', '-R', '9a4b1d4d11eee3c5362a4152216376e634bd14cf', '--cached', '--abbrev=40', '--full-index', '-M', '--raw', '-z', '--no-color'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'diff', '--abbrev=40', '--full-index', '-M', '--raw', '-z', '--no-color'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'read-tree', '--index-output=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6/.git/7v6h9n6e', 'HEAD~1'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'diff', '--abbrev=40', '--full-index', '-M', '--raw', '-z', '--no-color'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'read-tree', '--index-output=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6/.git/5zz0cn0t', 'HEAD~1'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'checkout-index', '--index', '--force', '--all'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'diff', '--abbrev=40', '--full-index', '-M', '--raw', '-z', '--no-color'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'checkout-index', '--index', '--force', '--all'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'checkout-index', '--index', '--all'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'checkout-index', '--index', '--stdin'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=<valid stream>)
git.cmd: DEBUG: Popen(['/Users/byron/dev/git/git', 'checkout-index', '--index', '--stdin'], cwd=/var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6, universal_newlines=False, shell=None, istream=<valid stream>)
git.cmd: DEBUG: AutoInterrupt wait stderr: b'git checkout-index: doesnt_exist_ever.txt.that is not in the cache\n'
test.lib.helper: INFO: Keeping repo after failure: /var/folders/03/pf3rx1yx0sn7lnwpcb_nq6c80000gn/T/non_bare_test_index_file_diffingdegmviq6
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 17 tests in 3.300s

FAILED (errors=1)

[Byron]

Ah, right, the assertion expects a CheckoutError, but gets a GitCmdError. Probably the error handling in git checkout-index was changed and now makes the program crash.

@mgorny I left some notes in #1104 which might help someone finalize the fix. As the GitCommandError has access to stderr, it should be possible to parse stderr lines and retrieve the necessary information from there.
Alternatively, one could make the test a little less pedantic assuming nobody really cares about exceptions anyway so they won't be handled in such detail, i.e. checking for which files actually failed.

[Byron]

The PR has been fixed up to 'make' the test succeed, which also fixes this issue.