CompareAndPullRequestPost() causes internal server error

[skaldesh]

Description

I have a repository with a vendor folder in the master branch. In separate feature branch, I deleted this folder and replaced it with a git submodule. I then tried to create a Pull Request in the gitea UI, which causes a 500 Internal Server Error:

Error in the logs:

2022/04/26 16:20:39 ...ers/web/repo/pull.go:1164:CompareAndPullRequestPost() [E] NewPullRequest: exit status 128 - vendor: unmerged (b9193c1afe1d0bd874d6f995bcbf1fc98d249588)
	fatal: git-write-tree: error building trees

Gitea Version

 1.16.6 built with GNU Make 4.3, go1.18.1 : bindata, timetzdata, sqlite, sqlite_unlock_notify

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

 2.30.3

Operating System

Official Gitea Docker image

How are you running Gitea?

Via docker-compose

Database

SQLite

[phostann]

The same error, wait for an answer.

[Fogapod]

Error persists on 1.17.0+rc1, git 2.30.2. Replaced git submodule with normal folder (just copied submodule contents in instead of using submodule) in feature branch and I cannot make a PR from that branch to main that has it.

2022/06/30 09:29:51 ...ers/web/repo/pull.go:1242:CompareAndPullRequestPost() [E] NewPullRequest: exit status 128 - core/utils/xhr.py: unmerged (091dbfdb1888b81e6628785cf00af192c80dd959)
        fatal: git-write-tree: error building trees
         - core/utils/xhr.py: unmerged (091dbfdb1888b81e6628785cf00af192c80dd959)
        fatal: git-write-tree: error building trees

2022/06/30 09:29:51 router: completed POST /org/repo/compare/master...replace-core for 1.2.3.4:0, 500 Internal Server Error in 181.8ms @ repo/pull.go:1124(repo.CompareAndPullRequestPost)

[LJFan]

When I new pull request (pull from branch A and merge into master), a similar error will appear.
But when I pull from master and merge into branch A, it succeeds.
It's weird.

2022/07/01 16:24:16 ...dules/git/command.go:146:RunWithContext() [D] D:/gitea/data/tmp/local-repo\pull.git1357600457: C:\Program Files\Git\cmd\git.exe -c credential.helper= -c protocol.version=2 -c uploadpack.allowfilter=true -c uploadpack.allowAnySHA1InWant=true -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= apply --check --cached --3way C:\WINDOWS\TEMP\patch1298657802
2022/07/01 16:24:16 ...ers/web/repo/pull.go:1164:CompareAndPullRequestPost() [E] NewPullRequest: git apply --check: exit status 1
2022/07/01 16:24:16 ...s/context/context.go:214:HTML() [D] Template: status/500

[SteadEXE]

When I try to merge master into my branch B, I have the same issue error 500.

[LJFan]

I have fixed it by using the git merge command to merge branch A into master locally.
Then I can new pull request (pull from branch A and merge into master).

[SteadEXE]

I did the same, I was stuck with the GUI

[zeripath]

It would be very helpful if you could create a simple test case to replicate this issue.

The likely problem is in TestPatch and is related to the three-way merge here.

The error will most likely be originating here:

gitea/services/pull/patch.go

Line 266 in 713bc6c

treeHash, err := git.NewCommandContext(ctx, "write-tree").RunInDir(tmpBasePath)

[zeripath]

Now the issue is how is one supposed to handle merging when a submodule is turned into a directory or vice versa?

[zeripath]

I have attempted to replicate this locally and on try.gitea.io and am unable to replicate on main.

https://try.gitea.io/arandomer/module-test/pulls/1

[parnic]

@zeripath and others - I am able to reproduce this on try.gitea.io (after running into it on our production server which is using Gitea v1.16.9). Here's what I did:

1. Create a repo with a file that will eventually be added as a submodule elsewhere: https://try.gitea.io/parnic/temp-repo
2. Create a repo that will eventually hold the submodule: https://try.gitea.io/parnic/submodule-replace-test
3. Add a directory that will be swapped out with the submodule afterward.
4. Create a branch that swaps out the directory with a submodule in the same place: https://try.gitea.io/parnic/submodule-replace-test/commits/branch/submodule-replace
5. Attempt to open a PR for that branch: https://try.gitea.io/parnic/submodule-replace-test/compare/main...submodule-replace
6. Click New Pull Request then Create Pull Request.
7. Get a 500 error (and, I assume, the same error on the server as is reported in this chain).

Is that enough to go off of? I can provide exact git commands if needed, but hopefully they're obvious from the above links.

[egasimus]

Create a branch that swaps out the directory with a submodule in the same place

Reproduced with same on local install - same thing on 1.16.8 and 1.16.9

Related: when trying to switch from the submodule branch to the directory branch, in order to do the same merge manually (replace the directory with the submodule), I get:

error: The following untracked working tree files would be overwritten by checkout:
  ... submodule files

Obviously this can be worked around by starting from a fresh checkout of the directory branch, but maybe the same thing is confusing git-write-tree? 🤷‍♂️

[zeripath]

OK adding some logging here, the problem appears to be that:

	160000 8d33ac46845bd6695b643385280e2d6fc2f945a6 3	temp

is left unmerged.

AHA I've found the bug