LFS files in forks fail to download via separate remote or PR ref

[parnic]

Gitea Version

1.15.6

Git Version

Seen on different servers running 2.25.1 and 2.34.0

Operating System

Ubuntu 20.04.3

How are you running Gitea?

One server is manually deployed from a GitHub release and started with a systemd script. The other uses the apt-installed version of Gitea.

Database

PostgreSQL

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

I can gather a log if that would be helpful. I've included clear repro steps in the description.

Description

LFS files are failing to download from forks. The only reason I can't repro on try.gitea.io is because LFS doesn't seem to be enabled. This is my minimal repro (visible at https://git.pernicious.games/try-gitea-lfs/lfs-fork-test and the fork at https://git.pernicious.games/cpickett/lfs-fork-test/src/branch/fork-1).

Note that we are pretty sure this used to work on our Gitea installation months ago, so it may be caused by a relatively recent Gitea version (maybe 1.15+).

The repro steps are:

• Create new repo with an LFS file
• Fork that repo
• Commit a new rev of the LFS file in a branch on the fork
• Open a PR for the main repo from the fork's branch

Now clone the base repo somewhere else and try to either pull from the remote or the PR ref:

• Clone base repo (e.g. git clone https://git.pernicious.games/try-gitea-lfs/lfs-fork-test.git)
• Add the fork as a remote and fetch it (e.g.: git remote add parnic https://git.pernicious.games/cpickett/lfs-fork-test.git, git fetch parnic)
• Try switching to the branch that exists in the fork with the modified LFS file (e.g.: git switch fork-1)
  • Receive error, e.g.:

Downloading test.lfs (9 B)
Error downloading object: test.lfs (7ffc8bb): Smudge error: Error downloading test.lfs (7ffc8bb3b0c443a3d170b1f6aa402a66c6de86c49dd2d74e452d6d82b855939f): [7ffc8bb3b0c443a3d170b1f6aa402a66c6de86c49dd2d74e452d6d82b855939f] Not Found: [404] Not Found

• Restore modified files and try switching to the PR ref instead (e.g.: git restore ., git fetch origin pull/1/head:test, git switch test)
  • Receive same error as above

Note that the LFS file at the specified revision/hash exists in the fork, e.g.: https://git.pernicious.games/cpickett/lfs-fork-test.git/info/lfs/objects/7ffc8bb3b0c443a3d170b1f6aa402a66c6de86c49dd2d74e452d6d82b855939f/direct
but it does not exist in the base repo: https://git.pernicious.games/try-gitea-lfs/lfs-fork-test.git/info/lfs/objects/7ffc8bb3b0c443a3d170b1f6aa402a66c6de86c49dd2d74e452d6d82b855939f/direct

This is important because when running git lfs fetch --all we see that it is trying to pull from the base org, not the fork:

>git lfs fetch --all
fetch: 2 object(s) found, done.
fetch: Fetching all references...
[7ffc8bb3b0c443a3d170b1f6aa402a66c6de86c49dd2d74e452d6d82b855939f] Not Found: [404] Not Found
error: failed to fetch some objects from 'https://git.pernicious.games/try-gitea-lfs/lfs-fork-test.git/info/lfs'

Finally, note that the LFS file does exist on the disk under the generic LFS directory. For this example, that's
/var/lib/gitea/data/lfs/7f/fc/8bb3b0c443a3d170b1f6aa402a66c6de86c49dd2d74e452d6d82b855939f

Screenshots

No response

[wxiaoguang]

Can @zeripath or @KN4CK3R or someone else help to confirm?

IIRC the LFS code gets some changes during 1.14 -> 1.15, I am not sure whether this is a permission problem or a database consistency problem.

[KN4CK3R]

Are you sure that worked before? After creating the PR the base repository contains the dangling pointer without the database reference:

(top = base, bottom = pr branch)

During 1.14 -> 1.15 I changed the LFS API code but this looks like we need to sync the LFS objects when creating a PR. And that functionality was never implemented I think.

[KN4CK3R]

Looks like, I was wrong. I tested 1.12 and it works with that version (but is not correct I think). After creating the PR blob list does only show a single pointer (that's wrong because of the new PR branch?). Seems like the download request associated the pointer, so my changes in #16865 or #15523 may be the reason of the problem.

[KN4CK3R]

The old code created the database references when downloading files. I changed that in #15523 so that only upload requests are able to do that.

Reasons:

• The download request should just not do that (?)
• To associate the file only Read-rights were enforced
• The user could associate every file from different repositories (private too) to this repo if he knows the pointer data (was fixed in Test if LFS object is accessible #16865)

So the question is how we would like to handle this problem. Personally I don't like to bring back file association in the download request. Auto-sync when creating a PR?

[lunny]

The old code created the database references when downloading files. I changed that in #15523 so that only upload requests are able to do that.

Reasons:

• The download request should just not do that (?)
• To associate the file only Read-rights were enforced
• The user could associate every file from different repositories (private too) to this repo if he knows the pointer data (was fixed in Test if LFS object is accessible #16865)

So the question is how we would like to handle this problem. Personally I don't like to bring back file association in the download request. Auto-sync when creating a PR?

I vote later.

[wxiaoguang]

And just FYI, I remember this issue: #17207 , are they related? That issue also said about the behavior differs between Gitea release.

[KN4CK3R]

Yes, I think they are related. Just putting the files in the directory does not associate the repo to them.

[stu1811]

I agree they are probably related and I've run into this issue as well. As a workaround you can do a git lfs fetch <UPSTREAM> and then a git lfs checkout to get the files.

[parnic]

We are hosting a private Gitea server behind a VPN, so I've temporarily reintroduced the associate-LFS-pointers-on-download code for a hacked-up local build, in case anyone else needs the temporary workaround: https://github.com/go-gitea/gitea/compare/v1.15.6...parnic-sks:associate-lfs-obj-on-download?expand=1

[stu1811]

And just FYI, I remember this issue: #17207 , are they related? That issue also said about the behavior differs between Gitea release.

I built 1.15.6 cherry-picked the changes from parnic-sks@768aaab and it did not fix issue #17207. Detail below.

We got a new bundle with LFS updates. I extracted the new LFS files, fetched the branch, and tried to push to origin. It would not push due hint: Your push was rejected due to missing or corrupt local objects. I also tried git lfs fetch --all and the push failed again.

[parnic]

And just FYI, I remember this issue: #17207 , are they related? That issue also said about the behavior differs between Gitea release.

I built 1.15.6 cherry-picked the changes from parnic-sks@768aaab and it did not fix issue #17207. Detail below.

We got a new bundle with LFS updates. I extracted the new LFS files, fetched the branch, and tried to push to origin. It would not push due hint: Your push was rejected due to missing or corrupt local objects. I also tried git lfs fetch --all and the push failed again.

That one commit is not enough, you need the entire branch diff. parnic-sks@3872374 was actually the correct fix, but it needs the commit after that to fix the nil user deref.

[stu1811]

And just FYI, I remember this issue: #17207 , are they related? That issue also said about the behavior differs between Gitea release.

I built 1.15.6 cherry-picked the changes from parnic-sks@768aaab and it did not fix issue #17207. Detail below.
We got a new bundle with LFS updates. I extracted the new LFS files, fetched the branch, and tried to push to origin. It would not push due hint: Your push was rejected due to missing or corrupt local objects. I also tried git lfs fetch --all and the push failed again.

That one commit is not enough, you need the entire branch diff. parnic-sks@3872374 was actually the correct fix, but it needs the commit after that to fix the nil user deref.

Sorry I mean I cherry-picked the whole branch.