refs/notes/* refused by pre-receive hook at the remote

[wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf]

Gitea version: Gitea Version: 1.16.0+dev-104-g3a6edd368
git: git version 2.33.0.rc2.250.ged5fa647cd
built from: sources
run with: systemd
OS: Arch Linux

I am getting this when I try to push notes with git push -u origin refs/notes/"*":refs/notes/"*", TLDR pre-receive hook rejects them.
as @zeripath noted, this might be indicative of a regression.

log (click me pls)

Aug 13 17:30:46 nebula gitea[3105419]: 2021/08/13 17:30:46 Started GET /api/internal/serv/command/3/wanderer/gitea-git-notes-test?mode=2&verb=git-receive-pack for 127.0.0.1:0
Aug 13 17:30:46 nebula gitea[3105419]: 2021/08/13 17:30:46 Completed GET /api/internal/serv/command/3/wanderer/gitea-git-notes-test?mode=2&verb=git-receive-pack 200 OK in 11.35454ms
Aug 13 17:30:47 nebula gitea[3105419]: 2021/08/13 17:30:47 Started POST /api/internal/hook/pre-receive/wanderer/gitea-git-notes-test for 127.0.0.1:0
Aug 13 17:30:47 nebula gitea[3105419]: 2021/08/13 17:30:47 ...ters/private/hook.go:470:HookPreReceive() [E] Unexpected ref: refs/notes/commits
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/routers/private/hook.go:470 (0x21ad554)
Aug 13 17:30:47 nebula gitea[3105419]:                 HookPreReceive: log.Error("Unexpected ref: %s", refFullName)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/modules/web/route.go:76 (0x203cfbb)
Aug 13 17:30:47 nebula gitea[3105419]:                 Wrap.func1: t(ctx)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:30:47 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/go-chi/chi/mux.go:436 (0x16f296a)
Aug 13 17:30:47 nebula gitea[3105419]:                 (*Mux).routeHTTP: h.ServeHTTP(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:30:47 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/routers/private/internal.go:31 (0x21c03a4)
Aug 13 17:30:47 nebula gitea[3105419]:                 CheckInternalToken.func1: next.ServeHTTP(w, req)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:30:47 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/modules/context/private.go:42 (0x171494f)
Aug 13 17:30:47 nebula gitea[3105419]:                 PrivateContexter.func1.1: next.ServeHTTP(ctx.Resp, ctx.Req)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:30:47 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/go-chi/chi/mux.go:70 (0x16f04ca)
Aug 13 17:30:47 nebula gitea[3105419]:                 (*Mux).ServeHTTP: mx.handler.ServeHTTP(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/go-chi/chi/mux.go:311 (0x16f68bb)
Aug 13 17:30:47 nebula gitea[3105419]:                 (*Mux).Mount.func1: handler.ServeHTTP(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:30:47 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/go-chi/chi/mux.go:436 (0x16f296a)
Aug 13 17:30:47 nebula gitea[3105419]:                 (*Mux).routeHTTP: h.ServeHTTP(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:30:47 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/routers/common/middleware.go:72 (0x20651bd)
Aug 13 17:30:47 nebula gitea[3105419]:                 Middlewares.func2.1: next.ServeHTTP(resp, req)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:30:47 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/routers/common/logger.go:23 (0x20643fb)
Aug 13 17:30:47 nebula gitea[3105419]:                 LoggerHandler.func1.1: next.ServeHTTP(w, req)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:30:47 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/go-chi/chi/middleware/strip.go:30 (0x2060fe7)
Aug 13 17:30:47 nebula gitea[3105419]:                 StripSlashes.func1: next.ServeHTTP(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:30:47 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/chi-middleware/proxy/middleware.go:37 (0x205c64e)
Aug 13 17:30:47 nebula gitea[3105419]:                 ForwardedHeaders.func1.1: h.ServeHTTP(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:30:47 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/routers/common/middleware.go:25 (0x2064cfc)
Aug 13 17:30:47 nebula gitea[3105419]:                 Middlewares.func1.1: next.ServeHTTP(context.NewResponse(resp), req)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:30:47 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/go-chi/chi/mux.go:87 (0x16f0250)
Aug 13 17:30:47 nebula gitea[3105419]:                 (*Mux).ServeHTTP: mx.handler.ServeHTTP(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/modules/web/route.go:318 (0x203c333)
Aug 13 17:30:47 nebula gitea[3105419]:                 (*Route).ServeHTTP: r.R.ServeHTTP(w, req)
Aug 13 17:30:47 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/gorilla/context/context.go:141 (0x130d5d3)
Aug 13 17:30:47 nebula gitea[3105419]:                 ClearHandler.func1: h.ServeHTTP(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:30:47 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2867 (0x7bbca2)
Aug 13 17:30:47 nebula gitea[3105419]:                 serverHandler.ServeHTTP: handler.ServeHTTP(rw, req)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:1932 (0x7b71cc)
Aug 13 17:30:47 nebula gitea[3105419]:                 (*conn).serve: serverHandler{c.server}.ServeHTTP(w, w.req)
Aug 13 17:30:47 nebula gitea[3105419]:         /usr/lib/go/src/runtime/asm_amd64.s:1371 (0x4747c0)
Aug 13 17:30:47 nebula gitea[3105419]:                 goexit: BYTE        $0x90        // NOP
Aug 13 17:30:47 nebula gitea[3105419]:
Aug 13 17:30:47 nebula gitea[3105419]: 2021/08/13 17:30:47 Completed POST /api/internal/hook/pre-receive/wanderer/gitea-git-notes-test 500 Internal Server Error in 50.831694ms
Aug 13 17:30:47 nebula gitea[3105419]: 2021/08/13 17:30:47 Started POST /api/internal/ssh/log for 127.0.0.1:0
Aug 13 17:30:47 nebula gitea[3105419]: 2021/08/13 17:30:47 Completed POST /api/internal/ssh/log 200 OK in 226.779µs
Aug 13 17:30:47 nebula gitea[3105419]: 2021/08/13 17:30:47 Started POST /api/internal/ssh/3/update/647 for 127.0.0.1:0
Aug 13 17:30:47 nebula gitea[3105419]: 2021/08/13 17:30:47 Completed POST /api/internal/ssh/3/update/647 200 OK in 5.372184ms
...
Aug 13 17:35:00 nebula gitea[3105419]: 2021/08/13 17:35:00 Started GET /api/internal/serv/command/3/wanderer/gitea-git-notes-test?mode=2&verb=git-receive-pack for 127.0.0.1:0
Aug 13 17:35:00 nebula gitea[3105419]: 2021/08/13 17:35:00 Completed GET /api/internal/serv/command/3/wanderer/gitea-git-notes-test?mode=2&verb=git-receive-pack 200 OK in 8.900563ms
Aug 13 17:35:01 nebula gitea[3105419]: 2021/08/13 17:35:01 Started POST /api/internal/hook/pre-receive/wanderer/gitea-git-notes-test for 127.0.0.1:0
Aug 13 17:35:01 nebula gitea[3105419]: 2021/08/13 17:35:01 ...ters/private/hook.go:470:HookPreReceive() [E] Unexpected ref: refs/notes/commits
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/routers/private/hook.go:470 (0x21ad554)
Aug 13 17:35:01 nebula gitea[3105419]:                 HookPreReceive: log.Error("Unexpected ref: %s", refFullName)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/modules/web/route.go:76 (0x203cfbb)
Aug 13 17:35:01 nebula gitea[3105419]:                 Wrap.func1: t(ctx)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:35:01 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/go-chi/chi/mux.go:436 (0x16f296a)
Aug 13 17:35:01 nebula gitea[3105419]:                 (*Mux).routeHTTP: h.ServeHTTP(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:35:01 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/routers/private/internal.go:31 (0x21c03a4)
Aug 13 17:35:01 nebula gitea[3105419]:                 CheckInternalToken.func1: next.ServeHTTP(w, req)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:35:01 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/modules/context/private.go:42 (0x171494f)
Aug 13 17:35:01 nebula gitea[3105419]:                 PrivateContexter.func1.1: next.ServeHTTP(ctx.Resp, ctx.Req)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:35:01 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/go-chi/chi/mux.go:70 (0x16f04ca)
Aug 13 17:35:01 nebula gitea[3105419]:                 (*Mux).ServeHTTP: mx.handler.ServeHTTP(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/go-chi/chi/mux.go:311 (0x16f68bb)
Aug 13 17:35:01 nebula gitea[3105419]:                 (*Mux).Mount.func1: handler.ServeHTTP(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:35:01 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/go-chi/chi/mux.go:436 (0x16f296a)
Aug 13 17:35:01 nebula gitea[3105419]:                 (*Mux).routeHTTP: h.ServeHTTP(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:35:01 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/routers/common/middleware.go:72 (0x20651bd)
Aug 13 17:35:01 nebula gitea[3105419]:                 Middlewares.func2.1: next.ServeHTTP(resp, req)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:35:01 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/routers/common/logger.go:23 (0x20643fb)
Aug 13 17:35:01 nebula gitea[3105419]:                 LoggerHandler.func1.1: next.ServeHTTP(w, req)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:35:01 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/go-chi/chi/middleware/strip.go:30 (0x2060fe7)
Aug 13 17:35:01 nebula gitea[3105419]:                 StripSlashes.func1: next.ServeHTTP(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:35:01 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/chi-middleware/proxy/middleware.go:37 (0x205c64e)
Aug 13 17:35:01 nebula gitea[3105419]:                 ForwardedHeaders.func1.1: h.ServeHTTP(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:35:01 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/routers/common/middleware.go:25 (0x2064cfc)
Aug 13 17:35:01 nebula gitea[3105419]:                 Middlewares.func1.1: next.ServeHTTP(context.NewResponse(resp), req)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:35:01 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/go-chi/chi/mux.go:87 (0x16f0250)
Aug 13 17:35:01 nebula gitea[3105419]:                 (*Mux).ServeHTTP: mx.handler.ServeHTTP(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/modules/web/route.go:318 (0x203c333)
Aug 13 17:35:01 nebula gitea[3105419]:                 (*Route).ServeHTTP: r.R.ServeHTTP(w, req)
Aug 13 17:35:01 nebula gitea[3105419]:         /opt/gitea/gitea-src/vendor/github.com/gorilla/context/context.go:141 (0x130d5d3)
Aug 13 17:35:01 nebula gitea[3105419]:                 ClearHandler.func1: h.ServeHTTP(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2049 (0x7b86e3)
Aug 13 17:35:01 nebula gitea[3105419]:                 HandlerFunc.ServeHTTP: f(w, r)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:2867 (0x7bbca2)
Aug 13 17:35:01 nebula gitea[3105419]:                 serverHandler.ServeHTTP: handler.ServeHTTP(rw, req)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/net/http/server.go:1932 (0x7b71cc)
Aug 13 17:35:01 nebula gitea[3105419]:                 (*conn).serve: serverHandler{c.server}.ServeHTTP(w, w.req)
Aug 13 17:35:01 nebula gitea[3105419]:         /usr/lib/go/src/runtime/asm_amd64.s:1371 (0x4747c0)
Aug 13 17:35:01 nebula gitea[3105419]:                 goexit: BYTE        $0x90        // NOP
Aug 13 17:35:01 nebula gitea[3105419]:
Aug 13 17:35:01 nebula gitea[3105419]: 2021/08/13 17:35:01 Completed POST /api/internal/hook/pre-receive/wanderer/gitea-git-notes-test 500 Internal Server Error in 121.080434ms
Aug 13 17:35:01 nebula gitea[3105419]: 2021/08/13 17:35:01 Started POST /api/internal/ssh/log for 127.0.0.1:0
Aug 13 17:35:01 nebula gitea[3105419]: 2021/08/13 17:35:01 Completed POST /api/internal/ssh/log 200 OK in 161.392µs
Aug 13 17:35:01 nebula gitea[3105419]: 2021/08/13 17:35:01 Started POST /api/internal/ssh/3/update/647 for 127.0.0.1:0
Aug 13 17:35:01 nebula gitea[3105419]: 2021/08/13 17:35:01 Completed POST /api/internal/ssh/3/update/647 200 OK in 26.496472ms

client-side, this is what I get

➜  git push -u origin refs/notes/"*":refs/notes/"*" --verbose
Pushing to thedomain:wanderer/gitea-git-notes-test.git
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
remote:
remote: Gitea: Internal Server Error
remote: Gitea: Internal Server Error
To thedomain:wanderer/gitea-git-notes-test.git
 ! [remote rejected] refs/notes/commits -> refs/notes/commits (pre-receive hook declined)
error: failed to push some refs to 'thedomain:wanderer/gitea-git-notes-test.git'

Originally posted by @wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf in #15373 (comment)

[a1012112796]

Hasn't found crash log by my test, wonder why.

[nitul1991]

I'd be happy to take a crack at identifying and resolving this issue if it's still up for grabs.