proxy.golang.org: some calls taking ~60s to 404

[myitcv]

Go version

go version go1.24.2 linux/arm64

Output of go env in your module/workspace:

AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/home/myitcv/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/home/myitcv/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2996555264=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='arm64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/dev/null'
GOMODCACHE='/home/myitcv/gostuff/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/myitcv/gostuff'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/myitcv/gos'
GOSUMDB='sum.golang.org'
GOTELEMETRY='on'
GOTELEMETRYDIR='/home/myitcv/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/home/myitcv/gos/pkg/tool/linux_arm64'
GOVCS=''
GOVERSION='go1.24.2'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

Ran:

go run -x cuelang.org/go/internal/ci/checks@v0.11.0-0.dev.0.20240903133435-46fb300df650

We do this as part of CI.

What did you see happen?

WORK=/tmp/go-build1295737349
# get https://proxy.golang.org/cuelang.org/@v/v0.11.0-0.dev.0.20240903133435-46fb300df650.info
# get https://proxy.golang.org/cuelang.org/go/internal/ci/checks/@v/v0.11.0-0.dev.0.20240903133435-46fb300df650.info
# get https://proxy.golang.org/cuelang.org/go/internal/@v/v0.11.0-0.dev.0.20240903133435-46fb300df650.info
# get https://proxy.golang.org/cuelang.org/go/internal/ci/@v/v0.11.0-0.dev.0.20240903133435-46fb300df650.info
# get https://proxy.golang.org/cuelang.org/@v/v0.11.0-0.dev.0.20240903133435-46fb300df650.info: 404 Not Found (1.491s)
# get https://proxy.golang.org/cuelang.org/go/internal/ci/checks/@v/v0.11.0-0.dev.0.20240903133435-46fb300df650.info: 404 Not Found (39.432s)
# get https://proxy.golang.org/cuelang.org/go/internal/ci/@v/v0.11.0-0.dev.0.20240903133435-46fb300df650.info: 404 Not Found (58.162s)
# get https://proxy.golang.org/cuelang.org/go/internal/@v/v0.11.0-0.dev.0.20240903133435-46fb300df650.info: 404 Not Found (58.169s)
# get https://proxy.golang.org/cuelang.org/go/@v/list
# get https://proxy.golang.org/cuelang.org/go/@v/list: 200 OK (0.065s)
/home/myitcv/.cache/go-build/35/3582ec5f9525b9f429159612520fad332d4ad1f61380658e1edd63b77041e45a-d/checks
2025/04/16 16:16:13 exit status 128
exit status 1

Some calls to the proxy taking nearly 60 seconds.

What did you expect to see?

A "fast" execution of this command. Even though we run this in CI, module and package caches are warm.

Are we missing something here with respect to what the proxy caches/does not cache?

Whilst the slow path can be ~60 seconds, another run afterwards is much faster. Which feels like the proxy caches the result "for some time" but then drop it later. And that the ~60 seconds is roughly the amount of time it takes to fully hydrate a cache from the remote VCS. But that's a pure guess.

(granted cuelang.org/go/internal/ci/checks is an "internal" main package, but my understanding is that this is not wrong per se)

[gabyhelp]

Related Issues

• proxy.golang.org: slow resolution of commit already pulled through the proxy #54977 (closed)
• cmd/go: get of module already present in proxy can be slow #50158
• cmd/go: repeated `run pkg@version` calls hit the proxy for no apparent reason #72944 (closed)
• proxy.golang.org: single intermittent failure of vanity URL causes stale cache #49916
• cmd/go: go get hits proxy to check retracted versions for transitive dependencies too much #42185 (closed)
• proxy.golang.org: "invalid version" reported for valid commit pulled through the proxy #53906
• cmd/go: test reported as cached, but not actually cached #41593
• proxy.golang.org: pulling @master through proxy does not update @latest for some time #37391 (closed)
• cmd/go,cmd/doc: http activity during 'go doc' #47196
• cmd/go: TestScript failures #65815 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[ansaba]

cc: @matloob and @jba
Need to check how cmd works in context to result being dropped.

[findleyr]

Certainly the 404 caching has a timeout, so I wonder why this problem isn't more pervasive. Something related to vanity URLs, perhaps?

Shouldn't finding the version for cuelang.org/go cancel the other requests?

[myitcv]

Shouldn't finding the version for cuelang.org/go cancel the other requests?

I don't believe so, because it's possible that a submodule could exist and shadow the package path (which results in an error), and cmd/go can't know that without checking.

[findleyr]

I don't believe so, because it's possible that a submodule could exist and shadow the package path (which results in an error), and cmd/go can't know that without checking.

Hmm, yes I suppose however the precedence is defined, we'll have to wait for all requests.
Then it appears this behavior is required. I don't know why this isn't a larger problem.

[myitcv]

I don't know why this isn't a larger problem.

I would share that sentiment. I suspect people do see problems, but it's never quite enough to tip over into "I should raise an issue" or "this is a consistent problem".

For example, I do the following sequence of steps multiple times per day:

$ cd $(mktemp -d)
$ go mod init mod.example
go: creating new go.mod: module mod.example
$ go get -x cuelang.org/go@latest
# get https://proxy.golang.org/cuelang.org/@v/list
# get https://proxy.golang.org/cuelang.org/go/@v/list
# get https://proxy.golang.org/cuelang.org/go/@v/list: 200 OK (0.159s)
# get https://proxy.golang.org/cuelang.org/@v/list: 404 Not Found (1.821s)
go: added cuelang.org/go v0.12.1

One or two times a day, I will see that 404 take up to 10 seconds to return.