x/all: no obvious way to download source code

[jnjackins]

What version of Go are you using (go version)?

$ go version
go version go1.21.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/Users/jnj/Library/Caches/go-build'
GOENV='/Users/jnj/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/jnj/pkg/mod'
GOOS='darwin'
GOPATH='/Users/jnj'
GOPRIVATE='github.com/segmentio'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/Users/jnj/src/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/Users/jnj/src/go/pkg/tool/darwin_amd64'
GOVCS=''
GOVERSION='go1.21.1'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/Users/jnj/src/go/src/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/Users/jnj/tmp/go-build1463137461=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

I tried to download the source for the golang.org/x/image module.

Following the instructions in the README:

Download/Install

The easiest way to install is to run go get -u golang.org/x/image/.... You can also manually git clone the repository to $GOPATH/src/golang.org/x/image.

$ go get -u golang.org/x/image/...
go: go.mod file not found in current directory or any parent directory.
	'go get' is no longer supported outside a module.
	To build and install a command, use 'go install' with a version,
	like 'go install example.com/cmd@latest'
	For more information, see https://golang.org/doc/go-get-install-deprecation
	or run 'go help get' or 'go help install'.

Neither the README, nor the git hosting website that golang.org/x/image redirects to provide a cloneable URL. If one tries to guess using the website URL:

$ git clone https://cs.opensource.google/go/x/image
Cloning into 'image'...
fatal: https://cs.opensource.google/go/x/image/info/refs not valid: is this a git repository?

Note that while I use x/image as an example, the majority of these modules provide non-working instructions for retrieving the source code.

This lack of a canonical way to download source code has apparently been an issue since module-aware go get was introduced, with tracking issues coming and going with no new solution provided other than "use the typical VCS clone command". Which is OK (though a worse experience), but seemingly the Go ecosystem never adapted by providing alternate instructions. This is not typically an issue on git hosts like github, where the clone URL is prominent, but I searched for a few minutes and found no mention of a repository clone URL for any of the golang.org/x modules.

What did you expect to see?

Ideally, a useable go command which downloads the source code, as we were all accustomed to in the pre-module days. Otherwise, simple (working) instructions for cloning these repositories, or at minimum a visible URL which can be used with git clone.

What did you see instead?

The errors shown above.

[gophun]

In my opinion these outdated instructions should just be removed.

How to add a module as a dependency to a project is something a Go developer learns once at the beginning of their journey and it doesn't need to be repeated in every README.

How to get the source code of a repository depends on where it's hosted. It shouldn't be in the README either, because the same repository can be hosted on various hosting platforms. It's the responsibility of the hosting platform to provide the user with a cloneable path or a download link.

[jnjackins]

To be clear I'm not trying to discuss dependencies or adding of them at all. I'm talking about downloading source code, and as you said the hosting platform does not seem to provide the necessary info. The READMEs attempt to provide instructions but these instructions have been broken by the tooling changes.

[gophun]

To be clear I'm not trying to discuss dependencies or adding of them at all.

But that was the purpose of the instructions in the READMEs. It was not to tell how to get the source code, but how to make it usable as a dependency. The fact that "go get" used to download the source code into GOPATH was a side effect. In the module world you still use "go get" to make it usable as a dependency, but you need to be in a folder with a "go.mod" file.

[gophun]

and as you said the hosting platform does not seem to provide the necessary info

In this case the right thing is to fix the hosting platform, not to copy some instructions into every README, which will be wrong as soon as the repository is hosted somewhere else.

[bcmills]

go mod download -json golang.org/x/image/...?

[seankhliao]

maybe we need to add the same notice as the main repo?

Our canonical Git repository is located at https://go.googlesource.com/go. There is a mirror of the repository at https://github.com/golang/go.

[aslatter]

As a usability case-study, if I want to do a VCS checkout of, say x/crypto, my process is:

1. Do a Google search for "golang x crypto" and arrive at https://pkg.go.dev/golang.org/x/crypto
2. Follow the source-code link to https://cs.opensource.google/go/x/crypto
3. Fail to find any link that would let me clone the repo (it's clear it's a Git repo, though)
4. Assume there is probably a mirror on GitHub and perform another search for "golang x crypto github"
5. Land on the GitHub mirror and follow the instructions for performing a "git clone": https://github.com/golang/crypto

[gl-yziquel]

Same issue.

This is extremely frustrating

[kaber2]

Indeed. Just wanted to update my local copy of x/crypto and ran into the same issue, then remembered i had it before a few years ago.

This is ridiculous. Its apparently an open source software hosting site, yet it provides no information how to actually clone the repositories. Obvious methods don't work.

Well thanks to @aslatter for the github link.

[gopherbot]

Change https://go.dev/cl/624115 mentions this issue: README: don't recommend go get

[gopherbot]

Change https://go.dev/cl/624155 mentions this issue: README: don't recommend go get

[gopherbot]

Change https://go.dev/cl/624175 mentions this issue: README: don't recommend go get

[gopherbot]

Change https://go.dev/cl/624195 mentions this issue: README: don't recommend go get