Speed up make

[silverwind]

• stop find from venturing into node_modules and other directories that do not contain *.go files
• only run find once for *.go files
• move image tempdir generation to that task
• rename GOFILES to GO_SOURCES_OWN for consistency and to indicate that vendor files are not included
• pre-resolve FOMANTIC_SOURCES

On my machine, the first action alone speeds up make initialization by around 400%.

[codecov-io]

Codecov Report

Merging #10560 into master will increase coverage by 0.07%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #10560      +/-   ##
==========================================
+ Coverage   43.75%   43.82%   +0.07%     
==========================================
  Files         586      584       -2     
  Lines       81764    81649     -115     
==========================================
+ Hits        35775    35786      +11     
+ Misses      41549    41426     -123     
+ Partials     4440     4437       -3

Impacted Files	Coverage Δ
models/user.go	49.79% <ø> (+0.49%)	⬆️
services/pull/patch.go	61.93% <ø> (+1.55%)	⬆️
routers/private/internal.go	82.6% <ø> (+51.96%)	⬆️
models/issue_assignees.go	70.24% <ø> (+6.33%)	⬆️
models/update.go	8.95% <ø> (+1.16%)	⬆️
modules/base/tool.go	65.02% <ø> (+2.07%)	⬆️
routers/api/v1/utils/utils.go	76% <ø> (+12.66%)	⬆️
modules/templates/helper.go	42.19% <ø> (+1.46%)	⬆️
models/unit.go	37.03% <0%> (-2.47%)	⬇️
modules/indexer/stats/db.go	50% <0%> (ø)	⬆️
... and 4 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 97c5ed4...ef56ec9. Read the comment docs.

[silverwind]

I wonder if templates/*.tmpl should be included in the EXECUTABLE dependencies. It seems like an oversight that they are absent given that they influence the binary.

[lafriks]

Compliance check fails

[guillep2k]

I wonder if templates/*.tmpl should be included in the EXECUTABLE dependencies. It seems like an oversight that they are absent given that they influence the binary.

Anything that builds into bindata.go should be a dependency, but we're forcing the generate step, so...

[silverwind]

So templates go into bindata? I thought they were built as part of the go compilation.

[lafriks]

Templates are included as files in bindata.go and compiled at runtime not at compilation step

[silverwind]

Ok. Once will keep that in mind for once we make the generate step properly track its prereqs. Not an issue currently as it runs unconditionally.

[jolheiser]

This change seems to output errors on Windows?

I get this a handful of times.

find: paths must precede expression: tools.go
Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]

Is it related to a specific version of find perhaps?
My find --version is 4.6.0

[silverwind]

Hmm, check your man find if it supports -prune. I did verify it's present on Linux and BSD, but Windows versions may differ after all.

You can also manually run the find:

find . -type d \( -path ./node_modules -o -path ./docs -o -path ./public -o -path ./options -o -path ./contrib -o -path ./data \) -prune -o -type f -name '*.go' -print

It should output a list of all go files.

[silverwind]

It does at least work on 4.6.0, Cygwin version:

$ find --version
find (GNU findutils) 4.6.0
Packaged by Cygwin (4.6.0-1)
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Eric B. Decker, James Youngman, and Kevin Dalley.
Features enabled: D_TYPE O_NOFOLLOW(enabled) LEAF_OPTIMISATION FTS(FTS_CWDFD) CBO(level=2)

[silverwind]

Also try changing '*.go' to "*.go". paths must precede expression seems related to shell expansion, but I can't see any happening here.

[jolheiser]

Seems it's related to escaping. The following works on Windows, but not on Linux.

GO_SOURCES ?= $(shell find . -type d \\( -path ./node_modules -o -path ./docs -o -path ./public -o -path ./options -o -path ./contrib -o -path ./data \\) -prune -o -type f -name '*.go' -print)

I think we can get a solution propped up shortly. 😃

[silverwind]

Backslash should need no escaping. Try this:

GO_SOURCES ?= $(shell find . -type d \( -path ./node_modules -o -path ./docs -o -path ./public -o -path ./options -o -path ./contrib -o -path ./data \) -prune -o -type f -name \*.go -print)

[jolheiser]

Nope, still angry. #10577 works, though.

Regardless, this PR is 🔥. Building is so much faster now.