Bazel build hangs on with 100% CPU

[Helcaraxan]

Summary

I've been trying out golangci-lint for a while in several small scale projects as well as in the setting of VSCode in a larger code-base. I am now looking to potentially start using golangci-lint as the principal CI linter in that same larger code-base (currently gometalinter and some other ones).

While trying out larger scale runs, with just a plain golangci-lint run and not with specific packages like VSCode provides, I have been running into a persistent issue that sees the run reaching the specified deadline (30s or the default 1m for example). However at that point the binary does not correctly exit and leaves one of its threads running at 100% CPU indefinitely. I have actually left it running for a whole night to check if it wasn't some clean-up or anything else taking extra long due to the code-base's size. Running only a single simple linter like lll that actually finished within the deadline does exit correctly.

Even stranger, I am only seeing this occur on my Linux but not on my Mac where I don't even reach the deadline by an order of magnitude. The full data (configuration, etc) can be found below.

Last but not least, in case that might make a difference (given the recent move to use golang.org/x/tools/go/packages), we are using Bazel for our build systems.

Data

Version of golangci-lint: golangci-lint --version

Same on both Linux and OSX

 -> golangci-lint --version
golangci-lint has version v1.12.3 built from 014a924 on Wed  5 Dec 09:50:04 GMT 2018

(Date differs from official release as I built it on my local machine from the v1.12.3 tag).

Config file: cat .golangci.yml

NB: Have removed all default settings that I tend to keep around for awareness in the file. This is the list of non-standard setting.

run:
  deadline: 5m

issues:
  max-per-linter: 0
  max-same-issues: 0
  new-from-rev: "HEAD~1"

linters-settings:
  goimports:
    local-prefixes: github.com

  gocyclo:
    min-complexity: 15

  misspell:
    locale: US

  lll:
    line-length: 150

  nakedret:
    max-func-lines: 0 # Warn on all naked returns.

  gocritic:
    enabled-checks:
      - appendAssign
      - assignOp
      - boolExprSimplify
      - builtinShadow
      - captLocal
      - caseOrder
      - commentedOutImport
      - defaultCaseOrder
      - dupArg
      - dupBranchBody
      - dupCase
      - dupSubExpr
      - elseif
      - emptyFallthrough
      - hugeParam
      - ifElseChain
      - importShadow
      - indexAlloc
      - methodExprCall
      - nestingReduce
      - offBy1
      - ptrToRefParam
      - regexpMust
      - singleCaseSwitch
      - sloppyLen
      - sloppyReassign
      - switchTrue
      - typeSwitchVar
      - typeUnparen
      - underef
      - unlambda
      - unnecessaryBlock
      - unslice
      - valSwap
      - wrapperFunc
      - yodaStyleExpr

Go environment: go version && go env

NOTE: In the setting of our Bazel build system we actually use a pinned Go version at 1.10.

OSX (Mojave)

 -> go version && go env
go version go1.11.1 darwin/amd64
GOARCH="amd64"
GOBIN="[REDACTED]"
GOCACHE="/Users/[REDACTED]/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="[REDACTED]"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/1y/y8rl2vbs3jzc3y9lsfms_5qm0000gn/T/go-build985692898=/tmp/go-build -gno-record-gcc-switches -fno-common"

Linux (Ubuntu 18.04 LTS)

go version go1.10 linux/amd64
GOARCH="amd64"
GOBIN="[REDACTED]"
GOCACHE="/home/[REDACTED]/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="[REDACTED]"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build168009782=/tmp/go-build -gno-record-gcc-switches"

Verbose output of running: golangci-lint run -v

OSX (Mojave)

 -> golangci-lint run ./src/[REDACTED]/... --verbose -E errcheck
INFO [config_reader] Config search paths: [[REDACTED]]
INFO [config_reader] Used config file ../.golangci.yaml
INFO Gocritic enabled checks: [appendAssign assignOp boolExprSimplify builtinShadow captLocal caseOrder commentedOutImport defaultCaseOrder dupArg dupBranchBody dupCase dupSubExpr elseif emptyFallthrough hugeParam ifElseChain importShadow indexAlloc methodExprCall nestingReduce offBy1 ptrToRefParam regexpMust singleCaseSwitch sloppyLen sloppyReassign switchTrue typeSwitchVar typeUnparen underef unlambda unnecessaryBlock unslice valSwap wrapperFunc yodaStyleExpr]
INFO [lintersdb] Active 1 linters: [errcheck]
INFO [loader] Go packages loading at mode load types and syntax took 14.834714393s
INFO [loader/astcache] Parsed AST of all pkg.GoFiles: [] for 669ns
INFO [loader/astcache] Parsed AST of all pkg.GoFiles: [] for 569ns
INFO [loader/astcache] Parsed AST of all pkg.GoFiles: [] for 349ns
INFO [loader/astcache] Parsed AST of all pkg.GoFiles: [] for 409ns
INFO [runner] worker.2 took 19.053µs
INFO [runner] worker.3 took 20.233µs
INFO [runner] worker.4 took 10.645µs
INFO [runner] worker.1 took 164.658536ms with stages: errcheck: 164.634234ms
INFO [runner] Workers idle times: #2: 153.091455ms, #3: 152.954657ms, #4: 152.899849ms
INFO [runner/max_from_linter] 11/61 issues from linter errcheck were hidden, use --max-issues-per-linter
INFO [runner] processing took 3.69527709s with stages: diff: 3.532721767s, exclude: 47.860214ms, path_prettifier: 46.737114ms, skip_dirs: 29.272698ms, autogenerated_exclude: 16.633593ms, cgo: 16.017746ms, nolint: 4.024796ms, source_code: 1.691471ms, uniq_by_line: 219.919µs, max_from_linter: 54.632µs, path_shortener: 20.769µs, max_per_file_from_linter: 20.665µs, max_same_issues: 1.343µs, skip_files: 363ns

[... removed linting errors ...]

INFO Memory: 151 samples, avg is 526.9MB, max is 1548.3MB
INFO Execution took 18.927478326s

Linux (Ubuntu 18.04 LTS)

NOTE: interrupted by Ctrl+C after ~6 minutes - htop is showing 1 parent and 8 child processes with 1 of them at 100% CPU)

 -> golangci-lint run -verbose -E errcheck src/[REDACTED]/...
INFO [config_reader] Config search paths: [[REDACTED]] 
INFO [config_reader] Used config file ../.golangci.yaml 
INFO Gocritic enabled checks: [appendAssign assignOp boolExprSimplify builtinShadow captLocal caseOrder commentedOutImport defaultCaseOrder dupArg dupBranchBody dupCase dupSubExpr elseif emptyFallthrough hugeParam ifElseChain importShadow indexAlloc methodExprCall nestingReduce offBy1 ptrToRefParam regexpMust singleCaseSwitch sloppyLen sloppyReassign switchTrue typeSwitchVar typeUnparen underef unlambda unnecessaryBlock unslice valSwap wrapperFunc yodaStyleExpr] 
INFO [lintersdb] Active 1 linters: [errcheck]     
INFO Memory: 3000 samples, avg is 132.9MB, max is 133.9MB 
INFO Execution took 5m0.000003409s                
^C

[jirfag]

hi!
It looks like we can't work with the Bazel build system yet :(
I couldn't find needed driver (instead of go list) for Bazel and found only issues bazel-contrib/rules_go#1644 and bazel-contrib/rules_go#512

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tpounds]

@Helcaraxan Is this still an issue with the most recent version or unreleased master branch? @jirfag Has done quite a bit of work to reduce memory usage (see: #337, #758, #764) that should help quite a bit here.

[Helcaraxan]

@tpounds I have not been able to reproduce this of late and we have been running golangci-lint as our main CI linter for the past few months. Had simply forgotten to close this PR. 🙂