runtime: failure to detect write barriers inside nowritebarrierrec / questionable write barriers

[dr2chase]

Go version

near dev tip as of mid-April 2025, e.g. 13b7c7d

Output of go env in your module/workspace:

AR='ar'
CC='clang'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='clang++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/Users/drchase/Library/Caches/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/Users/drchase/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/mz/1gp2q1rx7cj2_g_j8sml9kc40095tn/T/go-build343059180=/tmp/go-build -gno-record-gcc-switches -fno-common'
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMOD='/Users/drchase/work/go-quick/src/go.mod'
GOMODCACHE='/Users/drchase/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/drchase/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/Users/drchase/work/go-quick'
GOSUMDB='sum.golang.org'
GOTELEMETRY='on'
GOTELEMETRYDIR='/Users/drchase/Library/Application Support/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/Users/drchase/work/go-quick/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='devel go1.25-13b7c7d8d2 Sun Apr 13 20:20:58 2025 -0700'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

For other reasons, I modified the inlining budget from 80 to 800, then tried to rebuild with ./make.bash

What did you see happen?

A slow, bloated build.

What did you expect to see?

(annotated to add more information so as to avoid repeated work)

gp.syncGroup = nil
/Users/drchase/work/go-quick/src/runtime/mgc.go:646:16: write barrier prohibited by caller; gcStart
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1427:11: called by mallocgcSmallScanNoHeader
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1059:44: called by mallocgc
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1716:17: called by newobject
	/Users/drchase/work/go-quick/src/runtime/mprof.go:322:10: called by stkbucket
	/Users/drchase/work/go-quick/src/runtime/mgcmark.go:816:8: called by gcFlushBgCredit


gp.syncGroup = nil
/Users/drchase/work/go-quick/src/runtime/mgcmark.go:436:16: write barrier prohibited by caller; gcAssistAlloc
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1041:21: called by mallocgc
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1716:17: called by newobject
	/Users/drchase/work/go-quick/src/runtime/mprof.go:322:10: called by stkbucket
	/Users/drchase/work/go-quick/src/runtime/mgcmark.go:816:8: called by gcFlushBgCredit


span.largeType = nil // Tell the GC not to look at this yet.
/Users/drchase/work/go-quick/src/runtime/malloc.go:1545:17: write barrier prohibited by caller; mallocgcLarge
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1065:30: called by mallocgc
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1716:17: called by newobject
	/Users/drchase/work/go-quick/src/runtime/mprof.go:322:10: called by stkbucket
	/Users/drchase/work/go-quick/src/runtime/mgcmark.go:816:8: called by gcFlushBgCredit


!!! finc.alllink = allfin (in queuefinalizer)
/Users/drchase/work/go-quick/src/runtime/mheap.go:2463:17: write barrier prohibited by caller; freeSpecial
	/Users/drchase/work/go-quick/src/runtime/mgcsweep.go:599:16: called by (*sweepLocked).sweep
	/Users/drchase/work/go-quick/src/runtime/mcache.go:160:43: called by (*mcache).refill
	/Users/drchase/work/go-quick/src/runtime/malloc.go:961:11: called by (*mcache).nextFree
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1368:39: called by mallocgcSmallScanNoHeader
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1059:44: called by mallocgc
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1716:17: called by newobject
	/Users/drchase/work/go-quick/src/runtime/mprof.go:322:10: called by stkbucket
	/Users/drchase/work/go-quick/src/runtime/mgcmark.go:816:8: called by gcFlushBgCredit


!!! *header = typ (in heapSetTypeSmallHeader)
/Users/drchase/work/go-quick/src/runtime/malloc.go:1474:39: write barrier prohibited by caller; mallocgcSmallScanHeader
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1061:42: called by mallocgc
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1716:17: called by newobject
	/Users/drchase/work/go-quick/src/runtime/mprof.go:322:10: called by stkbucket
	/Users/drchase/work/go-quick/src/runtime/mgcmark.go:816:8: called by gcFlushBgCredit

I see several things going on here.

First, it indeed looks like operations that are superficially write-barrier triggering. They may not all be "real" write barriers, but they are definitely stores of pointer values into data structures.

Second, they were not detected until the inlining was turned very far up. There might be other write barriers that aren't getting detected.

Third, I think we need to fix the stack traces in the write-barrier-rec report, because inlining is missing I think, some of those I had to figure out carefully.

[dr2chase]

@golang/compiler

I initially thought this was not a problem because it all hangs on an unlock at the end of gcFlushBgCredit, but then realized that the caller depends on this. And also, not detected.

[gabyhelp]

Related Issues

• runtime: gcstoptheworld=2 doesn't work #53885 (closed)
• After updating to latest on master branch, strange one time compilation failure. #18592 (closed)
• runtime: write of Go pointer 0xc0000b2330 to non-Go memory 0x7f0f9772f080 #28458 (closed)
• cmd/compile: cannot build runtime in coverage mode on Wasm #70510
• unique: panic for Make(struct{}{}) #69458 (closed)
• runtime: cgoCheckPointer fatal error: can't happen #70016 (closed)
• iOS fatal error: bulkBarrierPreWrite: unaligned arguments #63255 (closed)

Related Code Changes

• runtime: noinline annotations necessary for high inlining budget
• cmd/internal/gc: inline writeBarrierEnabled check before calling writebarrierptr
• cmd/internal/gc: emit write barriers at lower level

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[prattmic]

cc @golang/runtime

[mknyszek]

One thing that's a little confusing is why it looks like stkbucket is being called by gcFlushBgCredit. I don't see calls there, and the CPU profiling signal handler doesn't allocate...

(Referring to:)

span.largeType = nil // Tell the GC not to look at this yet.
/Users/drchase/work/go-quick/src/runtime/malloc.go:1545:17: write barrier prohibited by caller; mallocgcLarge
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1065:30: called by mallocgc
	/Users/drchase/work/go-quick/src/runtime/malloc.go:1716:17: called by newobject
	/Users/drchase/work/go-quick/src/runtime/mprof.go:322:10: called by stkbucket
	/Users/drchase/work/go-quick/src/runtime/mgcmark.go:816:8: called by gcFlushBgCredit

[prattmic]

unlock -> unlockWithRank -> unlock2 -> (*mLockProfile).recordUnlock -> (*mLockProfile).store -> saveBlockEventStack -> stkbucket

[prattmic]

cc @rhysh

[prattmic]

cc @rhysh