Unintentional but unacceptable diagnostic language

[dabrahams]

Description

Not sure this bug really belongs on SPM. I just built swift-format and the build ended with:

remark: Incremental compilation has been disabled: it is not compatible with whoremark: Incremental compilation has been disabled: it is not compatible with whoremark: Incremental compilation has been disabled: it is not compatible with whoremark: Incremental compilation has been disabled: it is not compatible with whoremark: Incremental compilation has been disabled: it is not compatible with whoremark: Incremental compilation has been disabled: it is not compatible with whoremark: Incremental compilation has been disabled: it is not compatible with who[11/16] Co[25/25] Compiling SwiftFormatTestSupport DiagnosingTestCase.swift
Build complete! (103.43s)

Something needs to be inserting a newline in the output before all remarks.

Expected behavior

No response

Actual behavior

No response

Steps to reproduce

Build swift-format from the CLI with swift 5.7

Swift Package Manager version/commit hash

No response

Swift & OS version (output of swift --version && uname -a)

No response

[abertelrud]

Almost certainly the remark refers to "whole module optimization". Seems to be two bugs:

1. the message cut off
2. it needs a newline (as you point out)

Taking a look.

[abertelrud]

I do see the missing newline, but not the cutting off, so I get remark: Incremental compilation has been disabled: it is not compatible with whole module optimizationremark: Incremental compilation has been disabled: it is not compatible with whole module optimizationremark: Incremental compilation has been disabled: it is not compatible with whole module optimizationremark: Incremental compilation has been disabled: it is not compatible with whole module optimizationremark: Incremental compilation has been disabled: it is not compatible with whole module optimization[4/14] Compiling TSCLibc libc.swift

I only get this if I set TERM to something that doesn't support ANSI (or pipe the output), otherwise I don't.

Still not good, so checking what's going on.

[abertelrud]

Also seems unfortunate to be getting that same message so many times; either SwiftPM should dedupe or maybe that's something in the driver. In any case the missing newline seems to be the first thing to address here, and that is almost certainly on the SwiftPM side.

[abertelrud]

❯ So this looks like a combination of two things:

1. swift-driver emits both its JSON output and raw remarks to stderr (this seems unfortunate)
2. swift-package tries to interpret the JSON packets from the driver but when it can't it just emits the raw bytes, and ends up losing the newline (still looking into that one)

For example:

swiftc -parseable-output -whole-module-optimization -incremental -c foo.swift
remark: Incremental compilation has been disabled: it is not compatible with whole module optimization
1201
{
  "command_arguments" : [
    "-frontend",
    "-c",
    "foo.swift",
    "-target",
    "arm64-apple-macosx12.0",
    "-Xllvm",
    "-aarch64-use-tbi",
    "-enable-objc-interop",
    "-stack-check",
. . .

[abertelrud]

Filed swiftlang/swift-driver#1241 on the Swift Driver issue and will see what we do in SwiftPM about the current state of affairs. Since the output is always line-based, we should be able to do better in the JSON parser than we are currently doing.

[compnerd]

This is fairly reminiscent of the llbuild issue that we had on Windows - we would not buffer things correctly and interleave messages: #4192

[abertelrud]

That's a good point — thanks for pointing this out. In this case it seems that this is actually handled fairly well already by the JSON parser in SwiftPM, and an additional newline is all that's needed. That doesn't explain the truncation but does at least properly emit the diagnostics. I still think the driver should emit JSON for its own diagnostics, though, when -parseable-output is in effect.

[abertelrud]

I'm preparing a SwiftPM PR for the newline at least.

[abertelrud]

rdar://103608636