Skip to content

Dart CI package tests are run with the noisy compact reporter for package:test #54929

Open
@whesse

Description

@whesse

The Dart CI and CQ run tests for the packages in the Dart SDK source on builders like

https://ci.chromium.org/ui/p/dart/builders/luci.dart.ci.sandbox/pkg-linux-release

These builders run package tests written using package:test framework the same way they run any
Dart test on the runtime VM, by running the program using the Dart VM, with no special arguments because it uses the package:test framework.

The default reporter for package:test is --reporter compact which prints ANSI console control codes in stdout to try
and make a console keep printing progress on the same line without scrolling. In our test logs, they appear as strings like �[32m+10�[0m�[31m -1�[0m on every line.

It would be better to use the expanded, github, or json reporters, to produce better logs on stdout for users to see using our CI infrastructure. This could be done by passing the --reporter flag in the commands created by package:test_runner, or by setting the option in a dart_test.yaml file at the root of each package.

The json report can also be sent to a file, in addition to another report format printed on stdout.
There has been discussion of a failures-only reporting format as well on dart-lang/test#829

If we reported a structured format, then our test results systems could even read that format and track individual tests in a test file separately in our CI databases, but that would be an extensive feature addition.

So we should decide on the format to report on stdout, and change package:test_runner or else add dart_test.yaml files to our individual packages, to add the correct --reporter option when running these tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    analyzer-testarea-infrastructureUse area-infrastructure for SDK infrastructure issues, like continuous integration bot changes.area-testCross-cutting test issues (use area- labels for specific failures; not used for package:test).

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions