Build tools issues discovered by binary breakage changes in Scala 3 stdlib

[WojciechMazur]

One of the recent changes introduce a binary breakage for previously experimental API of scala.caps - #22849

The following projects were affected and started to fail with:

Error:  ## Exception when compiling 29 sources to /build/repo/modules/domain/js/target/scala-3.7.1-RC1-bin-20250328-d519790-NIGHTLY/classes
Error:  dotty.tools.dotc.core.TypeError$$anon$1: package scala contains object and package with same name: caps
Error:  one of them needs to be removed from classpath

Project	Version	Build URL	Notes
akka/akka-management	1.5.0-M1	Open CB logs
akka/alpakka	6.0.2	Open CB logs
circe/circe-config	0.10.1	Open CB logs
circe/circe-optics	0.15.0	Open CB logs
daenyth/cats-effect-guava	0.2.0	Open CB logs
davenverse/epimetheus-community	0.5.1	Open CB logs
davenverse/tsec	0.5.0	Open CB logs
dwolla/fs2-aws	3.0.0-RC2	Open CB logs
gemini-hlsw/refined-algebra	0.1.1	Open CB logs
gvolpe/trading	HEAD	Open CB logs
http4s/http4s-grpc	0.0.4	Open CB logs
rustedbones/pekko-http-avro	1.0.0	Open CB logs
rustedbones/pekko-http-scalapb	1.0.0	Open CB logs
spilne/spilne	1.0.0	Open CB logs
tpataky/duckling	0.0.2	Open CB logs
typelevel/spotted-leopards	0.1.0	Open CB logs
valdemargr/catch-effect	0.1.2	Open CB logs

More projects might be affected, it does not contain projects that failed due to regressions in older versions of Scala 3 compiler

Inspection of the crash details shows that classpath contains multiple scala3-library_3 entries:

   // redacted for readability
library version: version 2.13.16
compiler version: version 3.7.1-RC1-bin-20250328-d519790-NIGHTLY-git-d519790
settings: 
-bootclasspath 
  /root/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.7.1-RC1-bin-20250328-d519790-NIGHTLY/scala3-library_3-3.7.1-RC1-bin-20250328-d519790-NIGHTLY.jar
  /root/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.16/scala-library-2.13.16.jar 

-classpath 
  /build/repo/modules/domain/js/target/scala-3.7.1-RC1-bin-20250328-d519790-NIGHTLY/classes
  /root/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_sjs1_3/3.7.1-RC1-bin-20250328-d519790-NIGHTLY/scala3-library_sjs1_3-3.7.1-RC1-bin-20250328-d519790-NIGHTLY.jar
  /root/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.3.3/scala3-library_3-3.3.3.jar
 ...

The build tool uses scala3-library_3 in version based on the highest version found in transitive dependencies based on the eviction mechanism. This later leads to conflicting entries found when compiling user code

Build tool / configuration issues

sbt users

Some of the sbt users are using libraryDependencies := Seq(???) which overrides the defaults set by the sbt. It erases the Scala 3 library artifact injected based on scalaVersion (probably). As the results the eviction mechanism picks a scala3_library classpath entry based on the eviction rules in the set of transitive dependencies.
As the workaround Scala 3 Open Community Build replaces usages of libraryDependencies := with libraryDependencies ++= before starting the build

mill users

Issue was observed in a single mill project https://github.com/pidoveproject/scala-showdown-api - build logs. The problem was not yet analysed.

Typelevel ecosystem - typelevel/sbt-typelevel plugin

Most of typelevel libraries are using the sbt auto-plugin that was flowed in the past leading to additional inject of scala3_library artifact using incorrect version, see typelevel/sbt-typelevel#654 for details.
This problems was fixed in the sbt-typelevel 0.7.7 versions.
Scala 3 Open Commmunity Build is now forcing upgrade to at least this version to workaround this problem

Solution proposal

1. Adjust the build tool rules to enforce usage of scala3-library which version always equal to Scala compiler version - additional checks for SIP-51 compatibility might be needed
2. Inside the scalac compiler reject (filter out) -classpath entries for scala3_library_3 and force the usage of artifact present in -bootclasspath list matching the scalac version

[tgodzik]

Looks like this might be a blocker for 3.7.1.

[WojciechMazur]

I've stumbled different, but similar issues in OpenCB for travisbrown/dhallj - builds logs

This project defined some projects using only java sources in main, but used Scala in tests. Becouse of that it used autoScalaLibrary := false sbt setting to exclude Scala standard library.
When compiling any source, eg.

class CborSuite {
  def roundTripDouble(value: Double): Option[Double] = {
    ???
  }
}

we get following error

[error] -- Error: /Users/wmazur/projects/scala/community-build3/repo/modules/core/src/test/java/org/dhallj/cbor/CborSuite.scala:10:38 
[error] 10 |  def roundTripDouble(value: Double): Option[Double] = {
[error]    |                                      ^
[error]    |               package scala.runtime does not have a member type $throws

This error is based on Scala 3.3 LTS
$throws was introduced in Scala 3.1.0 as experimental.
All of project transitive dependencies were published using Scala 3.0.0 and that's the version that was used to compile the tests.

To some extent it shows that this issue with conflict between Scala compiler and available standard library exists in all of Scala 3 versions.

Since compiler assumes that some symbols needs to be present on classpath (even if they're not directly used) the standard library with matching version should be always present on classpath. Such a rule would be probably easiest to implement on the build-tool side, however, if we have any chance of of establishing it in the Scala compiler we should try such a approach.

[sjrd]

The rule is supposed to be that the version of the library on the classpath must be >= to the version of the compiler. But we cannot demand ==.