Follow-up work from integrating java8-compat

[lrytz]

scala/scala#7458 integrates converters from java8-compat into the standard libarary.

Done in the PR

• Check if function wrappers should be serializable (part of https://github.com/scala/scala/pull/7458/commits)
• Ensure converting a function back and forth gives the original function, instead of wrapping twice (part of https://github.com/scala/scala/pull/7458/commits)
• Improve Scaladocs in many places, especially Steppers, Accumulators. Give an overview somewhere how to efficiently use Java 8 streams, what are recommended patterns, how to avoid boxing. (Scaladocs for new jdk converters, move Accumulators to jdk package scala#7939)

A few tasks have been postponed past that PR

• fix some bugs in steppers (@Ichoran) (part of https://github.com/scala/scala/pull/7970/commits)
• Integrate missing tests from java8-compat (for StreamConverters, Accumulators, Steppers, maybe more) (Small fixes and more tests for jdk Converters scala#7970)
• make java8-compat a shim release for 2.13 that helps cross-building a codebase with 2.12 / 2.13 (Adapt to 2.13.0-RC1 scala-java8-compat#139)
• Integrate benchmarks from java8-compat. Compare performance with java8-compat for 2.12.
• Integrate tests that ensure no boxing occurs (https://gist.github.com/lrytz/900dcc6aba5479ae615098bb0d74b1ee). This is useful more generally, provide some infrastructure.
• Check if we should bring back functionality that was dropped from java8-compat, most likely as extension methods:
  • higher-order operations on steppers (that avoid boxing)
  • methods to(Factory) and toArray on steppers

[SethTisue]

and push it at least partway through the 2.13 community build, I guess the shim release thing will allow that?

[lrytz]

Hopefully it wouldn't be too hard to branch the projects and remove the java8-compat dependency?

[SethTisue]

if that's all it takes, then we don't even need to branch, we should be able to "removeDependency org.scala-lang.modules scala-java8-compat" in the project's entry in community.dbuild

[lrytz]

Code changes will be required too. To minimize the necessary code changes, we can probably do a shim release of java8-compat for 2.13 that provides the same imports/methods/types as the 2.12 version, but uses the 2.13 std lib implementation. But it won't be 100% compatible.

[joshlemer]

@lrytz not sure if this is right place to ask, considering these facts:

• the implicits of scala.collection.JavaConverters._ are moved to scala.jdk.CollectionConverters.Ops._
• for all converters other than collections (Optionals, Functions, Durations, etc), the actual implementations live in the scala.jdk package, not some other package. The collections are different in that the scala.jdk package just points to scala.collections.convert implementations
• only jdk converters exist in scala.collection.convert (no JS or Scala-Native stuff lives there)

Maybe scala.collection.convert should now reside somewhere within scala.jdk?

[lrytz]

Maybe scala.collection.convert should now reside somewhere within scala.jdk?

I'm not opposed, I don't know how much it matters.

It would work for the As[Java|Scala][Converters|Extensions] if we also moved the JavaCollectionWrappers. They could be private[jdk] instead of private[collection].

It would also work for StreamExtensions I think.

It would not work as-is for stepper implementations, as they access internals of collections that are marked private[collection], so some implementation details in the collections would have to be private[scala].

The convert package also holds the public Accumulator collection type. We discussed if that class should live somewhere else, in mutable for example, or in scala.jdk, but decided to stick with collection.convert. It's a collection, intended for streams interop, but not as generally useful to be in mutable. Mabye scala.jdk would be better, not sure..

[joshlemer]

Fair enough!

[szeiger]

We also need to consider cross-building. As long as there are deprecated forwarders for all the old functionality from scala-java8-compat you could still build with the 2.12 packages. Releasing an empty 2.13 version of scala-java8-compat would make this easier (like we do for collections-compat) but this can also be solved with conditional dependencies in the build.

But if we can't keep all the old stuff around (deprecated) for 2.13 we need to build something like collections-compat that allows building on 2.11/2.12 with the 2.13 package names.

[lrytz]

We won't get to 100% compatibility for cross-building, quite a lot of things have changed during the integration. But the basic use cases can be made to work. I guess there are two options: release java8-compat for 2.13 with the old API, or release java8-compat-compat for 2.12 with the new API.

[lrytz]

@joshlemer I moved the Accumulator classes to jdk in scala/scala#7939, thanks for starting the thought process :-) I think it makes sense because that collection is really inteded to be used with streams interop, it's a bit special with its hand-specialization for Int/Long/Double. So at this point, collection.convert only contains internal stuff, and I agree we could retire it eventually. But there's nothing in there anymore that a user would ever reference directly.

[joshlemer]

@lrytz <bikeshed>

Hey Lukas, sorry for the bikeshedding again but what do you think about having the import just be scala.jdk.CollectionConverters._ rather than scala.jdk.CollectionConverters.Ops._? For example, when implementing scala.util.chaining ops it was decided to change the import from scala.util.chaining.Ops._ to scala.util.chaining._. It's on one hand unimportant but on the other hand can't be auto-imported by IDE's/editors and must be a memorized path for users.

</bikeshed>

[lrytz]

What I'd like to avoid is mixing the explicit conversion methods (e.g., scala.jdk.CollectionConverters.asJava) and the implicit methods that provide the extension methods (e.g., scala.jdk.CollectionConverters.Ops.SeqHasAsJava) into the same object. Otherwise, importing CollectionConverters._ to get extension methods would bring all the explicit ones in scope, without prefix.

What we could do a swap: put the implicit methods in CollectionConverters, and have some other object that holds the explicit conversion methods.