Output of failing run tests is hard to find

[retronym]

--verbose ought to print the transcripts, as it does in the commmand line runner. We shouldn't have to look at the individual test case log file manually.

> partest --show-diff --verbose --srcpath scaladoc --grep SI-191
[info] Packaging /Users/jason/code/scala2/build-sbt/pack/lib/scala-partest-javaagent.jar ...
[info] Done packaging.
Partest version:
Compiler under test: $baseDir/compiler
Scala version is:    Scala compiler version 2.11.8-20160206-230105-fcaa3ab -- Copyright 2002-2016, LAMP/EPFL
Scalac options are:
Compilation Path:    /Users/jason/code/scala2/target/test/it-classes:$baseDir/test:$baseDir/compiler:$baseDir/library:/Users/jason/code/scala2/build-sbt/libs/classes/forkjoin:$baseDir/reflect:$baseDir/interactive:$baseDir/actors:$baseDir/repl-jline-embedded:$baseDir/repl-jline:$baseDir/repl:$baseDir/scalap:$baseDir/partest-extras:/Users/jason/code/scala2/build-sbt/pack/lib/scala-partest-javaagent.jar:$baseDir/scaladoc:/Users/jason/code/scala2/test/files/lib/annotations.jar:/Users/jason/code/scala2/test/files/lib/enums.jar:/Users/jason/code/scala2/test/files/lib/genericNest.jar:/Users/jason/code/scala2/test/files/lib/jsoup-1.3.1.jar:/Users/jason/code/scala2/test/files/lib/macro210.jar:/Users/jason/code/scala2/test/files/lib/methvsfield.jar:/Users/jason/code/scala2/test/files/lib/nest.jar:/Users/jason/code/scala2/test/lib:/Users/jason/.ivy2/cache/org.apache.ant/ant/jars/ant-1.9.4.jar:/Users/jason/.ivy2/cache/org.apache.ant/ant-launcher/jars/ant-launcher-1.9.4.jar:/Users/jason/.ivy2/cache/org.scala-lang.modules/scala-asm/bundles/scala-asm-5.0.4-scala-3.jar:/Users/jason/.ivy2/cache/org.scala-lang.modules/scala-xml_2.11/bundles/scala-xml_2.11-1.0.4.jar:/Users/jason/.ivy2/cache/org.scala-lang.modules/scala-parser-combinators_2.11/bundles/scala-parser-combinators_2.11-1.0.4.jar:/Users/jason/.ivy2/cache/jline/jline/jars/jline-2.12.1.jar:/Users/jason/.ivy2/local/org.scala-lang.modules/scala-partest_2.11/1.0.12-SNAPSHOT/jars/scala-partest_2.11.jar:/Users/jason/.ivy2/cache/com.googlecode.java-diff-utils/diffutils/jars/diffutils-1.3.0.jar:/Users/jason/.ivy2/cache/org.scala-sbt/test-interface/jars/test-interface-1.0.jar:/Users/jason/.ivy2/cache/org.scala-lang.modules/scala-partest-interface_2.11/jars/scala-partest-interface_2.11-0.7.0.jar:/Users/jason/.ivy2/cache/org.scalacheck/scalacheck_2.11/jars/scalacheck_2.11-1.11.6.jar:/Users/jason/.sbt/boot/scala-2.10.5/org.scala-sbt/sbt/0.13.9/test-agent-0.13.9.jar:/Users/jason/.sbt/boot/scala-2.10.5/org.scala-sbt/sbt/0.13.9/test-interface-1.0.jar
Java binaries in:    /Library/Java/JavaVirtualMachines/jdk1.8.0_71.jdk/Contents/Home/jre/bin
Java runtime is:     Java HotSpot(TM) 64-Bit Server VM (build 25.71-b15, mixed mode)
Java options are:    -Xmx1024M -Xms64M -XX:MaxPermSize=128M
baseDir:             /Users/jason/code/scala2/build-sbt/quick/classes
sourceDir:           /Users/jason/code/scala2/test/scaladoc

Selected 1 tests drawn from specified tests

# starting 1 test in run
% scalac run/SI-191.scala
!! 1 - run/SI-191.scala                          [non-zero exit code]
#0/1 passed, 1 failed in run

[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error]     partest
[error] (test/it:testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 9 s, completed 07/02/2016 12:38:16 AM
> eval IO.read(file("test/scaladoc/run/SI-191-run.log"))
[info] ans: String = Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128M; support was removed in 8.0
[info] java.lang.AssertionError: assertion failed
[info]  at scala.Predef$.assert(Predef.scala:156)
[info]  at Test$.check$1(SI-191.scala:49)
[info]  at Test$.testModel(SI-191.scala:52)
[info]  at scala.tools.partest.ScaladocModelTest.show(ScaladocModelTest.scala:71)
[info]  at scala.tools.partest.DirectTest.delayedEndpoint$scala$tools$partest$DirectTest$1(DirectTest.scala:92)
[info]  at scala.tools.partest.DirectTest$delayedInit$body.apply(DirectTest.scala:18)
[info]  at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
[info]  at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
[info]  at scala.App$$anonfun$main$1.apply(App.scala:76)
[info]  at scala.App$$anonfun$main$1.apply(App.scala:76)
[info]  at scala.collection.immutable.List.foreach(List.scala:381)
[info]  at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
[info]  at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:45)
[info]  at scala.App$class.main(App.scala:76)
[info]  at scala.tools.partest.DirectTest.main(DirectTest.scala:18)
[info]  at Test.main(SI-191.scala)
[info]  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[info]  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[info]  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[info]  at java.lang.reflect.Method.invoke(Method.java:497)
[info]  at scala.reflect.internal.util.ScalaClassLoader$$anonfun$run$1.apply(ScalaClassLoader.scala:70)
[info]  at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
[info]  at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.asContext(ScalaClassLoader.scala:101)
[info]  at scala.reflect.internal.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:70)
[info]  at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:101)
[info]  at scala.tools.nsc.CommonRunner$class.run(ObjectRunner.scala:22)
[info]  at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:39)
[info]  at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
[info]  at scala.tools.nsc.ObjectRunner$.runAndCatch(ObjectRunner.scala:39)
[info]  at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:65)
[info]  at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:87)
[info]  at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:98)
[info]  at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)
[info]  at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)

I attempted to thread this through. I got as far as:

diff --git a/src/main/scala/scala/tools/partest/nest/SBTRunner.scala b/src/main/scala/scala/tools/partest/nest/SBTRunner.scala
index acc956f..368e65f 100644
--- a/src/main/scala/scala/tools/partest/nest/SBTRunner.scala
+++ b/src/main/scala/scala/tools/partest/nest/SBTRunner.scala
@@ -62,7 +62,7 @@ class SBTRunner(partestFingerprint: Fingerprint, eventHandler: EventHandler, log
     case Pass(_)          => (Status.Success, new OptionalThrowable)
     case Updated(_)       => (Status.Success, new OptionalThrowable)
     case Skip(_, _)       => (Status.Skipped, new OptionalThrowable)
-    case Fail(_, _, _)    => (Status.Failure, new OptionalThrowable)
+    case Fail(_, reason, transcript)    => (Status.Failure, new OptionalThrowable(new Throwable(reason + "\n\n" + transcript.mkString("\n"))))
     case Crash(_, e, _)   => (Status.Error, new OptionalThrowable(e))
   }
 }

Which gets it into target/partest/test-reports/partest.xml files (via JUnitXMLListener).

[retronym]

/cc @SethTisue @szeiger

[szeiger]

I don't know what getting it into the report would buy us (IDE integration?) but printing it to the console like in ConsoleRunner is straight-forward: scala/scala-partest@master...szeiger:wip/verbose-output

[retronym]

Console output would be enough.