Skip to content

Improve javac error output parsing - Fix #336 #337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 20, 2023
Merged

Improve javac error output parsing - Fix #336 #337

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
48ac162
Fix typo in JavacCompiler::parseModernStream comment
kriegaex Dec 17, 2023
4377aa9
Improve javac error output parsing
kriegaex Dec 17, 2023
ac9412f
Parametrise some error message parser tests
kriegaex Dec 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions plexus-compilers/plexus-compiler-javac/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@
<artifactId>junit-jupiter-api</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-params</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest</artifactId>
Expand Down
77 changes: 46 additions & 31 deletions ...plexus-compiler-javac/src/main/java/org/codehaus/plexus/compiler/javac/JavacCompiler.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
import java.util.Properties;
import java.util.StringTokenizer;
import java.util.concurrent.ConcurrentLinkedDeque;
import java.util.regex.Pattern;

import org.codehaus.plexus.compiler.AbstractCompiler;
import org.codehaus.plexus.compiler.CompilerConfiguration;
Expand Down Expand Up @@ -627,6 +628,16 @@ private static CompilerResult compileInProcess0(Class<?> javacClass, String[] ar
return new CompilerResult(success, messages);
}

// Match ~95% of existing JDK exception name patterns (last checked for JDK 21)
private static final Pattern STACK_TRACE_FIRST_LINE = Pattern.compile("^(?:[\\w+.-]+\\.)[\\w$]*?(?:"
+ "Exception|Error|Throwable|Failure|Result|Abort|Fault|ThreadDeath|Overflow|Warning|"
+ "NotSupported|NotFound|BadArgs|BadClassFile|Illegal|Invalid|Unexpected|Unchecked|Unmatched\\w+"
+ ").*$");

// Match exception causes, existing and omitted stack trace elements
private static final Pattern STACK_TRACE_OTHER_LINE =
Pattern.compile("^(?:Caused by:\\s.*|\\s*at .*|\\s*\\.\\.\\.\\s\\d+\\smore)$");

/**
* Parse the output from the compiler into a list of CompilerMessage objects
*
Expand All @@ -643,13 +654,14 @@ static List<CompilerMessage> parseModernStream(int exitCode, BufferedReader inpu
StringBuilder buffer = new StringBuilder();

boolean hasPointer = false;
int stackTraceLineCount = 0;

while (true) {
line = input.readLine();

if (line == null) {
// javac output not detected by other parsing
// maybe better to ignore only the summary an mark the rest as error
// maybe better to ignore only the summary and mark the rest as error
String bufferAsString = buffer.toString();
if (buffer.length() > 0) {
if (bufferAsString.startsWith("javac:")) {
Expand All @@ -659,26 +671,44 @@ static List<CompilerMessage> parseModernStream(int exitCode, BufferedReader inpu
} else if (hasPointer) {
// A compiler message remains in buffer at end of parse stream
errors.add(parseModernError(exitCode, bufferAsString));
} else if (stackTraceLineCount > 0) {
// Extract stack trace from end of buffer
String[] lines = bufferAsString.split("\\R");
int linesTotal = lines.length;
buffer = new StringBuilder();
int firstLine = linesTotal - stackTraceLineCount;

// Salvage Javac localized message 'javac.msg.bug' ("An exception has occurred in the
// compiler ... Please file a bug")
if (firstLine > 0) {
final String lineBeforeStackTrace = lines[firstLine - 1];
// One of those two URL substrings should always appear, without regard to JVM locale.
// TODO: Update, if the URL changes, last checked for JDK 21.
if (lineBeforeStackTrace.contains("java.sun.com/webapps/bugreport")
|| lineBeforeStackTrace.contains("bugreport.java.com")) {
firstLine--;
}
}

// Note: For message 'javac.msg.proc.annotation.uncaught.exception' ("An annotation processor
// threw an uncaught exception"), there is no locale-independent substring, and the header is
// also multi-line. It was discarded in the removed method 'parseAnnotationProcessorStream',
// and we continue to do so.

for (int i = firstLine; i < linesTotal; i++) {
buffer.append(lines[i]).append(EOL);
}
errors.add(new CompilerMessage(buffer.toString(), CompilerMessage.Kind.ERROR));
}
}
return errors;
}

// A compiler error occurred, treat everything that follows as part of the error.
if (line.startsWith("An exception has occurred in the compiler")) {
buffer = new StringBuilder();

while (line != null) {
buffer.append(line);
buffer.append(EOL);
line = input.readLine();
}

errors.add(new CompilerMessage(buffer.toString(), CompilerMessage.Kind.ERROR));
return errors;
} else if (line.startsWith("An annotation processor threw an uncaught exception.")) {
CompilerMessage annotationProcessingError = parseAnnotationProcessorStream(input);
errors.add(annotationProcessingError);
if (stackTraceLineCount == 0 && STACK_TRACE_FIRST_LINE.matcher(line).matches()
|| STACK_TRACE_OTHER_LINE.matcher(line).matches()) {
stackTraceLineCount++;
} else {
stackTraceLineCount = 0;
}

// new error block?
Expand Down Expand Up @@ -714,21 +744,6 @@ static List<CompilerMessage> parseModernStream(int exitCode, BufferedReader inpu
}
}

private static CompilerMessage parseAnnotationProcessorStream(final BufferedReader input) throws IOException {
String line = input.readLine();
final StringBuilder buffer = new StringBuilder();

while (line != null) {
if (!line.startsWith("Consult the following stack trace for details.")) {
buffer.append(line);
buffer.append(EOL);
}
line = input.readLine();
}

return new CompilerMessage(buffer.toString(), CompilerMessage.Kind.ERROR);
}

private static boolean isMisc(String line) {
return startsWithPrefix(line, MISC_PREFIXES);
}
Expand Down
89 changes: 77 additions & 12 deletions ...mpiler-javac/src/test/java/org/codehaus/plexus/compiler/javac/ErrorMessageParserTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,20 +28,25 @@
import java.io.StringReader;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Stream;

import org.codehaus.plexus.compiler.CompilerMessage;
import org.codehaus.plexus.util.Os;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;

import static org.hamcrest.CoreMatchers.endsWith;
import static org.hamcrest.CoreMatchers.startsWith;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.is;
import static org.hamcrest.Matchers.notNullValue;
import static org.hamcrest.core.StringEndsWith.endsWith;
import static org.hamcrest.core.StringStartsWith.startsWith;

/**
* @author <a href="mailto:[email protected]">Trygve Laugst&oslash;l</a>
* @author Alexander Kriegisch
*/
public class ErrorMessageParserTest {
private static final String EOL = System.getProperty("line.separator");
Expand Down Expand Up @@ -775,19 +780,79 @@ public void testJava7Error() throws Exception {
assertThat(message2.getEndLine(), is(3));
}

@Test
public void testBugParade() throws Exception {
String out = "An exception has occurred in the compiler (1.7.0_80). "
+ "Please file a bug at the Java Developer Connection (http://java.sun.com/webapps/bugreport) after checking the Bug Parade for duplicates. "
+ "Include your program and the following diagnostic in your report. Thank you." + EOL
+ "com.sun.tools.javac.code.Symbol$CompletionFailure: class file for java.util.Optional not found";
@ParameterizedTest(name = "{0}")
@MethodSource("testBugParade_args")
public void testBugParade(String jdkAndLocale, String stackTraceHeader) throws Exception {
String stackTraceWithHeader = stackTraceHeader + stackTraceInternalCompilerError;

List<CompilerMessage> compilerErrors =
JavacCompiler.parseModernStream(4, new BufferedReader(new StringReader(out)));
List<CompilerMessage> compilerMessages =
JavacCompiler.parseModernStream(4, new BufferedReader(new StringReader(stackTraceWithHeader)));

assertThat(compilerErrors, notNullValue());
assertThat(compilerMessages, notNullValue());
assertThat(compilerMessages, hasSize(1));

assertThat(compilerErrors.size(), is(1));
String message = compilerMessages.get(0).getMessage().replaceAll(EOL, "\n");
// Parser retains stack trace header
assertThat(message, startsWith(stackTraceHeader));
assertThat(message, endsWith(stackTraceInternalCompilerError));
}

private static final String stackTraceInternalCompilerError =
"\tat com.sun.tools.javac.comp.MemberEnter.baseEnv(MemberEnter.java:1388)\n"
+ "\tat com.sun.tools.javac.comp.MemberEnter.complete(MemberEnter.java:1046)\n"
+ "\tat com.sun.tools.javac.code.Symbol.complete(Symbol.java:574)\n"
+ "\tat com.sun.tools.javac.code.Symbol$ClassSymbol.complete(Symbol.java:1037)\n"
+ "\tat com.sun.tools.javac.code.Symbol$ClassSymbol.flags(Symbol.java:973)\n"
+ "\tat com.sun.tools.javac.code.Symbol$ClassSymbol.getKind(Symbol.java:1101)\n"
+ "\tat com.sun.tools.javac.code.Kinds.kindName(Kinds.java:162)\n"
+ "\tat com.sun.tools.javac.comp.Check.duplicateError(Check.java:329)\n"
+ "\tat com.sun.tools.javac.comp.Check.checkUnique(Check.java:3435)\n"
+ "\tat com.sun.tools.javac.comp.Enter.visitTypeParameter(Enter.java:454)\n"
+ "\tat com.sun.tools.javac.tree.JCTree$JCTypeParameter.accept(JCTree.java:2224)\n"
+ "\tat com.sun.tools.javac.comp.Enter.classEnter(Enter.java:258)\n"
+ "\tat com.sun.tools.javac.comp.Enter.classEnter(Enter.java:272)\n"
+ "\tat com.sun.tools.javac.comp.Enter.visitClassDef(Enter.java:418)\n"
+ "\tat com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:693)\n"
+ "\tat com.sun.tools.javac.comp.Enter.classEnter(Enter.java:258)\n"
+ "\tat com.sun.tools.javac.comp.Enter.classEnter(Enter.java:272)\n"
+ "\tat com.sun.tools.javac.comp.Enter.visitTopLevel(Enter.java:334)\n"
+ "\tat com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:518)\n"
+ "\tat com.sun.tools.javac.comp.Enter.classEnter(Enter.java:258)\n"
+ "\tat com.sun.tools.javac.comp.Enter.classEnter(Enter.java:272)\n"
+ "\tat com.sun.tools.javac.comp.Enter.complete(Enter.java:486)\n"
+ "\tat com.sun.tools.javac.comp.Enter.main(Enter.java:471)\n"
+ "\tat com.sun.tools.javac.main.JavaCompiler.enterTrees(JavaCompiler.java:982)\n"
+ "\tat com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:857)\n"
+ "\tat com.sun.tools.javac.main.Main.compile(Main.java:523)\n"
+ "\tat com.sun.tools.javac.main.Main.compile(Main.java:381)\n"
+ "\tat com.sun.tools.javac.main.Main.compile(Main.java:370)\n"
+ "\tat com.sun.tools.javac.main.Main.compile(Main.java:361)\n"
+ "\tat com.sun.tools.javac.Main.compile(Main.java:56)\n"
+ "\tat com.sun.tools.javac.Main.main(Main.java:42)\n";

private static Stream<Arguments> testBugParade_args() {
return Stream.of(
Arguments.of(
"JDK 8 English",
"An exception has occurred in the compiler ({0}). Please file a bug at the Java Developer Connection (http://java.sun.com/webapps/bugreport) after checking the Bug Parade for duplicates. Include your program and the following diagnostic in your report. Thank you.\n"),
Arguments.of(
"JDK 8 Japanese",
"コンパイラで例外が発生しました({0})。Bug Paradeで重複がないかをご確認のうえ、Java Developer Connection (http://java.sun.com/webapps/bugreport)でbugの登録をお願いいたします。レポートには、そのプログラムと下記の診断内容を含めてください。ご協力ありがとうございます。\n"),
Arguments.of(
"JDK 8 Chinese",
"编译器 ({0}) 中出现异常错误。 如果在 Bug Parade 中没有找到该错误, 请在 Java Developer Connection (http://java.sun.com/webapps/bugreport) 中建立 Bug。请在报告中附上您的程序和以下诊断信息。谢谢。\n"),
Arguments.of(
"JDK 21 English",
"An exception has occurred in the compiler ({0}). Please file a bug against the Java compiler via the Java bug reporting page (https://bugreport.java.com) after checking the Bug Database (https://bugs.java.com) for duplicates. Include your program, the following diagnostic, and the parameters passed to the Java compiler in your report. Thank you.\n"),
Arguments.of(
"JDK 21 Japanese",
"コンパイラで例外が発生しました({0})。バグ・データベース(https://bugs.java.com)で重複がないかをご確認のうえ、Javaのバグ・レポート・ページ(https://bugreport.java.com)から、Javaコンパイラに対するバグの登録をお願いいたします。レポートには、該当のプログラム、次の診断内容、およびJavaコンパイラに渡されたパラメータをご入力ください。ご協力ありがとうございます。\n"),
Arguments.of(
"JDK 21 Chinese",
"编译器 ({0}) 中出现异常错误。如果在 Bug Database (https://bugs.java.com) 中没有找到有关该错误的 Java 编译器 Bug，请通过 Java Bug 报告页 (https://bugreport.java.com) 提交 Java 编译器 Bug。请在报告中附上您的程序、以下诊断信息以及传递到 Java 编译器的参数。谢谢。\n"),
Arguments.of(
"JDK 21 German",
"Im Compiler ({0}) ist eine Ausnahme aufgetreten. Erstellen Sie auf der Java-Seite zum Melden von Bugs (https://bugreport.java.com) einen Bugbericht, nachdem Sie die Bugdatenbank (https://bugs.java.com) auf Duplikate geprüft haben. Geben Sie in Ihrem Bericht Ihr Programm, die folgende Diagnose und die Parameter an, die Sie dem Java-Compiler übergeben haben. Vielen Dank.\n"));
}

@Test
Expand Down
Loading