QuotationAttributionAnnotator accesses ArrayList out of bounds (causes IndexOutOfBoundsException)

[krallus]

OS: Linux Mint 20.1 Ulyssa (base: Ubuntu 20.04 focal)
Java: openjdk version "11.0.9.1"
CoreNLP: 4.2.0 (also 4.1.0)

Command line:

java -Xmx10g edu.stanford.nlp.pipeline.StanfordCoreNLP -annotators tokenize,ssplit,pos,lemma,ner,depparse,coref,quote -file input.txt -outputFormat text

Contents of input.txt:

John Doe:"foo"

Note that the problem occurs when the first line of any document is of the above form. A name (first [middle] last), punctuation (tested with colon and comma), and then a quote.

Exception thrown:

Exception in thread "main" java.lang.IndexOutOfBoundsException: Index -1 out of bounds for length 6
	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
	at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
	at java.base/java.util.Objects.checkIndex(Objects.java:372)
	at java.base/java.util.ArrayList.get(ArrayList.java:459)
	at edu.stanford.nlp.quoteattribution.Sieves.QMSieves.TrigramSieve.trigramPatterns(TrigramSieve.java:71)
	at edu.stanford.nlp.quoteattribution.Sieves.QMSieves.TrigramSieve.doQuoteToMention(TrigramSieve.java:27)
	at edu.stanford.nlp.pipeline.QuoteAttributionAnnotator.annotate(QuoteAttributionAnnotator.java:251)
	at edu.stanford.nlp.pipeline.QuoteAnnotator.annotate(QuoteAnnotator.java:292)
	at edu.stanford.nlp.pipeline.AnnotationPipeline.annotate(AnnotationPipeline.java:76)
	at edu.stanford.nlp.pipeline.StanfordCoreNLP.annotate(StanfordCoreNLP.java:653)
	at edu.stanford.nlp.pipeline.StanfordCoreNLP.annotate(StanfordCoreNLP.java:663)
	at edu.stanford.nlp.pipeline.StanfordCoreNLP.processFiles(StanfordCoreNLP.java:1261)
	at edu.stanford.nlp.pipeline.StanfordCoreNLP.processFiles(StanfordCoreNLP.java:1095)
	at edu.stanford.nlp.pipeline.StanfordCoreNLP.run(StanfordCoreNLP.java:1361)
	at edu.stanford.nlp.pipeline.StanfordCoreNLP.main(StanfordCoreNLP.java:1430)

[AngledLuffa]

We fixed this a little while ago: 0b4bf8d I've lost track of exactly when I broke the serialization for some of the models in the dev branch, so the easiest thing would be if you can download that one patch and rebuild it locally. If that's not something you feel comfortable doing, let me know and I can put together a patched version for you.

[krallus]

Hello. Thank you for your reply. I applied the change you referenced. However, that did not fix this particular problem as VocativeSeive is not what throws the exception. The line that throws the exception is in TrigramSieve. I fixed this locally by making the following changes. However, I'm not familiar enough with the code base to know whether or not this will have unintended consequences.

--- a/src/edu/stanford/nlp/quoteattribution/Sieves/QMSieves/TrigramSieve.java
+++ b/src/edu/stanford/nlp/quoteattribution/Sieves/QMSieves/TrigramSieve.java
@@ -68,8 +68,8 @@ public class TrigramSieve extends QMSieve {
           }
           //VCQ
           if (lastNameIndex.second.equals(quoteBeginTokenIndex - 1 - offset)) {
-            CoreLabel secondPrevToken = docTokens.get(lastNameIndex.first - 1);
-            if (secondPrevToken.tag().startsWith("V")) {
+               CoreLabel secondPrevToken = lastNameIndex.first >= 1 ? docTokens.get(lastNameIndex.first - 1) : null;
+            if (secondPrevToken != null && secondPrevToken.tag().startsWith("V")) {
               fillInMention(quote, names.get(names.size() - 1), lastNameIndex.first, lastNameIndex.second, "trigram VCQ", NAME);
               continue;
             }

[AngledLuffa]

Thanks for the patch! Would you like to submit that (to our dev branch) in the form of a pull request, or should I just apply it for you? I don't exactly know the details of this code, but my belief is that skipping this particular test just makes it go further along to see if some other part of the sieve connects the quote. Letting it NPE is already an unintended consequence...

[krallus]

I submitted the pull request and I'm happy to have contributed. Thanks for your assistance.

[AngledLuffa]

Thank you!