$ doesn't match CRLF

[jminer]

I created a regex with multi_line set to true, but after debugging why the regex was matching in a unittest but not in a file, I found out that $ isn't matching the end of a line in the file. I'm using Windows so the newlines are \r\n.

[BurntSushi]

Could you please provide a test case that doesn't act as you expect?

[jminer]

Sure, here is a program that I would expect to print "Matched: true", but it prints "Matched: false":

extern crate regex;

use regex::RegexBuilder;

fn main() {
    let regex = RegexBuilder::new("^apple$").multi_line(true).compile().unwrap();
    let text = "\r\napple\r\nbanana";
    let mut matched = false;
    for _ in regex.captures_iter(text) {
        matched = true;
    }
    println!("Matched: {}", matched);
}

[BurntSushi]

That is indeed expected behavior. $ will only match \n in multi line mode. It's not clear to me whether supporting \r\n is feasible unfortunately.

[BatmanAoD]

@BurntSushi Would treating \r as an end-of-line character, and \n as non-EOL if preceded by \r, be an acceptable change? This should be doable in a DFA, I think, though it does mean two EOL states. It might be somewhat surprising behavior when \r is embedded in a line, but that seems like a much rarer case than EOL \r, and it's not even clear to me that treating carriage return as EOL is actually wrong.

[BurntSushi]

@BatmanAoD That sounds feasible from an implementation perspective, but I'm not a fan of implementing something that's wrong. (Essentially no systems use \r for line endings any more, and on Windows, it's \r\n, not \r.)

[BatmanAoD]

But the existing implementation is more wrong, so I'm not sure I understand that as an objection.

[BatmanAoD]

(Also, just last week I actually did run into something that uses \r on its own as EOL by default: Putty in serial mode! I was shocked.)

[mitsuhiko]

I just ran into this when trying to use it to parse some data coming in via HTTP. This is incredibly confusing at the very leasts as this is the only regex engine I know with this behavior.

[BurntSushi]

I believe RE2 also has this behavior. `\r?$` may be a not-ideal work around.

…

On Mar 22, 2017 3:03 PM, "Armin Ronacher" ***@***.***> wrote: I just ran into this when trying to use it to parse some data coming in via HTTP. This is incredibly confusing at the very leasts as this is the only regex engine I know with this behavior. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#244 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAb34k3E9W0z4yc8ouT2YOg23NVVsbWpks5roXCIgaJpZM4IwcKx> .

[BurntSushi]

I'm going to re-open this, but I don't have any immediate plans to work on it.

[mitsuhiko]

Just wanted to throw in that I was wrong. It's indeed the same behavior in Python and Go as well. I did never notice in the former because a similar code I was working with stripped the EOL chars whereas the API I used in rust only split on \n and left an empty \r hanging around.

So when parts of the data were recombined into a string later the \r were left in there.

[mitsuhiko]

So the regex behavior here does match other engines dispite what I said earlier.