POSIX character classes supported outside of char ranges `[...]`

[birkenfeld]

This crate's regex syntax allows the POSIX-like character classes outside of brackets. In all other regex incarnations I know, you have to write [[:alpha:]]. This crate seems to allow [:alpha:].

I think this is an extension to usual syntax extension, but sadly it makes the syntax needlessly incompatible. I assume you don't allow [: to start a normal character range?

[BurntSushi]

I'm laughing, because it looks like it is treated specially in the parser (so that, e.g., [:alphaa:] is just a regular character class). Here's the method in question:

    // Parses an ASCII class, e.g., `[:alnum:]+`.
    //
    // Start: `[`
    // End:   `+`
    //
    // Also supports negation, e.g., `[:^alnum:]`.
    //
    // This parsing routine is distinct from the others in that it doesn't
    // actually report any errors. Namely, if it fails, then the parser should
    // fall back to parsing a regular class.
    //
    // This method will only make progress in the parser if it succeeds.
    // Otherwise, the input remains where it started.
    fn maybe_parse_ascii(&mut self) -> Option<CharClass> 

Honestly, I don't remember explicitly thinking that we should diverge from the rest of the world on this point. I think this is what happened:

1. I personally never ever use [[:alpha:]] and their ilk, so I have zero experience with them.
2. Given my abundance of experience, I think I just naturally assumed that [:alpha:] would work.

I think you are indeed right that it doesn't.

I feel like the downside here is pretty minimal. It's true that it technically makes the syntax incompatible, but if you're porting a regex from some other regex engine to this one, then you're unlikely to trip over it. The reverse direction on the other hand...

Is this something worth changing for 1.0? I feel like the only compelling reason is to be consistent with other regex engines because more consistency is better when possible.

[birkenfeld]

I agree that if it reverts to normal ranges, the incompatibility is minimal. The only instance where I could see [:alpha:] meant as a character range is in generated regexes. Otherwise nobody would put the a and the : in there twice.

Still it is an incompatibility -- if there were no restraints on this crate's backwards compatibility I would remove it. There probably are though, so - your call :)

If you keep it, please explicitly mention this in the docs though. It will trip up people familiar with other regex libs otherwise.

[BurntSushi]

I plan on making a small number of breaking changes for 1.0, but I think this is the first one that breaks the semantics of matching. Nevertheless, it's pretty small, and consistency is important, so I'll add it to the 1.0 milestone for now. :-)

(If it doesn't get changed in 1.0 then I'd we're probably stuck with this for good.)

[BurntSushi]

Interestingly, some regex engines (like Go and presumably RE2) support formulations like [[:upper:][:blank:]] while Python's does not. I think that's a fine thing to do.

I will still fix this particular issue by treating [:upper:] as a character class containing the characters :upper:.

[birkenfeld]

Interestingly, some regex engines (like Go and presumably RE2) support formulations like [[:upper:][:blank:]] while Python's does not. I think that's a fine thing to do.

That's because Python's engine does not support character classes at all.