-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Fix escaping issue in filterExpression for RedisVectorStore file name #3202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
… filtering Signed-off-by: Minu Kim <[email protected]>
Thanks for the PR, I am afraid to merge this due to unintended sdie effects right before the release. Anything with Regex is always a difficult thing to deal with and make sure it is correct. |
… filtering Signed-off-by: Minu Kim <[email protected]>
Hi, thank you for the thoughtful feedback — I completely understand your concern about introducing regex-related changes right before a release. It's true that regex-related logic can be tricky and prone to side effects if not carefully verified. To help mitigate this, I’ve added comprehensive test coverage focused on exactly the kind of edge cases that could cause parsing issues. Specifically, the tests ensure that string literals containing dots (.) and backslashes () are properly escaped only within quoted strings, and not elsewhere in the expression. Here are the cases covered: Dot and backslash escaping inside file names and paths:
Multiple independent literals with dots:
Various file extensions:
List-style IN clause handling:
Each test verifies that escaping is applied consistently and correctly. I'm happy to improve test coverage further if that’s preferable. Thanks again for the review 🙏 |
Matcher matcher = pattern.matcher(expression); | ||
StringBuffer sb = new StringBuffer(); | ||
while (matcher.find()) { | ||
String content = matcher.group(1).replace("\\", "\\\\").replace(".", "\\."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
static Map<String, String> escapeText = {
// from, to
"\\", "\\\\",
".", "\\."
}
nit. how about extracting escape target texts to variable?
maybe we need more escape text pattern in the future.
… filtering Signed-off-by: Minu Kim <[email protected]>
@markpollack |
Issues#2945
Problem
When using
SearchRequest.Builder#filterExpression(String)
in a Redis-backed vector store, expressions that include file names such as'java编程思想.pdf'
or Windows-style paths like'C:\\docs\\file.txt'
cause parsing errors due to unescaped special characters (e.g.,.
or\
).These characters are treated as operators or escape tokens by the internal parser, leading to failures when the input is passed directly to
FilterExpressionTextParser
.Changes Introduced
Introduced an internal
escapeTextExpression(String)
method inSearchRequest.Builder
to automatically escape:\
) →\\
.
) →\.
filterExpression(String)
before parsing viaFilterExpressionTextParser
.