Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle unicode characters in filters #707

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Conversation

bdemers
Copy link
Member

@bdemers bdemers commented Dec 22, 2024

  • Updated ANTLR4 grammar to support unicode chars
  • Added test
  • Removed logic that escapes/unescapes JSON values, should is handled by at the JSON parser (jackson)

Fixes: #564

- Updated ANTLR4 grammar to support unicode chars
- Added test
- Removed logic that escapes/unescapes JSON values, should is handled by at the JSON parser (jackson)

Fixes: #564
Copy link

View details about the 'Apache Rat - Check' build 151 in the Build Scan 📊

Copy link

View details about the 'Maven Verify (with Java 21-zulu)' build 1419 in the Build Scan 📊

Copy link

View details about the 'Maven Verify (with Java 17-zulu)' build 1419 in the Build Scan 📊

@bdemers
Copy link
Member Author

bdemers commented Dec 22, 2024

@joshuapsteele, took a little longer to track the issue down, but when you get a chance can you take a look at this.

@chrisharm if you are still watching can you take a look at this too? I'm guessing this logic was here for a reason and I'm guessing I shouldn't be ripping it out... Do you have ideas for test cases for the original escape logic?

Copy link
Contributor

@joshuapsteele joshuapsteele left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me. Would just like to clarify where escapeJson() is now being used in the relevant flows here.

Comment on lines -91 to -96
// TODO change this to escapeJson() when dependencies get upgraded
String escaped = StringEscapeUtils.escapeEcmaScript((String) this.compareValue)
// StringEscapeUtils follows the outdated JSON spec requiring "/" to be escaped, this could subtly break things
.replaceAll("\\\\/", "/")
// We don't want single-quotes escaped, this will be unnecessary with escapeJson()
.replaceAll("\\\\'", "'");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, just now giving this a quick look, @bdemers. Can we note for posterity, even if just in this PR convo, where escapeJson() is now being used?

Comment on lines -163 to -170
String doubleEscaped = jsonValue.substring(1, jsonValue.length() - 1)
// StringEscapeUtils follows the outdated JSON spec requiring "/" to be escaped, this could subtly break things
.replaceAll("\\\\/", "\\\\\\\\/")
// Just in case someone needs a single-quote with a backslash in front of it, this will be unnecessary with escapeJson()
.replaceAll("\\\\'", "\\\\\\\\'");

// TODO change this to escapeJson() when dependencies get upgraded
return StringEscapeUtils.unescapeEcmaScript(doubleEscaped);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bdemers Can we note for posterity, even if just in this PR convo, where escapeJson() is now being used?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Special Characters Not Handled Correctly in Filters
2 participants