Fix backslash handling in rows_from_chunks #249
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This should fix #242.
In the
rows_from_chunks
function, when deciding whether or not a quote character indicates the end of a string, it only checked the preceding character to see if it was a backslash or not. But it didn't check if that backslash was escaped or not.So, if a row contains a string with a properly escaped backslash right before the closing quote (e.g.
"\\"
), the row chunker will think that the closing quote is escaped. So then the number of braces it sees in the rest of the input won't be balanced, and then it won't ever properly process the rest of the JSON.I changed
rows_from_chunks
to keep track of escaped character state in a string to avoid this, and added a couple of unit tests to check for this scenario.