Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Key Parsing Rough Edges #445

Open
GRVYDEV opened this issue Jun 16, 2022 · 1 comment
Open

Key Parsing Rough Edges #445

GRVYDEV opened this issue Jun 16, 2022 · 1 comment

Comments

@GRVYDEV
Copy link

GRVYDEV commented Jun 16, 2022

Hello,
I have recently configured snowflake-kafka-connector a couple of times and i noticed that the key parsing is quite fragile. I am curious if there is a way to potentially make the parsing clean up newlines in the key. Essentially before actually parsing the key, iterate through it and check if there is a valid ascii line after each newline character. If so, then remove the newline character. This may make it easier to get setup since I had quite a bit of trouble figuring out the root cause of the parse errors.

@JonathonO
Copy link

JonathonO commented Nov 15, 2022

@GRVYDEV out of interest, are you referring to 'rough edges' like in the RECORD_METADATA where the key seems to have escaped unicode characters before it? For example (edited for sake of brevity):

{
  ...
  "key": "\u0000\u0000\u0000\u0000\u0018\u00148f14e45fce",
  "offset": 6,
  "partition": 0,
  ...
}

Feels like it's an escape sequence of some sort but not seeing any references to this issue, specifically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants