Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BE: fixes: avro schema ser for nullable enums #685

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

DimaVilda
Copy link

  • Breaking change?

What changes did you make?
Fixed incorrect JSON schema generation for nullable enum types in Avro schemas. Previously, nullable enums were generated with generic "enum" key instead of the actual enum name, causing incorrect serialization format with error:

400 Bad Request
json value '{"enum":"CLUBS"}' is cannot be converted to any of union types

See bug reproduction in attached video!

Is there anything you'd like reviewers to focus on?
We can think about changing nullable enums json generation. I.e. instead of :

      {
        "enum_nullable_union": { "Suit": "SPADES"}
      }

Simply use

      {
        "enum_nullable_union": "SPADES"
      }

How Has This Been Tested? (put an "x" (case-sensitive!) next to an item)

  • No need to
  • Manually (please, describe, if necessary)
  • [ x] Unit checks
  • Integration checks
  • Covered by existing automation

Checklist (put an "x" (case-sensitive!) next to all the items, otherwise the build will fail)

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (e.g. ENVIRONMENT VARIABLES)
  • My changes generate no new warnings (e.g. Sonar is happy)
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged
kafkaUiBug.mp4

@DimaVilda DimaVilda requested a review from a team as a code owner November 29, 2024 15:57
@kapybro kapybro bot added status/triage Issues pending maintainers triage status/triage/manual Manual triage in progress status/triage/completed Automatic triage completed and removed status/triage Issues pending maintainers triage labels Nov 29, 2024
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi DimaVilda! 👋

Welcome, and thank you for opening your first PR in the repo!

Please wait for triaging by our maintainers.

Please take a look at our contributing guide.

@DimaVilda DimaVilda changed the title Bfixes: avro schema ser for nullable enums BE: fixes: avro schema ser for nullable enums Nov 29, 2024
@DimaVilda
Copy link
Author

DimaVilda commented Nov 30, 2024

hey there!
I'd like to suggest you change Avro ser process for union nullable enums so it looks like simple enum in json.
Example: consider you have a following nullable enum in your Avro *.avdl schema:

"name": "booksize"
"type" : [ "null", {
"type" : "enum",
"name" : "BookSizeEnum",
"symbols" : [ "ORIGINAL", "A3", "A4" ]
} ]

then kafka-ui createUnionSchema AFTER MY FIX OFC method will transform it to:

"booksize": {
"oneOf": [
{ "type": "null" },
{
"type": "object",
"properties": {
"BookSizeEnum": { // NOTE that now you have "enum" in master
"type": "string",
"enum": ["ORIGINAL", "A3", "A4"]
}
}
}
]
}

Which will generate a message for topic like:

{
"booksize": { "BookSizeEnum": "ORIGINAL"}
}

However a simple enum (not nullable) looks like:

{
"BookSizeEnum": "ORIGINAL"
}
In my opinion it make more hustle and mess. What I suggest to do is simplify it and make look a nullable enum like a simple not union nullable enum so for:

"name": "booksize"
"type" : [ "null", {
"type" : "enum",
"name" : "BookSizeEnum",
"symbols" : [ "ORIGINAL", "A3", "A4" ]
} ]

Kafka-ui will generate:

{
"BookSizeEnum": "ORIGINAL"
}

or for null:

{
"BookSizeEnum": null
}

What do you think? Can I create a following issue for it and then proceed with this ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/triage/completed Automatic triage completed status/triage/manual Manual triage in progress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant