Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix TINYINT(1) data type mapping in rds source #5214

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

oeyh
Copy link
Collaborator

@oeyh oeyh commented Nov 25, 2024

Description

TINYINT(1) columns show up as BIT type when we query the column types, which results in error:

2024-11-25T14:06:41,668 [rds-source-binlog-processor-1] ERROR org.opensearch.dataprepper.plugins.source.rds.stream.BinlogEventListener - Failed to process change event of type EXT_WRITE_ROWS
java.lang.IllegalArgumentException: Unsupported value type. The value is of type: class java.lang.Integer
    at org.opensearch.dataprepper.plugins.source.rds.datatype.impl.NumericTypeHandler.handleBit(NumericTypeHandler.java:63) ~[rds-source-2.11.0-SNAPSHOT.jar:?]
    at org.opensearch.dataprepper.plugins.source.rds.datatype.impl.NumericTypeHandler.handleNumericType(NumericTypeHandler.java:38) ~[rds-source-2.11.0-SNAPSHOT.jar:?]
    at org.opensearch.dataprepper.plugins.source.rds.datatype.impl.NumericTypeHandler.handle(NumericTypeHandler.java:25) ~[rds-source-2.11.0-SNAPSHOT.jar:?]
    at org.opensearch.dataprepper.plugins.source.rds.datatype.impl.NumericTypeHandler.handle(NumericTypeHandler.java:12) ~[rds-source-2.11.0-SNAPSHOT.jar:?]
    at org.opensearch.dataprepper.plugins.source.rds.datatype.DataTypeHelper.getDataByColumnType(DataTypeHelper.java:29) ~[rds-source-2.11.0-SNAPSHOT.jar:?]
    at org.opensearch.dataprepper.plugins.source.rds.stream.BinlogEventListener.handleRowChangeEvent(BinlogEventListener.java:373) ~[rds-source-2.11.0-SNAPSHOT.jar:?]
    at org.opensearch.dataprepper.plugins.source.rds.stream.BinlogEventListener.handleInsertEvent(BinlogEventListener.java:269) ~[rds-source-2.11.0-SNAPSHOT.jar:?]
    at org.opensearch.dataprepper.plugins.source.rds.stream.BinlogEventListener.lambda$handleEventAndErrors$1(BinlogEventListener.java:448) ~[rds-source-2.11.0-SNAPSHOT.jar:?]
    at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:141) ~[micrometer-core-1.13.0.jar:1.13.0]
    at org.opensearch.dataprepper.plugins.source.rds.stream.BinlogEventListener.handleEventAndErrors(BinlogEventListener.java:448) ~[rds-source-2.11.0-SNAPSHOT.jar:?]
    at org.opensearch.dataprepper.plugins.source.rds.stream.BinlogEventListener.lambda$processEvent$0(BinlogEventListener.java:442) ~[rds-source-2.11.0-SNAPSHOT.jar:?]

This PR sets JDBC connector property tinyInt1isBit to false so that the column type returned for TINYINT(1) would be TINYINT instead of BIT.

Reference: https://dev.mysql.com/doc/connector-j/en/connector-j-connp-props-result-sets.html#cj-conn-prop_tinyInt1isBit

Issues Resolved

N/A

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant