[Debezium] Adding all Timestamp converters #865
Conversation
lib/debezium/types.go
Outdated
| MicroDuration SupportedDebeziumType = "io.debezium.time.MicroDuration" | ||
| DateKafkaConnect SupportedDebeziumType = "org.apache.kafka.connect.data.Date" | ||
| Timestamp SupportedDebeziumType = "io.debezium.time.Timestamp" | ||
| TimestampKafkaConnect SupportedDebeziumType = "org.apache.kafka.connect.data.Timestamp" |
There was a problem hiding this comment.
Only change here is that I renamed this to TimestampKafkaConnect to be more correct
| Timestamp SupportedDebeziumType = "io.debezium.time.Timestamp" | ||
| MicroTimestamp SupportedDebeziumType = "io.debezium.time.MicroTimestamp" | ||
| NanoTimestamp SupportedDebeziumType = "io.debezium.time.NanoTimestamp" | ||
| // Dates |
There was a problem hiding this comment.
Just organized this
Tang8330
changed the title
Tang8330
changed the title
Tang8330
changed the title
lib/debezium/schema_test.go
Outdated
| @@ -85,11 +85,6 @@ func TestField_ToKindDetails(t *testing.T) { | |||
| assert.Equal(t, typing.Integer, Field{Type: Int32}.ToKindDetails()) | |||
| assert.Equal(t, typing.Integer, Field{Type: Int64}.ToKindDetails()) | |||
| } | |||
| { | |||
There was a problem hiding this comment.
Is this change intentional?
There was a problem hiding this comment.
Deleting this? No, must've happened from the merge. Thanks for the catch
lib/typing/ext/variables.go
Outdated
| @@ -35,3 +35,10 @@ var SupportedTimeFormatsLegacy = []string{ | |||
| PostgresTimeFormatNoTZ, | |||
| AdditionalTimeFormat, | |||
| } | |||
|
|
|||
| // RFC 339 variants | |||
There was a problem hiding this comment.
Nit: missing a 3 in RFC 3339
| } | ||
|
|
||
| // Represents the number of milliseconds since the epoch, and does not include timezone information. | ||
| return ext.NewExtendedTime(time.UnixMilli(castedValue).In(time.UTC), ext.DateTimeKindType, ext.RFC3339Millisecond), nil |
There was a problem hiding this comment.
Do we need to call IsValid() on the return value from NewExtendedTime?
There was a problem hiding this comment.
I don't think so, if we are able to correctly get time.Time and parse the year to be larger than 9999, it still passes time.Format(...)
| } | ||
| { | ||
| // micros is preserved despite it being all zeroes. | ||
| converted, err := MicroTimestamp{}.Convert(int64(1_712_609_795_827_123)) |
There was a problem hiding this comment.
Should the input for this test end in _000_000?
There was a problem hiding this comment.
Yes it should. Updating the test
Changes
Refactoring the following converters: