Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MQTT Connector: Can't use _key in WITHTARGET #968

Open
jadamon42 opened this issue Jul 17, 2023 · 1 comment
Open

MQTT Connector: Can't use _key in WITHTARGET #968

jadamon42 opened this issue Jul 17, 2023 · 1 comment

Comments

@jadamon42
Copy link

jadamon42 commented Jul 17, 2023

What version of the Stream Reactor are you reporting this issue for?

MQTT Sink Connector

What is the expected behaviour?

I want to do some kcql like the following, given that the key for my kafka record is an AVRO object

INSERT INTO `some/mqtt/topic/$_key.uniqueKey`
SELECT * 
FROM some.kafka.topic
WITHTARGET _key.uniqueKey

What was observed?

Doesn't work. We can do INSERT INTO _key but dynamic targets only work if the target is a field within the value.

EDIT

Also found out this fails too:

INSERT INTO `some/static/mqtt/topic`
SELECT _key.uniqueKey, someField, someField2
FROM some.kafka.topic

causes the following:

Encountered error A KCQL error occurred.Can't find the field '_key' 
@jadamon42
Copy link
Author

I'm having all sorts of issues with this connector. My avro schema has a field that is an array of records ([{}, {}, {}]). I want to include that what I'm sending to my MQTT topic, but I get A KCQL error occurred.Can't flatten from schema:Schema{ARRAY} by selecting 'myFieldThatIsAnArray'

I tried looking into this on my own, as came across using WITHSTRUCTURE as a possible solution. This is essentially what I tried doing:

INSERT INTO blah
SELECT feild1.subfield AS alias, field2.subfield AS alias2, field2.someArray
FROM blah WITHSTRUCTURE

But I got a really strange error: Encountered error A KCQL error occurred.alias2 can't be found in field1,field2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants