Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flink: port table refresh (#8555) to v1.15 and v1.16 #8616

Merged
merged 3 commits into from
Sep 22, 2023

Conversation

bryanck
Copy link
Contributor

@bryanck bryanck commented Sep 21, 2023

This PR ports the changes from #8555 (optional table refresh) to Flink version 1.15 and 1.16.

This includes two minor updates for all versions, that were missed in the original PR:

  • typo fix TABLE_REFRSH_INTERVAL -> TABLE_REFRESH_INTERVAL
  • import fix org.apache.calcite.linq4j.function.Experimental -> org.apache.flink.annotation.Experimental

@@ -379,7 +379,7 @@ public void testWriteRowWithTableRefreshInterval() throws Exception {
.map(CONVERTER::toInternal, FlinkCompatibilityUtil.toTypeInfo(SimpleDataUtil.ROW_TYPE));

Configuration flinkConf = new Configuration();
flinkConf.setString(FlinkWriteOptions.TABLE_REFRSH_INTERVAL.key(), "100ms");
flinkConf.setString(FlinkWriteOptions.TABLE_REFRESH_INTERVAL.key(), "100ms");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this!

Copy link
Contributor

@amogh-jahagirdar amogh-jahagirdar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@@ -18,6 +18,8 @@
*/
package org.apache.iceberg.flink;

import java.time.Duration;
import org.apache.calcite.linq4j.function.Experimental;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like the wrong import is being used here

import java.util.Map;
import org.apache.calcite.linq4j.function.Experimental;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong import?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we fix this one we should also fix it for flink 1.17

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, apologies, I fixed these.

Copy link
Contributor

@nastra nastra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I added a comment about a (potentially) wrong import on the Experimental usage

@pvary
Copy link
Contributor

pvary commented Sep 22, 2023

@bryanck: is this a clean backport? We tend to fix, change everything in 1.17 code , and then do a clean backport to 1.16, 1.15 whenever possible. If there is some change, we highlight it in comments on github.

@bryanck
Copy link
Contributor Author

bryanck commented Sep 22, 2023

@pvary This is a clean backport. Two minor items were discovered on this review which were applied to all versions, I added those to the description.

Copy link
Contributor

@nastra nastra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @bryanck. Also thanks to @Fokko / @amogh-jahagirdar / @pvary for reviewing

@nastra nastra merged commit 37d4890 into apache:master Sep 22, 2023
13 checks passed
@pvary
Copy link
Contributor

pvary commented Sep 22, 2023

@pvary This is a clean backport. Two minor items were discovered on this review which were applied to all versions, I added those to the description.

Thanks!
Well done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants