Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][Connectors-v2]Paimon version upgrade to 0.8.2 #8074

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from

Conversation

hawk9821
Copy link
Contributor

@hawk9821 hawk9821 commented Nov 17, 2024

Purpose of this pull request

doris has so far supported the Piamon . doris-paimon , Paimon version upgrade to 0.8.2 .

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

@@ -636,7 +636,7 @@ public void testChangelogLookup(TestContainer container) throws Exception {
.collect(Collectors.toList());
log.info("paimon records: {}", actual2);
Assertions.assertEquals(2, actual2.size());
Assertions.assertEquals(Arrays.asList("[+U, 1, Aa, 200]", "[+I, 2, Bb, 90]"), actual2);
Assertions.assertEquals(Arrays.asList("[+I, 1, Aa, 200]", "[+I, 2, Bb, 90]"), actual2);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why change this?

@@ -19,7 +19,7 @@
######

env {
parallelism = 2
parallelism = 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@@ -636,7 +636,7 @@ public void testChangelogLookup(TestContainer container) throws Exception {
.collect(Collectors.toList());
log.info("paimon records: {}", actual2);
Assertions.assertEquals(2, actual2.size());
Assertions.assertEquals(Arrays.asList("[+U, 1, Aa, 200]", "[+I, 2, Bb, 90]"), actual2);
Assertions.assertEquals(Arrays.asList("[+I, 1, Aa, 200]", "[+I, 2, Bb, 90]"), actual2);
Copy link
Contributor

@dailai dailai Nov 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your test case may be not all right. Your test case which paimon to paimon in streaming mode only read the data files. Currently, We have not supported read the changelog file in paimon source reader. We just only support streaming read. If you want to read changelog file, you should use it like this:

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stream read is not equivalent to reading the changelog file, I will commmit a Pr for read the changelog file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants