Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iceberg write.parquet.compression-codec being set even if file-format is not parquet #20401

Closed
oneonestar opened this issue Jan 17, 2024 · 1 comment
Labels
bug Something isn't working iceberg Iceberg connector

Comments

@oneonestar
Copy link
Member

oneonestar commented Jan 17, 2024

In Trino 436 (Iceberg 1.4.3), write.parquet.compression-codec property is also being set even if the file-format is not parquet. This doesn't happen in Trino 428 (Iceberg 1.3.1), not sure about Trino 431 to Trino 435 (Iceberg 1.4.1)
Is this an intended behavior?

trino> CREATE TABLE test.property_test (c1 integer) WITH (format = 'ORC');
CREATE TABLE
trino> SELECT * FROM test."property_test$properties";
               key               | value
---------------------------------+-------
 write.format.default            | ORC
 write.parquet.compression-codec | zstd
(2 rows)

trino> CREATE TABLE test.property_test (c1 integer) WITH (format = 'AVRO');
CREATE TABLE
trino> SELECT * FROM test."property_test$properties";
               key               | value
---------------------------------+-------
 write.format.default            | AVRO
 write.parquet.compression-codec | zstd
(2 rows)

Ref:
apache/iceberg#8593 (comment)
https://github.com/trinodb/trino/pull/19434/files#diff-f5cd96d1ec33fae497df9613afcefff034f01c99adf961e927ccd76e637c7b81R4707

@ebyhr
Copy link
Member

ebyhr commented Nov 28, 2024

This is not a bug of Trino Iceberg connector. It should be fixed in Iceberg library though they probably won't fix it, though. apache/iceberg#9503 (comment)

@ebyhr ebyhr closed this as not planned Won't fix, can't repro, duplicate, stale Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working iceberg Iceberg connector
Development

No branches or pull requests

3 participants