To fix the table configuration documentation for write.distrubition-mode #8517
Conversation
…ion from none to hash
| @@ -74,7 +74,7 @@ Iceberg tables support table properties to configure table behavior, like the de | |||
| | write.metadata.metrics.column.col1 | (not set) | Metrics mode for column 'col1' to allow per-column tuning; none, counts, truncate(length), or full | | |||
| | write.target-file-size-bytes | 536870912 (512 MB) | Controls the size of files generated to target about this many bytes | | |||
| | write.delete.target-file-size-bytes | 67108864 (64 MB) | Controls the size of delete files generated to target about this many bytes | | |||
| | write.distribution-mode | none | Defines distribution of write data: __none__: don't shuffle rows; __hash__: hash distribute by partition key ; __range__: range distribute by partition key or sort key if table has an SortOrder | | |||
| | write.distribution-mode | hash | Defines distribution of write data: __none__: don't shuffle rows; __hash__: hash distribute by partition key ; __range__: range distribute by partition key or sort key if table has an SortOrder | | |||
There was a problem hiding this comment.
Looking at code, it looks a bit more nuanced: none for unpartitioned table, hash for partitioned table, range for sorted table.
There was a problem hiding this comment.
Shall we document this in the description and leave the default value as none.
Defines distribution of write data: none: don’t shuffle rows; hash: hash distribute by partition key ; range: range distribute by partition key or sort key if table has an SortOrder. Default value none for unpartitioned table, hash for partitioned table and range for sorted table.
There was a problem hiding this comment.
Let's say the default is hash ... or "depends"? I still think none is wrong since almost no one is using unpartitioned tables
There was a problem hiding this comment.
This is not a normal case because default itself is depended on other factors.
There was a problem hiding this comment.
@RussellSpitzer Please suggest how to proceed ?
There was a problem hiding this comment.
For the default we could say "(Depends on Partitioning and Sort Order)"
|
Thanks for pr, also please make the PR description clear what it does (we can put the link to the issue in the PR description if you want) |
atifiu
changed the title
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions. |
|
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. |
Changing default value in documentation from
nonetohashforwrite.distribution-modeas reported in the issue.