Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Spec: Clarify missing fields when writing (apache#8672)
* Spec: Carify missing fields when writing Jan raised a point on slack of the symantic meaning of a field that can be written: https://apache-iceberg.slack.com/archives/C03LG1D563F/p1695834739711569 There are two options: - The field is not part of the schema, and omitted from the file - The field is part of the schema, but the value is not written (nullable) My personal take on this is that we should use static schema's when writing Avro files, so that all the fields that are either optional or required are in the schema. I'm well aware of that this doesn't impose any issues if you dogfood the Iceberg Avro reader, where you can add required fields, for example the `134: content` field in the manifest. However, I think we should try to stick to the concept of write strict, read permissive where we try to encourage people to write all the fields that are in the spec (even they if the value itself is all null). * Add manifest-list explicitly Co-authored-by: JFinis <[email protected]> * Update wording * Comments * Retain formatting * Thanks Steven --------- Co-authored-by: JFinis <[email protected]>
- Loading branch information