Doc: Add status page for different implementations. (#11772)

* Partial * Initial * Update site/docs/status.md Co-authored-by: Fokko Driesprong <[email protected]> * Update site/docs/status.md Co-authored-by: Fokko Driesprong <[email protected]> * Update site/docs/status.md Co-authored-by: Fokko Driesprong <[email protected]> * Update site/docs/status.md Co-authored-by: Fokko Driesprong <[email protected]> * Update site/docs/status.md Co-authored-by: Fokko Driesprong <[email protected]> * Update site/docs/status.md Co-authored-by: Fokko Driesprong <[email protected]> * Update site/docs/status.md Co-authored-by: Fokko Driesprong <[email protected]> * Update site/docs/status.md Co-authored-by: Fokko Driesprong <[email protected]> * Update site/docs/status.md Co-authored-by: Fokko Driesprong <[email protected]> * Update site/docs/status.md Co-authored-by: Fokko Driesprong <[email protected]> * Resolve comments * Fix comments * Fix comments * Move position * Remove 's' in implementations * Address comments * Add avro data file formats * Add puffin statistics --------- Co-authored-by: Fokko Driesprong <[email protected]>
apache · Dec 23, 2024 · 55f10ca · 55f10ca
1 parent f7748f2
commit 55f10ca
Show file tree

Hide file tree

Showing 2 changed files with 371 additions and 1 deletion.
diff --git a/site/docs/status.md b/site/docs/status.md
@@ -0,0 +1,369 @@
+---
+title: "Implementation Status"
+---
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements.  See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License.  You may obtain a copy of the License at
+ -
+ -   http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# Implementation Status
+
+Apache Iceberg's table specification is implemented in multiple languages. This page provides an overview of the current
+capabilities.
+
+## Libraries
+
+This section lists the libraries that implement the Apache Iceberg specification.
+
+- [Java](https://mvnrepository.com/artifact/org.apache.iceberg)
+- [PyIceberg](https://pypi.org/project/pyiceberg/)
+- [Rust](https://crates.io/crates/iceberg)
+- [Go](https://pkg.go.dev/github.com/apache/iceberg-go)
+
+## Data Types
+
+| Data Type      | Java | PyIceberg | Rust | Go |
+|----------------|------|-----------|------|----|
+| boolean        | Y    | Y         | Y    | Y  |
+| int            | Y    | Y         | Y    | Y  |
+| long           | Y    | Y         | Y    | Y  |
+| float          | Y    | Y         | Y    | Y  |
+| double         | Y    | Y         | Y    | Y  |
+| decimal        | Y    | Y         | Y    | Y  |
+| date           | Y    | Y         | Y    | Y  |
+| time           | Y    | Y         | Y    | Y  |
+| timestamp      | Y    | Y         | Y    | Y  |
+| timestamptz    | Y    | Y         | Y    | Y  |
+| timestamp_ns   | Y    | Y         | Y    | Y  |
+| timestamptz_ns | Y    | Y         | Y    | Y  |
+| string         | Y    | Y         | Y    | Y  |
+| uuid           | Y    | Y         | Y    | Y  |
+| fixed          | Y    | Y         | Y    | Y  |
+| binary         | Y    | Y         | Y    | Y  |
+| variant        | Y    | Y         | Y    | Y  |
+| list           | Y    | Y         | Y    | Y  |
+| map            | Y    | Y         | Y    | Y  |
+| struct         | Y    | Y         | Y    | Y  |
+
+## Data File Formats
+
+| Format  | Java | PyIceberg | Rust | Go |
+|---------|------|-----------|------|----|
+| Parquet | Y    | Y         | Y    | Y  |
+| ORC     | Y    | N         | N    | N  |
+| Puffin  | Y    | N         | N    | N  |
+| Avro    | Y    | N         | N    | N  |
+
+## File IO
+
+| Storage           | Java | PyIceberg | Rust | Go |
+|-------------------|------|-----------|------|----|
+| Local Filesystem  | Y    | Y         | Y    | Y  |
+| Hadoop Filesystem | Y    | Y         | Y    | Y  |
+| S3 Compatible     | Y    | Y         | Y    | Y  |
+| GCS Compatible    | Y    | Y         | Y    | Y  |
+
+## Table Maintenance Operations
+
+### Table Spec V1
+
+| Operation                   | Java | PyIceberg | Rust | Go |
+|-----------------------------|------|-----------|------|----|
+| Update schema               | Y    | N         | Y    | N  |
+| Update partition spec       | Y    | N         | Y    | N  |
+| Update table properties     | Y    | Y         | Y    | N  |
+| Replace sort order          | Y    | N         | N    | N  |
+| Update table location       | Y    | N         | N    | N  |
+| Update statistics           | Y    | N         | N    | N  |
+| Update partition statistics | Y    | N         | N    | N  |
+| Expire snapshots            | Y    | N         | N    | N  |
+| Manage snapshots            | Y    | N         | N    | N  |
+
+### Table Spec V2
+
+| Operation                   | Java | PyIceberg | Rust | Go |
+|-----------------------------|------|-----------|------|----|
+| Update schema               | Y    | Y         | N    | N  |
+| Update partition spec       | Y    | Y         | N    | N  |
+| Update table properties     | Y    | Y         | Y    | N  |
+| Replace sort order          | Y    | N         | N    | N  |
+| Update table location       | Y    | N         | N    | N  |
+| Update statistics           | Y    | N         | N    | N  |
+| Update partition statistics | Y    | N         | N    | N  |
+| Expire snapshots            | Y    | N         | N    | N  |
+| Manage snapshots            | Y    | N         | N    | N  |
+
+## Table Update Operations
+
+### Table Spec V1
+
+| Operation         | Java | PyIceberg | Rust | Go |
+|-------------------|------|-----------|------|----|
+| Append data files | Y    | Y         | N    | N  |
+| Rewrite files     | Y    | Y         | N    | N  |
+| Rewrite manifests | Y    | Y         | N    | N  |
+| Overwrite files   | Y    | Y         | N    | N  |
+| Delete files      | Y    | N         | N    | N  |
+
+### Table Spec V2
+
+| Operation         | Java | PyIceberg | Rust | Go |
+|-------------------|------|-----------|------|----|
+| Append data files | Y    | Y         | N    | N  |
+| Rewrite files     | Y    | Y         | N    | N  |
+| Rewrite manifests | Y    | Y         | N    | N  |
+| Overwrite files   | Y    | Y         | N    | N  |
+| Row delta         | Y    | N         | N    | N  |
+| Delete files      | Y    | Y         | N    | N  |
+
+## Table Read Operations
+
+### Table Spec V1
+
+| Operation                   | Java | PyIceberg | Rust | Go |
+|-----------------------------|------|-----------|------|----|
+| Plan with data file         | Y    | Y         | Y    | Y  |
+| Plan with puffin statistics | Y    | Y         | Y    | Y  |
+| Read data file              | Y    | N         | Y    | N  |
+
+### Table Spec V2
+
+| Operation                   | Java | PyIceberg | Rust | Go |
+|-----------------------------|------|-----------|------|----|
+| Plan with data file         | Y    | Y         | Y    | Y  |
+| Plan with position deletes  | Y    | Y         | N    | N  |
+| Plan with equality deletes  | Y    | Y         | N    | N  |
+| Plan with puffin statistics | Y    | N         | N    | N  |
+| Read data file              | Y    | Y         | Y    | N  |
+| Read with position deletes  | Y    | Y         | N    | N  |
+| Read with equality deletes  | Y    | N         | N    | N  |
+
+## Table Write Operations
+
+### Table Spec V1
+
+| Operation   | Java | PyIceberg | Rust | Go |
+|-------------|------|-----------|------|----|
+| Append data | Y    | Y         | Y    | N  |
+
+### Table Spec V2
+
+| Operation              | Java | PyIceberg | Rust | Go |
+|------------------------|------|-----------|------|----|
+| Append data            | Y    | Y         | Y    | N  |
+| Write position deletes | Y    | N         | N    | N  |
+| Write equality deletes | Y    | N         | N    | N  |
+
+## Catalogs
+
+### Rest Catalog
+
+#### Table Spec V1
+
+| Table Operation | Java | PyIceberg | Rust | Go |
+|-----------------|------|-----------|------|----|
+| listTable       | Y    | Y         | Y    | Y  |
+| createTable     | Y    | Y         | Y    | Y  |
+| dropTable       | Y    | Y         | Y    | Y  |
+| loadTable       | Y    | Y         | Y    | Y  |
+| updateTable     | Y    | Y         | Y    | Y  |
+| renameTable     | Y    | Y         | Y    | Y  |
+| tableExists     | Y    | Y         | Y    | Y  |
+
+#### Table Spec V2
+
+| Table Operation | Java | PyIceberg | Rust | Go |
+|-----------------|------|-----------|------|----|
+| listTable       | Y    | Y         | Y    | Y  |
+| createTable     | Y    | Y         | Y    | Y  |
+| dropTable       | Y    | Y         | Y    | Y  |
+| loadTable       | Y    | Y         | Y    | Y  |
+| updateTable     | Y    | Y         | Y    | Y  |
+| renameTable     | Y    | Y         | Y    | Y  |
+| tableExists     | Y    | Y         | Y    | Y  |
+
+#### View Spec V1
+
+| View Operation | Java | PyIceberg | Rust | Go |
+|----------------|------|-----------|------|----|
+| createView     | Y    | N         | N    | N  |
+| dropView       | Y    | N         | N    | N  |
+| listView       | Y    | N         | N    | N  |
+| viewExists     | Y    | N         | N    | N  |
+| replaceView    | Y    | N         | N    | N  |
+| renameView     | Y    | N         | N    | N  |
+
+#### Namespace Operations
+
+| Namespace Operation       | Java | PyIceberg | Rust | Go |
+|---------------------------|------|-----------|------|----|
+| listNamespaces            | Y    | Y         | Y    | Y  |
+| createNamespace           | Y    | Y         | Y    | Y  |
+| dropNamespace             | Y    | Y         | Y    | Y  |
+| namespaceExists           | Y    | N         | Y    | Y  |
+| updateNamespaceProperties | Y    | Y         | Y    | Y  |
+| loadNamespaceMetadata     | Y    | N         | Y    | Y  |
+
+### Sql Catalog
+
+The sql catalog is a catalog backed by a sql database, which is called jdbc catalog in java.
+
+| Database | Java | PyIceberg | Rust | Go |
+|----------|------|-----------|------|----|
+| Postgres | Y    | Y         | Y    | N  |
+| MySQL    | Y    | Y         | Y    | N  |
+| SQLite   | Y    | Y         | Y    | N  |
+
+#### Table Spec V1
+
+| Table Operation | Java | PyIceberg | Rust | Go |
+|-----------------|------|-----------|------|----|
+| listTable       | Y    | Y         | Y    | Y  |
+| createTable     | Y    | Y         | Y    | Y  |
+| dropTable       | Y    | Y         | Y    | Y  |
+| loadTable       | Y    | Y         | Y    | Y  |
+| updateTable     | Y    | Y         | Y    | Y  |
+| renameTable     | Y    | Y         | Y    | Y  |
+| tableExists     | Y    | Y         | Y    | Y  |
+
+#### Table Spec V2
+
+| Table Operation | Java | PyIceberg | Rust | Go |
+|-----------------|------|-----------|------|----|
+| listTable       | Y    | Y         | Y    | Y  |
+| createTable     | Y    | Y         | Y    | Y  |
+| dropTable       | Y    | Y         | Y    | Y  |
+| loadTable       | Y    | Y         | Y    | Y  |
+| updateTable     | Y    | Y         | Y    | Y  |
+| renameTable     | Y    | Y         | Y    | Y  |
+| tableExists     | Y    | Y         | Y    | Y  |
+
+#### View Spec V1
+
+| View Operation | Java | PyIceberg | Rust | Go |
+|----------------|------|-----------|------|----|
+| createView     | Y    | N         | N    | N  |
+| dropView       | Y    | N         | N    | N  |
+| listView       | Y    | N         | N    | N  |
+| viewExists     | Y    | N         | N    | N  |
+| replaceView    | Y    | N         | N    | N  |
+| renameView     | Y    | N         | N    | N  |
+
+#### Namespace Operations
+
+| Namespace Operation       | Java | PyIceberg | Rust | Go |
+|---------------------------|------|-----------|------|----|
+| listNamespaces            | Y    | N         | N    | N  |
+| createNamespace           | Y    | N         | N    | N  |
+| dropNamespace             | Y    | Y         | Y    | Y  |
+| namespaceExists           | Y    | N         | N    | N  |
+| updateNamespaceProperties | Y    | Y         | Y    | Y  |
+| loadNamespaceMetadata     | Y    | N         | N    | N  |
+
+### Glue Catalog
+
+#### Table Spec V1
+
+| Table Operation | Java | PyIceberg | Rust | Go |
+|-----------------|------|-----------|------|----|
+| listTable       | Y    | Y         | Y    | Y  |
+| createTable     | Y    | Y         | Y    | Y  |
+| dropTable       | Y    | Y         | Y    | Y  |
+| loadTable       | Y    | Y         | Y    | Y  |
+| updateTable     | Y    | Y         | Y    | Y  |
+| renameTable     | Y    | Y         | Y    | Y  |
+| tableExists     | Y    | Y         | Y    | Y  |
+
+#### Table Spec V2
+
+| Table Operation | Java | PyIceberg | Rust | Go |
+|-----------------|------|-----------|------|----|
+| listTable       | Y    | Y         | Y    | Y  |
+| createTable     | Y    | Y         | Y    | Y  |
+| dropTable       | Y    | Y         | Y    | Y  |
+| loadTable       | Y    | Y         | Y    | Y  |
+| updateTable     | Y    | Y         | Y    | Y  |
+| renameTable     | Y    | Y         | Y    | Y  |
+| tableExists     | Y    | Y         | Y    | Y  |
+
+#### View Spec V1
+
+| View Operation | Java | PyIceberg | Rust | Go |
+|----------------|------|-----------|------|----|
+| createView     | Y    | N         | N    | N  |
+| dropView       | Y    | N         | N    | N  |
+| listView       | Y    | N         | N    | N  |
+| viewExists     | Y    | N         | N    | N  |
+| replaceView    | Y    | N         | N    | N  |
+| renameView     | Y    | N         | N    | N  |
+
+#### Namespace Operations
+
+| Namespace Operation       | Java | PyIceberg | Rust | Go |
+|---------------------------|------|-----------|------|----|
+| listNamespaces            | Y    | N         | N    | N  |
+| createNamespace           | Y    | N         | N    | N  |
+| dropNamespace             | Y    | N         | N    | N  |
+| namespaceExists           | Y    | N         | N    | N  |
+| updateNamespaceProperties | Y    | Y         | Y    | Y  |
+| loadNamespaceMetadata     | Y    | N         | N    | N  |
+
+### Hive Metastore Catalog
+
+#### Table Spec V1
+
+| Table Operation | Java | PyIceberg | Rust | Go |
+|-----------------|------|-----------|------|----|
+| listTable       | Y    | Y         | Y    | Y  |
+| createTable     | Y    | Y         | Y    | Y  |
+| dropTable       | Y    | Y         | Y    | Y  |
+| loadTable       | Y    | Y         | Y    | Y  |
+| updateTable     | Y    | Y         | Y    | Y  |
+| renameTable     | Y    | Y         | Y    | Y  |
+| tableExists     | Y    | Y         | Y    | Y  |
+
+#### Table Spec V2
+
+| Table Operation | Java | PyIceberg | Rust | Go |
+|-----------------|------|-----------|------|----|
+| listTable       | Y    | Y         | Y    | Y  |
+| createTable     | Y    | Y         | Y    | Y  |
+| dropTable       | Y    | Y         | Y    | Y  |
+| loadTable       | Y    | Y         | Y    | Y  |
+| updateTable     | Y    | Y         | Y    | Y  |
+| renameTable     | Y    | Y         | Y    | Y  |
+| tableExists     | Y    | Y         | Y    | Y  |
+
+#### View Spec V1
+
+| View Operation | Java | PyIceberg | Rust | Go |
+|----------------|------|-----------|------|----|
+| createView     | Y    | N         | N    | N  |
+| dropView       | Y    | N         | N    | N  |
+| listView       | Y    | N         | N    | N  |
+| viewExists     | Y    | N         | N    | N  |
+| replaceView    | Y    | N         | N    | N  |
+| renameView     | Y    | N         | N    | N  |
+
+#### Namespace Operations
+
+| Namespace Operation       | Java | PyIceberg | Rust | Go |
+|---------------------------|------|-----------|------|----|
+| listNamespaces            | Y    | N         | N    | N  |
+| createNamespace           | Y    | N         | N    | N  |
+| dropNamespace             | Y    | N         | N    | N  |
+| namespaceExists           | Y    | N         | N    | N  |
+| updateNamespaceProperties | Y    | Y         | Y    | Y  |
+| loadNamespaceMetadata     | Y    | N         | N    | N  |
diff --git a/site/nav.yml b/site/nav.yml
@@ -47,10 +47,11 @@ nav:
     - View spec: view-spec.md
     - Puffin spec: puffin-spec.md
     - AES GCM Stream spec: gcm-stream-spec.md
+    - Implementation status: status.md
     - Multi-engine support: multi-engine-support.md
     - How to release: how-to-release.md
     - Terms: terms.md
-    - ASF: 
+    - ASF:
       - Sponsorship: https://www.apache.org/foundation/thanks.html
       - Events: https://www.apache.org/events/current-event.html
       - Privacy: https://privacy.apache.org/policies/privacy-policy-public.html