This repositry provides code samples and tutorials for Google Cloud services.
-
Pubsub to BigQuery A demo of ingesting Pubsub messages to BigQuery using Pubsub BigQuery subscription.
-
Kafka to BigQuery A demo of ingesting data from Kafka to BigQuery using dataflow template and JavaScript UDF.
-
Kyuubi integration with Dataproc A step by step guide of depolying Kyuubi and integrate with Dataproc.
-
Dataproc on GKE quick start A step by step guide of deploying Dataproc on GKE with customized IAM and Spark properties.
-
Hive metastore on Dataproc A step by step guide of deploying a 3 master Dataproc cluster to serve as Hive metastore.
-
Query Hive table on Google Cloud Storage (GCS) using StarRocks A step by step guide of how to connect to Hive metastore and query data on Google Cloud Storage (GCS) using StarRocks.
-
Support Streaming data to GCS on Flink 1.11 This doc describes how to build Flink Google Storage plugin on Flink 1.11 to support streaming writes to Google Storage.
-
Streaming data to GCS with Flink using iceberg format A step by step guide of how to stream data to Google Cloud Storage (GCS) with Flink using iceberg format.