[CDAP-21070] Delegate all remaining methods of output committer

[itsankit-google]

regression caused by PR: #15717

Tested in CDAP Sandbox:

2024-10-21 17:48:27,763 - INFO  [Executor task launch worker for task 0.0 in stage 4.0 (TID 2):c.g.c.h.g.GoogleCloudStorageFileSystem@101] - Successfully repaired 'gs://28014f78-fef0-4e4c-9b29-ea3a8ccddc50/28014f78-fef0-4e4c-9b29-ea3a8ccddc50/input/bqsink_table_ankit-28014f78-fef0-4e4c-9b29-ea3a8ccddc50/_temporary/0/_temporary/' directory.
2024-10-21 17:48:27,806 - DEBUG [spark-listener-group-shared:i.c.c.a.r.s.SparkTransactionHandler@144] - Spark job ended: JobTransaction{jobId=2, stageIds=[2, 3, 4], transaction=Transaction{readPointer: 9223372036854775806, transactionId: 1729532852523000000, writePointer: 1729532852523000000, invalids: [], inProgress: [], firstShortInProgress: 9223372036854775807, type: LONG, checkpointWritePointers: [], visibilityLevel: SNAPSHOT}}
2024-10-21 17:48:28,659 - INFO  [Driver:c.g.c.h.g.GoogleCloudStorageFileSystem@101] - Successfully repaired 'gs://28014f78-fef0-4e4c-9b29-ea3a8ccddc50/28014f78-fef0-4e4c-9b29-ea3a8ccddc50/input/bqsink_table_ankit-28014f78-fef0-4e4c-9b29-ea3a8ccddc50/' directory.
2024-10-21 17:48:29,071 - INFO  [Driver:i.c.p.g.b.s.BigQueryOutputFormat@316] - Importing into table 'xxxxxxx.bqsink_table_ankit' from 1 paths; path[0] is 'gs://28014f78-fef0-4e4c-9b29-ea3a8ccddc50/28014f78-fef0-4e4c-9b29-ea3a8ccddc50/input/bqsink_table_ankit-28014f78-fef0-4e4c-9b29-ea3a8ccddc50/part-r-00000.avro'; awaitCompletion: true
2024-10-21 17:48:29,296 - INFO  [Driver:i.c.p.g.b.s.BigQueryOutputFormat@419] - Using schema '{"fields":[{"mode":"NULLABLE","name":"Name","type":"STRING"},{"mode":"NULLABLE","name":"StreetAddress","type":"STRING"},{"mode":"NULLABLE","name":"City","type":"STRING"},{"mode":"NULLABLE","name":"State","type":"STRING"}]}' for the load job config.
2024-10-21 17:48:36,099 - INFO  [Driver:i.c.p.g.b.s.BigQueryOutputFormat@454] - Imported into table 'xxxxxxxx.bqsink_table_ankit' from 1 paths; path[0] is 'gs://28014f78-fef0-4e4c-9b29-ea3a8ccddc50/28014f78-fef0-4e4c-9b29-ea3a8ccddc50/input/bqsink_table_ankit-28014f78-fef0-4e4c-9b29-ea3a8ccddc50/part-r-00000.avro'
2024-10-21 17:48:36,595 - INFO  [Driver:c.g.c.h.g.GoogleCloudStorageFileSystem@101] - Successfully repaired 'gs://28014f78-fef0-4e4c-9b29-ea3a8ccddc50/28014f78-fef0-4e4c-9b29-ea3a8ccddc50/input/' directory.
2024-10-21 17:48:37,498 - DEBUG [Driver:i.c.c.s.a.AllowlistAccessEnforcer@62] - Skipping authorization enforcement for user 'yarn'
2024-10-21 17:48:37,533 - DEBUG [Driver:i.c.c.s.a.AllowlistAccessEnforcer@62] - Skipping authorization enforcement for user 'yarn'
2024-10-21 17:48:37,566 - INFO  [Driver:i.c.c.a.r.s.SparkMainWrapper$@95] - Spark main returned class io.cdap.cdap.etl.spark.batch.BatchSparkPipelineDriver
2024-10-21 17:48:37,980 - DEBUG [SparkDriverService:i.c.c.a.r.s.d.SparkDriverService@160] - Heartbeat loop completed with state 'COMPLETED' and terminate timestamp 'null'
2024-10-21 17:48:37,982 - INFO  [phase-1-spark-exec-service-executor-55:i.c.c.a.r.s.d.SparkExecutionService@203] - Spark program completed phase-1 6486cf91-8fd4-11ef-831f-42010a0a013a
2024-10-21 17:48:37,990 - DEBUG [SparkDriverService:i.c.c.a.r.s.d.SparkDriverService@178] - SparkDriverService completed with program completion state 'COMPLETED'
2024-10-21 17:48:38,083 - INFO  [SparkDriverService:o.s.j.s.AbstractConnector@383] - Stopped Spark@7a26638f{HTTP/1.1, (http/1.1)}{0.0.0.0:0}
2024-10-21 17:48:38,241 - INFO  [CoarseGrainedExecutorBackend-stop-executor:c.g.c.d.DataprocSparkPlugin@111] - Shutting down executor plugin. metrics=null
2024-10-21 17:48:38,245 - INFO  [CoarseGrainedExecutorBackend-stop-executor:c.g.c.d.DataprocSparkPlugin@111] - Shutting down executor plugin. metrics=null
2024-10-21 17:48:38,266 - INFO  [SparkDriverService:c.g.c.d.DataprocSparkPlugin@41] - Shutting down driver plugin. metrics=null
2024-10-21 17:48:38,288 - DEBUG [main:i.c.c.l.a.LogAppenderInitializer@137] - Stopping log appender TMSLogAppender
2024-10-21 17:48:38,348 - DEBUG [main:i.c.c.l.a.LogAppenderInitializer@137] - Stopping log appender TMSLogAppender
2024-10-21 17:48:38,777 - DEBUG [SparkDriverService:i.c.c.a.r.s.SparkRuntimeEnv$@354] - Shutting down Server and ThreadPool used by Spark org.apache.spark.SparkContext@15609f7d
2024-10-21 17:48:38,789 - INFO  [SparkDriverHttpService STOPPING:i.c.h.NettyHttpService@258] - Stopping HTTP Service phase-1-http-service
2024-10-21 17:48:38,826 - DEBUG [SparkDriverService:i.c.c.i.a.r.d.r.RemoteExecutionDiscoveryService@148] - Update discoverable runtime with address r-ankit-cdf-latest-cdf-test-322418-dot-r.datafusion-dev.googleusercontent.com/74.125.26.132:443 and payload https://
2024-10-21 17:48:39,709 - DEBUG [SparkDriverService:i.c.c.a.r.s.SparkRuntimeUtils@405] - Uploaded event logs file /hadoop/yarn/nm-local-dir/usercache/yarn/appcache/application_1729532529888_0002/container_1729532529888_0002_01_000001/tmp/spark-events15144915353023861972/1729532843917.lz4 for program run program_run:default.ProberCampaignPipeline.aa1672c7-8fc2-11ef-a086-92a297024e3f.spark.phase-1.6486cf91-8fd4-11ef-831f-42010a0a013a
2024-10-21 17:48:39,725 - DEBUG [main:i.c.c.l.a.LogAppenderInitializer@137] - Stopping log appender TMSLogAppender
2024-10-21 17:48:40,868 - DEBUG [spark-submitter-phase-1-6486cf91-8fd4-11ef-831f-42010a0a013a:i.c.c.a.r.s.s.AbstractSparkSubmitter@203] - SparkSubmit returned for program:default.ProberCampaignPipeline.aa1672c7-8fc2-11ef-a086-92a297024e3f.spark.phase-1 6486cf91-8fd4-11ef-831f-42010a0a013a
2024-10-21 17:48:40,875 - INFO  [SparkExecutionService STOPPING:i.c.c.a.r.s.d.SparkExecutionService@141] - Shutting down SparkExecutionService
2024-10-21 17:48:40,887 - INFO  [SparkExecutionService STOPPING:i.c.c.a.r.s.d.SparkExecutionService@151] - Shutting down HTTP server
2024-10-21 17:48:40,896 - INFO  [SparkExecutionService STOPPING:i.c.h.NettyHttpService@258] - Stopping HTTP Service phase-1-spark-exec-service
2024-10-21 17:48:43,958 - INFO  [SparkRunner-phase-1:i.c.p.g.b.s.BigQuerySink@218] - Job JobId{project=xxxxxx, job=4d19497e-46f9-4cee-92b6-db820672f3e2, location=US} affected 11 rows
2024-10-21 17:48:43,960 - INFO  [SparkRunner-phase-1:i.c.p.g.b.s.BigQuerySink@281] - Job JobId{project=xxxxxx, job=4d19497e-46f9-4cee-92b6-db820672f3e2, location=US} loaded 492 bytes
2024-10-21 17:48:43,964 - DEBUG [SparkRunner-phase-1:i.c.c.a.r.s.SparkRuntimeService@1019] - Running Spark shutdown hook org.apache.spark.util.SparkShutdownHookManager$$anon$2@5ed502fd
2024-10-21 17:48:44,011 - DEBUG [SparkRunner-phase-1:i.c.c.a.r.s.SparkRuntimeService@480] - Spark program completed: SparkRuntimeContext{id=program:default.ProberCampaignPipeline.aa1672c7-8fc2-11ef-a086-92a297024e3f.spark.phase-1, runId=6486cf91-8fd4-11ef-831f-42010a0a013a}
2024-10-21 17:48:44,108 - INFO  [action-phase-1-0:i.c.c.i.a.r.w.WorkflowDriver@352] - Spark Program 'phase-1' in workflow completed

[sonarcloud]

Quality Gate failed

Failed conditions
26.1% Coverage on New Code (required ≥ 80%)

See analysis details on SonarCloud