support UPSERTs and UPDATEs in WriteTableQuery

[qrort]

No description provided.

[github-actions]

📝 Test results

ydbcp

26 tests (26 passed)

%%{init: {"theme":"base","themeVariables":{"fontFamily":"monospace","pieSectionTextSize":"24px","darkMode":true,"pie1":"#2da44e"}}}%%
pie showData
"Passed" : 26

Loading

📦 Package	🟢 Passed	🔴 Failed	🟡 Skipped	⏳ Duration
🟡 ydbcp/cmd/ydbcp	0	0	0	0ms
🧪 Tests (none) 🖨️ Output (none)
🟡 ydbcp/internal/config	0	0	0	0ms
🧪 Tests (none) 🖨️ Output (none)
🟡 ydbcp/internal/connectors/client	0	0	0	0ms
🧪 Tests (none) 🖨️ Output (none)
🟡 ydbcp/internal/connectors/db	0	0	0	0ms
🧪 Tests (none) 🖨️ Output (none)
🟢 ydbcp/internal/connectors/db/yql/queries	4	0	0	25ms
🧪 Tests 🟢TestQueryBuilder_CreateCreate 🟢TestQueryBuilder_Read 🟢TestQueryBuilder_UpdateCreate 🟢TestQueryBuilder_UpdateUpdate 🖨️ Output === RUN TestQueryBuilder_Read 2024-07-29T14:43:43.549Z DEBUG queries/read.go:145 read query {"yql": "DECLARE $param0 AS String;\nDECLARE $param1 AS String;\nDECLARE $param2 AS String;\nDECLARE $param3 AS String;\nSELECT column1, column2, column3 FROM table1 WHERE (column1 = $param0 OR column1 = $param1) AND (column2 = $param2 OR column2 = $param3)"} --- PASS: TestQueryBuilder_Read (0.00s) === RUN TestQueryBuilder_UpdateUpdate 2024-07-29T14:43:43.555Z DEBUG queries/write.go:252 write query {"yql": "DECLARE $id_0 AS Uuid;\nDECLARE $status_0 AS String;\nUPDATE Backups SET status = $status_0 WHERE id = $id_0;\nDECLARE $id_1 AS Uuid;\nDECLARE $status_1 AS String;\nDECLARE $message_1 AS String;\nUPDATE Operations SET status = $status_1, message = $message_1 WHERE id = $id_1"} --- PASS: TestQueryBuilder_UpdateUpdate (0.00s) === RUN TestQueryBuilder_CreateCreate 2024-07-29T14:43:43.557Z DEBUG queries/write.go:252 write query {"yql": "DECLARE $id_0 AS Uuid;\nDECLARE $container_id_0 AS String;\nDECLARE $database_0 AS String;\nDECLARE $initiated_0 AS String;\nDECLARE $s3_endpoint_0 AS String;\nDECLARE $s3_region_0 AS String;\nDECLARE $s3_bucket_0 AS String;\nDECLARE $s3_path_prefix_0 AS String;\nDECLARE $status_0 AS String;\nUPSERT INTO Backups (id, container_id, database, initiated, s3_endpoint, s3_region, s3_bucket, s3_path_prefix, status) VALUES ($id_0, $container_id_0, $database_0, $initiated_0, $s3_endpoint_0, $s3_region_0, $s3_bucket_0, $s3_path_prefix_0, $status_0);\nDECLARE $id_1 AS Uuid;\nDECLARE $type_1 AS String;\nDECLARE $status_1 AS String;\nDECLARE $container_id_1 AS String;\nDECLARE $database_1 AS String;\nDECLARE $backup_id_1 AS Uuid;\nDECLARE $initiated_1 AS String;\nDECLARE $created_at_1 AS Timestamp;\nDECLARE $operation_id_1 AS String;\nUPSERT INTO Operations (id, type, status, container_id, database, backup_id, initiated, created_at, operation_id) VALUES ($id_1, $type_1, $status_1, $container_id_1, $database_1, $backup_id_1, $initiated_1, $created_at_1, $operation_id_1)"} --- PASS: TestQueryBuilder_CreateCreate (0.00s) === RUN TestQueryBuilder_UpdateCreate 2024-07-29T14:43:43.558Z DEBUG queries/write.go:252 write query {"yql": "DECLARE $id_0 AS Uuid;\nDECLARE $status_0 AS String;\nUPDATE Backups SET status = $status_0 WHERE id = $id_0;\nDECLARE $id_1 AS Uuid;\nDECLARE $type_1 AS String;\nDECLARE $status_1 AS String;\nDECLARE $container_id_1 AS String;\nDECLARE $database_1 AS String;\nDECLARE $backup_id_1 AS Uuid;\nDECLARE $initiated_1 AS String;\nDECLARE $created_at_1 AS Timestamp;\nDECLARE $operation_id_1 AS String;\nUPSERT INTO Operations (id, type, status, container_id, database, backup_id, initiated, created_at, operation_id) VALUES ($id_1, $type_1, $status_1, $container_id_1, $database_1, $backup_id_1, $initiated_1, $created_at_1, $operation_id_1)"} --- PASS: TestQueryBuilder_UpdateCreate (0.00s)
🟢 ydbcp/internal/handlers	20	0	0	16ms
🧪 Tests 🟢TestRBOperationHandlerCancellingOperationCancelled 🟢TestRBOperationHandlerCancellingOperationCompletedSuccessfully 🟢TestRBOperationHandlerCancellingOperationInProgress 🟢TestRBOperationHandlerDeadlineExceededForCancellingOperation 🟢TestRBOperationHandlerDeadlineExceededForPendingOperation 🟢TestRBOperationHandlerInvalidOperationResponse 🟢TestRBOperationHandlerPendingOperationCancelled 🟢TestRBOperationHandlerPendingOperationCompletedSuccessfully 🟢TestRBOperationHandlerPendingOperationInProgress 🟢TestRBOperationHandlerRetriableErrorForPendingOperation 🟢TestTBOperationHandlerCancellingOperationCancelled 🟢TestTBOperationHandlerCancellingOperationCompletedSuccessfully 🟢TestTBOperationHandlerCancellingOperationInProgress 🟢TestTBOperationHandlerDeadlineExceededForCancellingOperation 🟢TestTBOperationHandlerDeadlineExceededForPendingOperation 🟢TestTBOperationHandlerInvalidOperationResponse 🟢TestTBOperationHandlerPendingOperationCancelled 🟢TestTBOperationHandlerPendingOperationCompletedSuccessfully 🟢TestTBOperationHandlerPendingOperationInProgress 🟢TestTBOperationHandlerRetriableErrorForPendingOperation 🖨️ Output === RUN TestRBOperationHandlerInvalidOperationResponse 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:30 received operation {"id": "22525435-3da0-445f-9e24-3b58a9e53a2f", "type": "RB", "state": "PENDING", "message": ""} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "22525435-3da0-445f-9e24-3b58a9e53a2f", "type": "RB", "ydb_operation_id": "1"} --- PASS: TestRBOperationHandlerInvalidOperationResponse (0.00s) === RUN TestRBOperationHandlerDeadlineExceededForPendingOperation 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:30 received operation {"id": "5eb52a72-427f-4c11-8732-4a6a5cf2271b", "type": "RB", "state": "PENDING", "message": ""} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "5eb52a72-427f-4c11-8732-4a6a5cf2271b", "type": "RB", "ydb_operation_id": "1"} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:98 cancelling operation {"reason": "TTL", "id": "5eb52a72-427f-4c11-8732-4a6a5cf2271b", "type": "RB", "ydb_operation_id": "1"} --- PASS: TestRBOperationHandlerDeadlineExceededForPendingOperation (0.00s) === RUN TestRBOperationHandlerPendingOperationInProgress 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:30 received operation {"id": "1bd49b30-24e6-4182-9f4c-a41ea06bc793", "type": "RB", "state": "PENDING", "message": ""} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "1bd49b30-24e6-4182-9f4c-a41ea06bc793", "type": "RB", "ydb_operation_id": "1"} --- PASS: TestRBOperationHandlerPendingOperationInProgress (0.00s) === RUN TestRBOperationHandlerPendingOperationCompletedSuccessfully 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:30 received operation {"id": "ffb30eda-66e5-42ce-868d-31378d58f6fa", "type": "RB", "state": "PENDING", "message": ""} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "ffb30eda-66e5-42ce-868d-31378d58f6fa", "type": "RB", "ydb_operation_id": "1"} 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:119 forgetting operation {"id": "ffb30eda-66e5-42ce-868d-31378d58f6fa", "type": "RB", "ydb_operation_id": "1"} --- PASS: TestRBOperationHandlerPendingOperationCompletedSuccessfully (0.00s) === RUN TestRBOperationHandlerPendingOperationCancelled 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:30 received operation {"id": "93914a82-e30f-4760-a58e-5d10c9f62182", "type": "RB", "state": "PENDING", "message": ""} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "93914a82-e30f-4760-a58e-5d10c9f62182", "type": "RB", "ydb_operation_id": "1"} 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:119 forgetting operation {"id": "93914a82-e30f-4760-a58e-5d10c9f62182", "type": "RB", "ydb_operation_id": "1"} --- PASS: TestRBOperationHandlerPendingOperationCancelled (0.00s) === RUN TestRBOperationHandlerDeadlineExceededForCancellingOperation 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:30 received operation {"id": "033a863e-d8c8-406f-b654-4cb9311c6b44", "type": "RB", "state": "CANCELLING", "message": ""} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "033a863e-d8c8-406f-b654-4cb9311c6b44", "type": "RB", "ydb_operation_id": "1"} --- PASS: TestRBOperationHandlerDeadlineExceededForCancellingOperation (0.00s) === RUN TestRBOperationHandlerCancellingOperationInProgress 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:30 received operation {"id": "7a00404f-f2fb-42be-a9f2-9c318ee7acad", "type": "RB", "state": "CANCELLING", "message": ""} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "7a00404f-f2fb-42be-a9f2-9c318ee7acad", "type": "RB", "ydb_operation_id": "1"} --- PASS: TestRBOperationHandlerCancellingOperationInProgress (0.00s) === RUN TestRBOperationHandlerCancellingOperationCompletedSuccessfully 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:30 received operation {"id": "31dca5a2-5c49-4552-b3b0-6905a7624317", "type": "RB", "state": "CANCELLING", "message": ""} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "31dca5a2-5c49-4552-b3b0-6905a7624317", "type": "RB", "ydb_operation_id": "1"} 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:119 forgetting operation {"id": "31dca5a2-5c49-4552-b3b0-6905a7624317", "type": "RB", "ydb_operation_id": "1"} --- PASS: TestRBOperationHandlerCancellingOperationCompletedSuccessfully (0.00s) === RUN TestRBOperationHandlerCancellingOperationCancelled 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:30 received operation {"id": "7b141273-102c-4ec1-a1ef-9caad1283682", "type": "RB", "state": "CANCELLING", "message": ""} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "7b141273-102c-4ec1-a1ef-9caad1283682", "type": "RB", "ydb_operation_id": "1"} 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:119 forgetting operation {"id": "7b141273-102c-4ec1-a1ef-9caad1283682", "type": "RB", "ydb_operation_id": "1"} --- PASS: TestRBOperationHandlerCancellingOperationCancelled (0.00s) === RUN TestRBOperationHandlerRetriableErrorForPendingOperation 2024-07-29T14:43:44.247Z INFO handlers/restore_backup.go:30 received operation {"id": "43a51ce1-73cb-4c9c-bf0e-ef9be528fe1a", "type": "RB", "state": "PENDING", "message": ""} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "43a51ce1-73cb-4c9c-bf0e-ef9be528fe1a", "type": "RB", "ydb_operation_id": "1"} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:68 received retriable error {"id": "43a51ce1-73cb-4c9c-bf0e-ef9be528fe1a", "type": "RB", "ydb_operation_id": "1"} --- PASS: TestRBOperationHandlerRetriableErrorForPendingOperation (0.00s) === RUN TestTBOperationHandlerInvalidOperationResponse 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "3696353e-20fa-446d-a829-17f3336cd0ae", "type": "TB", "ydb_operation_id": "1"} --- PASS: TestTBOperationHandlerInvalidOperationResponse (0.00s) === RUN TestTBOperationHandlerDeadlineExceededForPendingOperation 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "8262a106-8f66-476e-94a7-e37856dc22c2", "type": "TB", "ydb_operation_id": "1"} 2024-07-29T14:43:44.247Z INFO handlers/utils.go:98 cancelling operation {"reason": "TTL", "id": "8262a106-8f66-476e-94a7-e37856dc22c2", "type": "TB", "ydb_operation_id": "1"} --- PASS: TestTBOperationHandlerDeadlineExceededForPendingOperation (0.00s) === RUN TestTBOperationHandlerPendingOperationInProgress 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "06bec0f0-503a-4b54-bc94-2b0c85b62dfe", "type": "TB", "ydb_operation_id": "1"} --- PASS: TestTBOperationHandlerPendingOperationInProgress (0.00s) === RUN TestTBOperationHandlerPendingOperationCompletedSuccessfully 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "874ad676-8e43-4085-a22d-3c829d5b0033", "type": "TB", "ydb_operation_id": "1"} --- PASS: TestTBOperationHandlerPendingOperationCompletedSuccessfully (0.00s) === RUN TestTBOperationHandlerPendingOperationCancelled 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "58ee4599-ce25-4760-9203-3e07eda0e248", "type": "TB", "ydb_operation_id": "1"} --- PASS: TestTBOperationHandlerPendingOperationCancelled (0.00s) === RUN TestTBOperationHandlerDeadlineExceededForCancellingOperation 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "9d1b739b-8989-4b29-b995-f7e0bdfcbc4b", "type": "TB", "ydb_operation_id": "1"} --- PASS: TestTBOperationHandlerDeadlineExceededForCancellingOperation (0.00s) === RUN TestTBOperationHandlerCancellingOperationInProgress 2024-07-29T14:43:44.247Z INFO handlers/utils.go:45 getting operation status {"id": "d9d3ecb9-2f5a-49b6-bbfb-dd90659a7c6c", "type": "TB", "ydb_operation_id": "1"} --- PASS: TestTBOperationHandlerCancellingOperationInProgress (0.00s) === RUN TestTBOperationHandlerCancellingOperationCompletedSuccessfully 2024-07-29T14:43:44.248Z INFO handlers/utils.go:45 getting operation status {"id": "ef8adfc4-a066-4724-8103-58e16aeadc88", "type": "TB", "ydb_operation_id": "1"} --- PASS: TestTBOperationHandlerCancellingOperationCompletedSuccessfully (0.00s) === RUN TestTBOperationHandlerCancellingOperationCancelled 2024-07-29T14:43:44.248Z INFO handlers/utils.go:45 getting operation status {"id": "f2d60f3d-f1c2-47e8-bd2f-4fc273e0f313", "type": "TB", "ydb_operation_id": "1"} --- PASS: TestTBOperationHandlerCancellingOperationCancelled (0.00s) === RUN TestTBOperationHandlerRetriableErrorForPendingOperation 2024-07-29T14:43:44.248Z INFO handlers/utils.go:45 getting operation status {"id": "22e64c96-c1c3-4490-bf59-b661794c7921", "type": "TB", "ydb_operation_id": "1"} 2024-07-29T14:43:44.248Z INFO handlers/utils.go:68 received retriable error {"id": "22e64c96-c1c3-4490-bf59-b661794c7921", "type": "TB", "ydb_operation_id": "1"} --- PASS: TestTBOperationHandlerRetriableErrorForPendingOperation (0.00s)
🟢 ydbcp/internal/processor	2	0	0	8ms
🧪 Tests 🟢TestOperationHandlerRegistry 🟢TestProcessor 🖨️ Output === RUN TestProcessor 2024-07-29T14:43:44.246Z DEBUG processor/processor.go:76 Operation Processor started {"period": "10s"} 2024-07-29T14:43:44.246Z DEBUG processor/processor.go:108 operation already running {"operation": "Operation, id db8d7497-8873-494b-9b47-8b0f4b05993a, type TB, state PENDING"} 2024-07-29T14:43:44.246Z DEBUG processor/processor.go:108 operation already running {"operation": "Operation, id db8d7497-8873-494b-9b47-8b0f4b05993a, type TB, state PENDING"} 2024-07-29T14:43:44.246Z DEBUG processor/processor.go:118 start operation handler {"operation": "Operation, id db8d7497-8873-494b-9b47-8b0f4b05993a, type TB, state PENDING"} 2024-07-29T14:43:44.246Z DEBUG processor/processor_test.go:41 TB handler called for operation {"operation": "Operation, id db8d7497-8873-494b-9b47-8b0f4b05993a, type TB, state PENDING"} 2024-07-29T14:43:44.246Z DEBUG processor/processor.go:137 operation handler is finished {"operationID": "db8d7497-8873-494b-9b47-8b0f4b05993a"} 2024-07-29T14:43:44.246Z DEBUG processor/processor.go:83 Operation Processor stopped --- PASS: TestProcessor (0.00s) === RUN TestOperationHandlerRegistry --- PASS: TestOperationHandlerRegistry (0.00s)
🟡 ydbcp/internal/types	0	0	0	0ms
🧪 Tests (none) 🖨️ Output (none)
🟡 ydbcp/internal/util/ticker	0	0	0	0ms
🧪 Tests (none) 🖨️ Output (none)
🟡 ydbcp/internal/util/timer	0	0	0	0ms
🧪 Tests (none) 🖨️ Output (none)
🟡 ydbcp/internal/util/xlog	0	0	0	0ms
🧪 Tests (none) 🖨️ Output (none)
🟡 ydbcp/pkg/proto/ydbcp/v1alpha1	0	0	0	0ms
🧪 Tests (none) 🖨️ Output (none)