Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

write source_paths_to_exclude to db #44

Merged
merged 1 commit into from
Aug 21, 2024
Merged

write source_paths_to_exclude to db #44

merged 1 commit into from
Aug 21, 2024

Conversation

qrort
Copy link
Collaborator

@qrort qrort commented Aug 21, 2024

No description provided.

Copy link

📝 Test results

ydbcp

28 tests (28 passed)
%%{init: {"theme":"base","themeVariables":{"fontFamily":"monospace","pieSectionTextSize":"24px","darkMode":true,"pie1":"#2da44e"}}}%%
pie showData
"Passed" : 28

Loading
📦 Package🟢 Passed🔴 Failed🟡 Skipped⏳ Duration
🟡 ydbcp/cmd/ydbcp0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟡 ydbcp/internal/auth0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟡 ydbcp/internal/config0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟡 ydbcp/internal/connectors/client0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟡 ydbcp/internal/connectors/db0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟢 ydbcp/internal/connectors/db/yql/queries4007ms
🧪 Tests
  • 🟢TestQueryBuilder_CreateCreate
  • 🟢TestQueryBuilder_Read
  • 🟢TestQueryBuilder_UpdateCreate
  • 🟢TestQueryBuilder_UpdateUpdate
🖨️ Output
=== RUN   TestQueryBuilder_Read
2024-08-21T16:13:00.701Z	DEBUG	queries/read.go:135	read query	{"yql": "SELECT column1, column2, column3 FROM table1 WHERE (column1 = $param0 OR column1 = $param1) AND (column2 = $param2 OR column2 = $param3)"}
--- PASS: TestQueryBuilder_Read (0.00s)
=== RUN   TestQueryBuilder_UpdateUpdate
2024-08-21T16:13:00.701Z	DEBUG	queries/write.go:270	write query	{"yql": "UPDATE Backups SET status = $status_0 WHERE id = $id_0;\nUPDATE Operations SET status = $status_1, message = $message_1 WHERE id = $id_1"}
--- PASS: TestQueryBuilder_UpdateUpdate (0.00s)
=== RUN   TestQueryBuilder_CreateCreate
2024-08-21T16:13:00.701Z	DEBUG	queries/write.go:270	write query	{"yql": "UPSERT INTO Backups (id, container_id, database, endpoint, initiated, s3_endpoint, s3_region, s3_bucket, s3_path_prefix, status, message) VALUES ($id_0, $container_id_0, $database_0, $endpoint_0, $initiated_0, $s3_endpoint_0, $s3_region_0, $s3_bucket_0, $s3_path_prefix_0, $status_0, $message_0);\nUPSERT INTO Operations (id, type, status, initiated, created_at, container_id, database, endpoint, backup_id, operation_id, message) VALUES ($id_1, $type_1, $status_1, $initiated_1, $created_at_1, $container_id_1, $database_1, $endpoint_1, $backup_id_1, $operation_id_1, $message_1)"}
--- PASS: TestQueryBuilder_CreateCreate (0.00s)
=== RUN   TestQueryBuilder_UpdateCreate
2024-08-21T16:13:00.701Z	DEBUG	queries/write.go:270	write query	{"yql": "UPDATE Backups SET status = $status_0 WHERE id = $id_0;\nUPSERT INTO Operations (id, type, status, initiated, created_at, container_id, database, endpoint, backup_id, operation_id, message, paths, paths_to_exclude) VALUES ($id_1, $type_1, $status_1, $initiated_1, $created_at_1, $container_id_1, $database_1, $endpoint_1, $backup_id_1, $operation_id_1, $message_1, $paths_1, $paths_to_exclude_1)"}
--- PASS: TestQueryBuilder_UpdateCreate (0.00s)
🟢 ydbcp/internal/handlers200028ms
🧪 Tests
  • 🟢TestRBOperationHandlerCancellingOperationCancelled
  • 🟢TestRBOperationHandlerCancellingOperationCompletedSuccessfully
  • 🟢TestRBOperationHandlerCancellingOperationInProgress
  • 🟢TestRBOperationHandlerDeadlineExceededForCancellingOperation
  • 🟢TestRBOperationHandlerDeadlineExceededForPendingOperation
  • 🟢TestRBOperationHandlerInvalidOperationResponse
  • 🟢TestRBOperationHandlerPendingOperationCancelled
  • 🟢TestRBOperationHandlerPendingOperationCompletedSuccessfully
  • 🟢TestRBOperationHandlerPendingOperationInProgress
  • 🟢TestRBOperationHandlerRetriableErrorForPendingOperation
  • 🟢TestTBOperationHandlerCancellingOperationCancelled
  • 🟢TestTBOperationHandlerCancellingOperationCompletedSuccessfully
  • 🟢TestTBOperationHandlerCancellingOperationInProgress
  • 🟢TestTBOperationHandlerDeadlineExceededForCancellingOperation
  • 🟢TestTBOperationHandlerDeadlineExceededForPendingOperation
  • 🟢TestTBOperationHandlerInvalidOperationResponse
  • 🟢TestTBOperationHandlerPendingOperationCancelled
  • 🟢TestTBOperationHandlerPendingOperationCompletedSuccessfully
  • 🟢TestTBOperationHandlerPendingOperationInProgress
  • 🟢TestTBOperationHandlerRetriableErrorForPendingOperation
🖨️ Output
=== RUN   TestRBOperationHandlerInvalidOperationResponse
2024-08-21T16:13:01.235Z	INFO	handlers/restore_backup.go:33	received operation	{"id": "393a0db6-3644-4223-b281-5e3bff58ca26", "type": "RB", "state": "PENDING", "message": ""}
2024-08-21T16:13:01.235Z	INFO	handlers/utils.go:48	getting operation status	{"id": "393a0db6-3644-4223-b281-5e3bff58ca26", "type": "RB", "ydb_operation_id": "1"}
--- PASS: TestRBOperationHandlerInvalidOperationResponse (0.00s)
=== RUN   TestRBOperationHandlerDeadlineExceededForPendingOperation
2024-08-21T16:13:01.235Z	INFO	handlers/restore_backup.go:33	received operation	{"id": "400dad73-18ab-4514-bd65-17cc5bffbea8", "type": "RB", "state": "PENDING", "message": ""}
2024-08-21T16:13:01.235Z	INFO	handlers/utils.go:48	getting operation status	{"id": "400dad73-18ab-4514-bd65-17cc5bffbea8", "type": "RB", "ydb_operation_id": "1"}
2024-08-21T16:13:01.235Z	INFO	handlers/utils.go:101	cancelling operation	{"reason": "TTL", "id": "400dad73-18ab-4514-bd65-17cc5bffbea8", "type": "RB", "ydb_operation_id": "1"}
--- PASS: TestRBOperationHandlerDeadlineExceededForPendingOperation (0.00s)
=== RUN   TestRBOperationHandlerPendingOperationInProgress
2024-08-21T16:13:01.235Z	INFO	handlers/restore_backup.go:33	received operation	{"id": "e0e13617-f25a-476f-8ed0-2d96162ec994", "type": "RB", "state": "PENDING", "message": ""}
2024-08-21T16:13:01.235Z	INFO	handlers/utils.go:48	getting operation status	{"id": "e0e13617-f25a-476f-8ed0-2d96162ec994", "type": "RB", "ydb_operation_id": "1"}
--- PASS: TestRBOperationHandlerPendingOperationInProgress (0.00s)
=== RUN   TestRBOperationHandlerPendingOperationCompletedSuccessfully
2024-08-21T16:13:01.235Z	INFO	handlers/restore_backup.go:33	received operation	{"id": "75faee36-11cc-4feb-8b1d-f6585910c1dc", "type": "RB", "state": "PENDING", "message": ""}
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "75faee36-11cc-4feb-8b1d-f6585910c1dc", "type": "RB", "ydb_operation_id": "1"}
2024-08-21T16:13:01.236Z	INFO	handlers/restore_backup.go:124	forgetting operation	{"id": "75faee36-11cc-4feb-8b1d-f6585910c1dc", "type": "RB", "ydb_operation_id": "1"}
--- PASS: TestRBOperationHandlerPendingOperationCompletedSuccessfully (0.00s)
=== RUN   TestRBOperationHandlerPendingOperationCancelled
2024-08-21T16:13:01.236Z	INFO	handlers/restore_backup.go:33	received operation	{"id": "b0a72a03-134d-4d57-81c5-d5d3b21ea574", "type": "RB", "state": "PENDING", "message": ""}
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "b0a72a03-134d-4d57-81c5-d5d3b21ea574", "type": "RB", "ydb_operation_id": "1"}
2024-08-21T16:13:01.236Z	INFO	handlers/restore_backup.go:124	forgetting operation	{"id": "b0a72a03-134d-4d57-81c5-d5d3b21ea574", "type": "RB", "ydb_operation_id": "1"}
--- PASS: TestRBOperationHandlerPendingOperationCancelled (0.00s)
=== RUN   TestRBOperationHandlerDeadlineExceededForCancellingOperation
2024-08-21T16:13:01.236Z	INFO	handlers/restore_backup.go:33	received operation	{"id": "da48747e-2c5b-4665-b6ee-579cfac651d7", "type": "RB", "state": "CANCELLING", "message": ""}
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "da48747e-2c5b-4665-b6ee-579cfac651d7", "type": "RB", "ydb_operation_id": "1"}
--- PASS: TestRBOperationHandlerDeadlineExceededForCancellingOperation (0.00s)
=== RUN   TestRBOperationHandlerCancellingOperationInProgress
2024-08-21T16:13:01.236Z	INFO	handlers/restore_backup.go:33	received operation	{"id": "1660253c-bb1e-48bb-bae0-44ae24252523", "type": "RB", "state": "CANCELLING", "message": ""}
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "1660253c-bb1e-48bb-bae0-44ae24252523", "type": "RB", "ydb_operation_id": "1"}
--- PASS: TestRBOperationHandlerCancellingOperationInProgress (0.00s)
=== RUN   TestRBOperationHandlerCancellingOperationCompletedSuccessfully
2024-08-21T16:13:01.236Z	INFO	handlers/restore_backup.go:33	received operation	{"id": "79b27404-66d7-47aa-9d37-bad6eda46f60", "type": "RB", "state": "CANCELLING", "message": ""}
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "79b27404-66d7-47aa-9d37-bad6eda46f60", "type": "RB", "ydb_operation_id": "1"}
2024-08-21T16:13:01.236Z	INFO	handlers/restore_backup.go:124	forgetting operation	{"id": "79b27404-66d7-47aa-9d37-bad6eda46f60", "type": "RB", "ydb_operation_id": "1"}
--- PASS: TestRBOperationHandlerCancellingOperationCompletedSuccessfully (0.00s)
=== RUN   TestRBOperationHandlerCancellingOperationCancelled
2024-08-21T16:13:01.236Z	INFO	handlers/restore_backup.go:33	received operation	{"id": "5a611431-3e88-4bc3-9937-80c4fde67a16", "type": "RB", "state": "CANCELLING", "message": ""}
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "5a611431-3e88-4bc3-9937-80c4fde67a16", "type": "RB", "ydb_operation_id": "1"}
2024-08-21T16:13:01.236Z	INFO	handlers/restore_backup.go:124	forgetting operation	{"id": "5a611431-3e88-4bc3-9937-80c4fde67a16", "type": "RB", "ydb_operation_id": "1"}
--- PASS: TestRBOperationHandlerCancellingOperationCancelled (0.00s)
=== RUN   TestRBOperationHandlerRetriableErrorForPendingOperation
2024-08-21T16:13:01.236Z	INFO	handlers/restore_backup.go:33	received operation	{"id": "7b3b70eb-8602-4870-a3c0-94bf9f8c10da", "type": "RB", "state": "PENDING", "message": ""}
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "7b3b70eb-8602-4870-a3c0-94bf9f8c10da", "type": "RB", "ydb_operation_id": "1"}
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:71	received retriable error	{"id": "7b3b70eb-8602-4870-a3c0-94bf9f8c10da", "type": "RB", "ydb_operation_id": "1"}
--- PASS: TestRBOperationHandlerRetriableErrorForPendingOperation (0.00s)
=== RUN   TestTBOperationHandlerInvalidOperationResponse
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "95e15ccd-b3e4-4db8-8656-fc222a00cf03", "type": "TB", "ydb_operation_id": "1"}
--- PASS: TestTBOperationHandlerInvalidOperationResponse (0.00s)
=== RUN   TestTBOperationHandlerDeadlineExceededForPendingOperation
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "43579ce9-25e4-438b-a876-a428150156df", "type": "TB", "ydb_operation_id": "1"}
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:101	cancelling operation	{"reason": "TTL", "id": "43579ce9-25e4-438b-a876-a428150156df", "type": "TB", "ydb_operation_id": "1"}
--- PASS: TestTBOperationHandlerDeadlineExceededForPendingOperation (0.00s)
=== RUN   TestTBOperationHandlerPendingOperationInProgress
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "cf3fc441-8aa2-4de4-b137-8eeefe0f7225", "type": "TB", "ydb_operation_id": "1"}
--- PASS: TestTBOperationHandlerPendingOperationInProgress (0.00s)
=== RUN   TestTBOperationHandlerPendingOperationCompletedSuccessfully
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "2d2c8994-67a0-477f-a0b7-23cf6e2897ff", "type": "TB", "ydb_operation_id": "1"}
--- PASS: TestTBOperationHandlerPendingOperationCompletedSuccessfully (0.00s)
=== RUN   TestTBOperationHandlerPendingOperationCancelled
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "0665ff10-b2c8-4822-acf1-f5599dc79f8b", "type": "TB", "ydb_operation_id": "1"}
--- PASS: TestTBOperationHandlerPendingOperationCancelled (0.00s)
=== RUN   TestTBOperationHandlerDeadlineExceededForCancellingOperation
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "283d426b-2a8b-43c9-bcb4-9f76b58c908f", "type": "TB", "ydb_operation_id": "1"}
--- PASS: TestTBOperationHandlerDeadlineExceededForCancellingOperation (0.00s)
=== RUN   TestTBOperationHandlerCancellingOperationInProgress
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "ef205a4c-2a50-4e0d-953c-92d04c3966f2", "type": "TB", "ydb_operation_id": "1"}
--- PASS: TestTBOperationHandlerCancellingOperationInProgress (0.00s)
=== RUN   TestTBOperationHandlerCancellingOperationCompletedSuccessfully
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "86d4d418-8fc9-45f1-968c-0854de782e76", "type": "TB", "ydb_operation_id": "1"}
--- PASS: TestTBOperationHandlerCancellingOperationCompletedSuccessfully (0.00s)
=== RUN   TestTBOperationHandlerCancellingOperationCancelled
2024-08-21T16:13:01.236Z	INFO	handlers/utils.go:48	getting operation status	{"id": "b93fb332-82a0-4261-921d-e84a38c87d10", "type": "TB", "ydb_operation_id": "1"}
--- PASS: TestTBOperationHandlerCancellingOperationCancelled (0.00s)
=== RUN   TestTBOperationHandlerRetriableErrorForPendingOperation
2024-08-21T16:13:01.237Z	INFO	handlers/utils.go:48	getting operation status	{"id": "ad802234-85d5-4d92-a98f-1b4487d12a34", "type": "TB", "ydb_operation_id": "1"}
2024-08-21T16:13:01.237Z	INFO	handlers/utils.go:71	received retriable error	{"id": "ad802234-85d5-4d92-a98f-1b4487d12a34", "type": "TB", "ydb_operation_id": "1"}
--- PASS: TestTBOperationHandlerRetriableErrorForPendingOperation (0.00s)
🟢 ydbcp/internal/processor20016ms
🧪 Tests
  • 🟢TestOperationHandlerRegistry
  • 🟢TestProcessor
🖨️ Output
=== RUN   TestProcessor
2024-08-21T16:13:01.223Z	DEBUG	processor/processor.go:76	Operation Processor started	{"period": "10s"}
2024-08-21T16:13:01.224Z	DEBUG	processor/processor.go:108	operation already running	{"operation": "Operation, id edd2956e-cbd5-44bb-a55f-07aebf1fbb18, type TB, state PENDING"}
2024-08-21T16:13:01.224Z	DEBUG	processor/processor.go:108	operation already running	{"operation": "Operation, id edd2956e-cbd5-44bb-a55f-07aebf1fbb18, type TB, state PENDING"}
2024-08-21T16:13:01.224Z	DEBUG	processor/processor.go:120	start operation handler	{"operation": "Operation, id edd2956e-cbd5-44bb-a55f-07aebf1fbb18, type TB, state PENDING"}
2024-08-21T16:13:01.224Z	DEBUG	processor/processor_test.go:40	TB handler called for operation	{"operation": "Operation, id edd2956e-cbd5-44bb-a55f-07aebf1fbb18, type TB, state PENDING"}
2024-08-21T16:13:01.224Z	DEBUG	processor/processor.go:137	operation handler is finished	{"operationID": "edd2956e-cbd5-44bb-a55f-07aebf1fbb18"}
2024-08-21T16:13:01.224Z	DEBUG	processor/processor.go:83	Operation Processor stopped
--- PASS: TestProcessor (0.00s)
=== RUN   TestOperationHandlerRegistry
--- PASS: TestOperationHandlerRegistry (0.00s)
🟡 ydbcp/internal/server0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟢 ydbcp/internal/server/services/backup20014ms
🧪 Tests
  • 🟢TestEndpointSecureValidation
  • 🟢TestEndpointValidation
🖨️ Output
=== RUN   TestEndpointValidation
--- PASS: TestEndpointValidation (0.00s)
=== RUN   TestEndpointSecureValidation
--- PASS: TestEndpointSecureValidation (0.00s)
🟡 ydbcp/internal/server/services/operation0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟡 ydbcp/internal/types0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟡 ydbcp/internal/util/ticker0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟡 ydbcp/internal/util/timer0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟡 ydbcp/internal/util/xlog0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟡 ydbcp/pkg/plugins/auth0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟡 ydbcp/pkg/proto/ydbcp/v1alpha10000ms
🧪 Tests(none)
🖨️ Output
(none)
🟡 ydbcp/plugins/auth_nebius0000ms
🧪 Tests(none)
🖨️ Output
(none)
🟡 ydbcp/plugins/auth_nebius/proto/iam/v10000ms
🧪 Tests(none)
🖨️ Output
(none)

@qrort qrort merged commit 33de8d0 into main Aug 21, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants