[Terrarium - Providers]: Implementation of Storage Feature - 3 #72

anjanikshree12 · 2024-04-28T07:10:27Z

Follow up PR to #71

This PR includes:

Definition of APIs and Implementation of handler functions that interacts with grpc services to fetch the required files.
Unit tests added for helper functions.
Created a cmd for provider-storage-service
Update allInOne, gateway, providers-v1-rest & browse services.

Description:

The plan is to extend the rest protocol implementation of providers v1, to design 3 APIs that could be used to download the required files/provider binaries as per the provider registry protocol from the storage system.

https://<host-name>/providers/v1/{organization_name}/{name}/{version}/{os}/{arch}/terraform-provider-{name}_{version}_{os}_{arch}.zip
https://<host-name>/providers/v1/{organization_name}/{name}/{version}/terraform-provider-{name}_{version}_SHA256SUMS
https://<host-name>/providers/v1/{organization_name}/{name}/{version}/terraform-provider-{name}_{version}_SHA256SUMS.sig

The files needed as per the provider registry protocol for download through the URL is provider binary archive, shasum file & shasum signature file.

"download_url": "https://<host-name>/providers/v1/{organization_name}/{name}/{version}/{os}/{arch}/terraform-provider-{name}_{version}_{os}_{arch}.zip",
"shasums_url": "https://<host-name>/providers/v1/{organization_name}/{name}/{version}/terraform-provider-{name}_{version}_SHA256SUMS",
"shasums_signature_url": "https://<host-name>/providers/v1/{organization_name}/{name}/{version}/terraform-provider-{name}_{version}_SHA256SUMS.sig"

S3 Directory tree structure to be followed.

--organisation1
     |---- provider1
              |------ version
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS.sig
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         continued ...
              |------ version2
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS.sig
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         continued ...
     |---- provider2
              |------ version
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS.sig
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         continued ...
              |------ version2
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS
                         |----- terraform-provider-<provider-name>_<version>_SHA256SUMS.sig
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         |----- terraform-provider-<provider-name>_<version>_<os>_<arch>.zip
                         continued ...

Note: Shasum file and signature is common for each version, so getting stored at version level.

adcharre

Only a few minor comments, looking good

internal/restapi/providers/v1/handler.go

Co-authored-by: Adam Charrett <[email protected]>

This reverts commit 65c52bc.

This reverts commit ca2601f.

This reverts commit 71911b8.

This reverts commit 7501142.

This reverts commit 26b635f.

This reverts commit 1281216.

This reverts commit 2a7a675.

* implementation of consumer service * update in functions as we remove consumer service * go fmt * Update internal/provider/services/storage/storage.go Co-authored-by: Adam Charrett <[email protected]> * updated storage.go * enhance error logs * updated storage_test * fix * fix * updated storage_test and mock server * [Terrarium - Providers]: Implementation of Storage Feature - 3 (#72) --------- Co-authored-by: Anjani Kumar Srivastava <[email protected]> Co-authored-by: Adam Charrett <[email protected]>

* protobuff definitions for consumer service * updated gen_proto.sh * removed ConsumerService as not required * format * [Terrarium - Providers]: Implementation of Storage Feature - 2 (#71) * implementation of consumer service * update in functions as we remove consumer service * go fmt * Update internal/provider/services/storage/storage.go Co-authored-by: Adam Charrett <[email protected]> * updated storage.go * enhance error logs * updated storage_test * fix * fix * updated storage_test and mock server * [Terrarium - Providers]: Implementation of Storage Feature - 3 (#72) --------- Co-authored-by: Anjani Kumar Srivastava <[email protected]> Co-authored-by: Adam Charrett <[email protected]> --------- Co-authored-by: Anjani Kumar Srivastava <[email protected]> Co-authored-by: Adam Charrett <[email protected]>

APIs & handlers Implementation.

2c17e75

anjanikshree12 self-assigned this Apr 28, 2024

anjanikshree12 added the terrarium-providers label Apr 28, 2024

Anjani Kumar Srivastava added 2 commits April 28, 2024 08:55

updated helper test api name

d210af0

go fmt

c505f59

anjanikshree12 requested review from odelanno, adcharre, ilijad1, dkPranav and vmilovanovicc April 28, 2024 09:01

adcharre reviewed Apr 29, 2024

View reviewed changes

internal/restapi/providers/v1/handler.go Outdated Show resolved Hide resolved

internal/restapi/providers/v1/handler.go Outdated Show resolved Hide resolved

internal/restapi/providers/v1/handler.go Outdated Show resolved Hide resolved

anjanikshree12 and others added 16 commits April 30, 2024 00:36

Update internal/restapi/providers/v1/handler.go

2ecf809

Co-authored-by: Adam Charrett <[email protected]>

Update internal/provider/services/storage/storage.go

2a7a675

Co-authored-by: Adam Charrett <[email protected]>

updated storage.go

1281216

enhance error logs

26b635f

updated storage_test

7501142

fix

71911b8

fix

ca2601f

err2->err renaming

65c52bc

Revert "err2->err renaming"

be0fc69

This reverts commit 65c52bc.

err2->err renaming

1cab2c8

Revert "fix"

868924a

This reverts commit ca2601f.

Revert "fix"

5a67a5f

This reverts commit 71911b8.

Revert "updated storage_test"

c5f6fa0

This reverts commit 7501142.

Revert "enhance error logs"

7b34376

This reverts commit 26b635f.

Revert "updated storage.go"

f851a2e

This reverts commit 1281216.

Revert "Update internal/provider/services/storage/storage.go"

511a1b0

This reverts commit 2a7a675.

anjanikshree12 requested a review from adcharre April 30, 2024 04:00

adcharre approved these changes Apr 30, 2024

View reviewed changes

anjanikshree12 merged commit 8ee56e1 into ciedev-3259-2 Apr 30, 2024
1 check passed

anjanikshree12 deleted the ciedev-3259-3 branch April 30, 2024 15:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Terrarium - Providers]: Implementation of Storage Feature - 3 #72

[Terrarium - Providers]: Implementation of Storage Feature - 3 #72

anjanikshree12 commented Apr 28, 2024 •

edited

Loading

adcharre left a comment

[Terrarium - Providers]: Implementation of Storage Feature - 3 #72

[Terrarium - Providers]: Implementation of Storage Feature - 3 #72

Conversation

anjanikshree12 commented Apr 28, 2024 • edited Loading

This PR includes:

Description:

S3 Directory tree structure to be followed.

adcharre left a comment

Choose a reason for hiding this comment

anjanikshree12 commented Apr 28, 2024 •

edited

Loading