From ae834d51bca3358915af189eba357c869b3caa3e Mon Sep 17 00:00:00 2001 From: Michael Ferris Date: Tue, 3 Dec 2024 11:52:24 -0800 Subject: [PATCH] Updated object lock docs --- .../dcs/buckets/object-lifecycles/page.md | 7 + app/(docs)/dcs/buckets/object-lock/page.md | 161 ++++++++++++++++++ app/(docs)/dcs/objects/page.md | 106 ++++++++++++ src/markdoc/partials/create-bucket.md | 1 + .../override-governance-mode-callout.md | 9 + src/markdoc/partials/s3-credentials.md | 50 ++++-- 6 files changed, 322 insertions(+), 12 deletions(-) create mode 100644 app/(docs)/dcs/buckets/object-lock/page.md create mode 100644 src/markdoc/partials/override-governance-mode-callout.md diff --git a/app/(docs)/dcs/buckets/object-lifecycles/page.md b/app/(docs)/dcs/buckets/object-lifecycles/page.md index 2d6e85d88..34072429e 100644 --- a/app/(docs)/dcs/buckets/object-lifecycles/page.md +++ b/app/(docs)/dcs/buckets/object-lifecycles/page.md @@ -1,11 +1,18 @@ --- title: Object Lifecycles +docId: 55e7ac9b-cf21-41fa-be19-087f55de1066 weight: 10 metadata: title: Setting Object Lifecycles description: Explaining how to specify Time to Live for an object in Storj using different tools like rclone, aws cli, and uplink. --- +{% callout type="info" %} +**Object Lock and TTL are Mutually Exclusive** + +You cannot apply a TTL to a locked object, and you cannot lock an object that already has an active TTL. Attempting to combine these features will result in an error. For more information on Object Lock, see the [Object Lock documentation](docId:e94a86fc-3deb-42f0-b7fe-1f80c675930a). +{% /callout %} + ## Setting Object TTL with Custom HTTP Header diff --git a/app/(docs)/dcs/buckets/object-lock/page.md b/app/(docs)/dcs/buckets/object-lock/page.md new file mode 100644 index 000000000..23ab1a32e --- /dev/null +++ b/app/(docs)/dcs/buckets/object-lock/page.md @@ -0,0 +1,161 @@ +--- +title: Object Lock (Beta) +docId: e94a86fc-3deb-42f0-b7fe-1f80c675930a +weight: 1 +metadata: + title: Using Object Lock to Protect Data + description: + Comprehensive guide on enabling and managing Object Lock. +--- + +Object Lock allows you to protect objects from being deleted or overwritten for a specified period or indefinitely. Object Lock operates in three modes: + +- **Governance Mode**: Protects objects against accidental deletion but allows authorized users to bypass the lock. +- **Compliance Mode**: Provides immutable protection; no user can delete or modify the object until the retention period expires. +- **Legal Hold**: Prevents an object from being deleted until the legal hold is explicitly removed, regardless of retention settings. + +{% partial file="override-governance-mode-callout.md" /%} + +## Enabling Object Lock on a Bucket + +### For New Buckets + +{% tabs %} + +{% tab label="aws cli" %} + +```shell {% title="aws cli" %} +aws s3api create-bucket \ + --bucket my-object-lock-bucket \ + --object-lock-enabled-for-bucket \ + --endpoint-url https://gateway.storjshare.io +``` + +{% /tab %} + +{% tab label="Storj Console" %} + +{% partial file="create-bucket.md" /%} + +{% /tab %} + +{% /tabs %} + +### For Existing Buckets + +#### Step 1: Enable Versioning + +{% tabs %} + +{% tab label="aws cli" %} + +```shell {% title="aws cli" %} +aws s3api put-bucket-versioning \ + --bucket my-existing-bucket \ + --versioning-configuration Status=Enabled \ + --endpoint-url https://gateway.storjshare.io +``` + +{% /tab %} + +{% tab label="Storj Console" %} + +1. Navigate to the Browse Buckets page. +2. Locate the bucket you want to modify and open the quick action menu. +3. Select "Enable Versioning". + +{% /tab %} + +{% /tabs %} + +#### Step 2: Enable Object Lock + +{% tabs %} + +{% tab label="aws cli" %} + +```shell {% title="aws cli" %} +aws s3api put-object-lock-configuration \ + --bucket my-existing-bucket \ + --object-lock-configuration "ObjectLockEnabled=Enabled" \ + --endpoint-url https://gateway.storjshare.io + +``` + +{% /tab %} + +{% tab label="Storj Console" %} +1. Navigate to the Browse Buckets page. +2. Locate the bucket you want to modify and open the quick action menu. +3. Select "Object Lock" +4. [Optional] Set default mode and retention period +5. Click "Set Lock" to save + +{% /tab %} + +{% /tabs %} + +## Adding, Updating, and Deleting a Default Object Lock Configuration +Default Object Lock configurations apply retention settings to all new objects uploaded to the bucket. + +### Add/Update Default Configuration +{% tabs %} + +{% tab label="aws cli" %} + +```shell {% title="aws cli" %} +aws s3api put-object-lock-configuration \ + --bucket my-object-lock-bucket \ + --object-lock-configuration "ObjectLockEnabled=Enabled,Rule={DefaultRetention={Mode=GOVERNANCE,Days=30}}" \ + --endpoint-url https://gateway.storjshare.io +``` + +{% /tab %} + +{% tab label="Storj Console" %} + +1. Navigate to the Browse Buckets page. +2. Locate the bucket you want to modify and open the quick action menu. +3. Select "Object Lock" +4. Set a new default mode and retention period + - If you want to remove a default configuration, deselecting the mode will remove the default +5. Click "Set Lock" to save + +{% /tab %} + +{% /tabs %} + +### Delete Default Configuration +{% tabs %} + +{% tab label="aws cli" %} + +```shell {% title="aws cli" %} +aws s3api put-object-lock-configuration \ + --bucket my-object-lock-bucket \ + --object-lock-configuration 'ObjectLockEnabled=Enabled' \ + --endpoint-url https://gateway.storjshare.io +``` + +{% /tab %} + +{% tab label="Storj Console" %} + +1. Navigate to the Browse Buckets page. +2. Locate the bucket you want to modify and open the quick action menu. +3. Select "Object Lock" +4. To remove a default configuration, deselecting the mode will remove the default +5. Click "Set Lock" to save + +{% /tab %} + +{% /tabs %} + +## Object Operations +Please see the docs for [Locking an Object](docId:uyuWpwchZx29f28UGAILP#locking-an-object). + +## Important Notes +- **Object Version Required**: Locking an existing object requires the version id to be specified. +- **Retention Rules**: The retention period can only be extended, not reduced or removed. +- **Mutual Exclusivity with TTL**: Objects with active [TTL Settings](docId:55e7ac9b-cf21-41fa-be19-087f55de1066) cannot be locked, and locked objects cannot have TTL applied. +- **S3 Compatibility**: For full S3 compatibility information please see [](docId:gjrGzPNnhpYrAGTTAUaj) \ No newline at end of file diff --git a/app/(docs)/dcs/objects/page.md b/app/(docs)/dcs/objects/page.md index 334e2c78f..607f6006c 100644 --- a/app/(docs)/dcs/objects/page.md +++ b/app/(docs)/dcs/objects/page.md @@ -105,3 +105,109 @@ uplink cp sj://my-bucket ~/Downloads/storj-tree.png {% /tab %} {% /tabs %} + +## Locking an Object + +### Locking a New Object Upon Upload + +{% tabs %} + +{% tab label="aws cli" %} + +```shell {% title="aws cli" %} +aws s3api put-object \ + --bucket my-object-lock-bucket \ + --key my-file.txt \ + --body my-file.txt \ + --object-lock-mode COMPLIANCE \ + --object-lock-retain-until-date 2025-01-01T00:00:00Z \ + --endpoint-url https://gateway.storjshare.io +``` + +{% /tab %} + +{% tab label="Storj Console" %} + +You cannot lock an object on upload via the Storj Console. You must first upload then lock the object version OR have default retention settings on the bucket. + +{% /tab %} + +{% /tabs %} + +### Locking an Existing Object + +{% tabs %} + +{% tab label="aws cli" %} + +```shell {% title="aws cli" %} +aws s3api put-object-retention \ + --bucket my-object-lock-bucket \ + --key my-file.txt \ + --version-id \ + --retention "Mode=COMPLIANCE,RetainUntilDate=2025-06-01T00:00:00Z" \ + --endpoint-url https://gateway.storjshare.io +``` + +{% /tab %} + +{% tab label="Storj Console" %} + +1. Navigate to **Browse** on the left side menu + +2. Open your bucket from the list + +3. Toggle "Show Versions" + +4. Locate the object you want to modify and open the quick action menu. + +5. Select "Legal Hold" to add a legal hold + +6. Select "Lock" to add a retention period in Compliance or Governance Mode + +7. Select a Lock Type and Retention Priod + +8. Click "Set Lock" to save + +{% /tab %} + +{% /tabs %} + +### Extending Retention Period + +Retention periods can only be extended, not reduced or removed. + +{% tabs %} + +{% tab label="aws cli" %} + +```shell {% title="aws cli" %} +aws s3api put-object-retention \ + --bucket my-object-lock-bucket \ + --key my-file.txt \ + --version-id \ + --retention "Mode=COMPLIANCE,RetainUntilDate=2025-07-01T00:00:00Z" \ + --endpoint-url https://gateway.storjshare.io +``` + +{% /tab %} + +{% tab label="Storj Console" %} + +1. Navigate to **Browse** on the left side menu + +2. Open your bucket from the list + +3. Toggle "Show Versions" + +4. Locate the object you want to modify and open the quick action menu. + +5. Select "Lock" to extend a retention period in Compliance or Governance Mode + +6. Select a Lock Type and Retention Priod + +7. Click "Set Lock" to save + +{% /tab %} + +{% /tabs %} diff --git a/src/markdoc/partials/create-bucket.md b/src/markdoc/partials/create-bucket.md index 5539e1190..ff6bbd9fe 100644 --- a/src/markdoc/partials/create-bucket.md +++ b/src/markdoc/partials/create-bucket.md @@ -5,6 +5,7 @@ 3. Assign the bucket an easily identifiable name, such as "my-bucket". 4. Optional: Enable Object Lock (required for immutability in many applications). + - If you enable Object Lock, you can also set a default retention period using either Governance or Compliance Mode 5. Optional: Enable Object Versioning (note that this will be enabled by default if Object Lock is enabled) diff --git a/src/markdoc/partials/override-governance-mode-callout.md b/src/markdoc/partials/override-governance-mode-callout.md new file mode 100644 index 000000000..65c15ff57 --- /dev/null +++ b/src/markdoc/partials/override-governance-mode-callout.md @@ -0,0 +1,9 @@ +{% callout type="warning" %} +**Governance Mode Override Permissions** + +By default, the account owner and any user with default project access have Governance Mode override permissions due to Storj's macaroon-based access system, which restricts capabilities rather than explicitly granting permissions. + +To avoid unintentionally granting Governance Mode override permissions, use S3 credentials that explicitly restrict this capability for any operations involving Object Lock. + +Note: Storj does not currently support role-based access controls (RBAC) for default restrictions by roles. This functionality may be added in the future. +{% /callout %} \ No newline at end of file diff --git a/src/markdoc/partials/s3-credentials.md b/src/markdoc/partials/s3-credentials.md index 4174d2381..4549b2084 100644 --- a/src/markdoc/partials/s3-credentials.md +++ b/src/markdoc/partials/s3-credentials.md @@ -10,14 +10,10 @@ Create S3 credentials in the Storj console: - **Name:** The name of the credentials (e.g. my-access) - **Type:** S3 Credentials - -4. Click **Next** to provide permissions, either Full Access or Advanced: +5. Choose **Full Access** or **Advanced** + - In most cases, you DO NOT want to choose full access. - - **Permissions:** All - - **Buckets:** Feel free to specify the bucket you created above (e.g. my-bucket), or leave as “All” - - **End date**: provide an expiration date for these credentials (optional) - -5. Click **Next** to provide Access encryption Information +6. Provide Access encryption Information {% callout type="warning" %} In order to see the data uploaded to your bucket in the Storj console, you must unlock the bucket with the same encryption passphrase as the credentials. {% /callout %} @@ -27,8 +23,38 @@ Create S3 credentials in the Storj console: - **Enter a new passphrase**: use this option, if you would like to provide your own new encryption phrase - **Generate 12-word passphrase**: use this option, if you would like to generate a new encryption phrase -6. Click **Create Access** to finish creation of your S3 credentials - -7. Click **Confirm** the Confirm details pop-up message - -8. Your S3 credentials are created. Write them down and store them, or click the **Download all** button. You will need these credentials for the following steps. \ No newline at end of file +7. Select the permissions you want to allow: + - Read + - Write + - List + - Delete + +8. Select the object lock permissions you want to allow + - PutObjectRetention + - GetObjectRetention + - BypassGovernanceRetention + - PutObjectLegalHold + - GetObjectLegalHold + - PutObjectLockConfiguration + - GetObjectLockConfiguration + +9. Choose the buckets you want the access to include: + - All Buckets + - Select Buckets + +10. Set an expiration + +11. Click **Create Access** to finish creation of your S3 credentials + +12. Your S3 credentials are created. Write them down and store them, or click the **Download all** button. You will need these credentials for the following steps. + +### Object Lock Permission Details +| Permission Name | Description | +|---|---| +| PutObjectRetention | Allows you to set retention policies, protecting objects from deletion or modification until the retention period expires. | +| GetObjectRetention | Allows you to view the retention settings of objects, helping ensure compliance with retention policies. | +| BypassGovernanceRetention | Allows you to bypass governance-mode retention, enabling deletion of objects before the retention period ends. | +| PutObjectLegalHold | Allows you to place a legal hold on objects, preventing deletion or modification regardless of retention policies. | +| GetObjectLegalHold | Allows you to view the legal hold status of objects, which is useful for auditing and compliance purposes. | +| PutObjectLockConfiguration | Allows you to set retention policies on the specified bucket, automatically applying them to every new object added to that bucket. | +| GetObjectLockConfiguration | Allows you to view the default retention policies configured for the specified bucket. | \ No newline at end of file