Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature(multi-bucket): add multi-bucket support to storage components #5562

Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
82099b3
initial commit to add 'bucket' property to storage components
jordanvn Aug 10, 2024
f45e688
chore: use StorageBucket type in StorageImagePathProps
jordanvn Aug 12, 2024
130914d
remove duplicate StorageBucket type declaration
jordanvn Aug 20, 2024
6759502
chore: update aws-amplify version to include multi-bucket support
jordanvn Aug 20, 2024
6d200a9
docs: include references to new 'bucket' prop and its usage
jordanvn Aug 20, 2024
77de102
Merge branch 'main' into multi-bucket-support
jordanvn Aug 20, 2024
5ea81d5
more explicitly clarifying that can be a string in docs example
jordanvn Aug 20, 2024
98b51db
Merge branch 'main' into multi-bucket-support
jordanvn Sep 23, 2024
79d1209
chore: changing reference of storage manager to file uploader
jordanvn Sep 23, 2024
9916524
chore: updating yarn.lock
jordanvn Sep 23, 2024
bb0c2f7
chore: undoing unnecessary linting changes
jordanvn Sep 23, 2024
0fab899
chore: moving yarn.lock from main branch parity
jordanvn Sep 24, 2024
77119df
Merge branch 'main' into multi-bucket-support
jordanvn Sep 24, 2024
cd12258
chore: updating yarn.lock to main
jordanvn Sep 24, 2024
4c6eae2
chore: add missing references to 'bucket'
jordanvn Sep 24, 2024
d36d03e
chore: adding tests and new example app
jordanvn Sep 26, 2024
f3953b6
Merge branch 'main' into multi-bucket-support
jordanvn Sep 26, 2024
fabf129
chore: add end of file line
jordanvn Sep 27, 2024
8cbbd05
chore: add changeset
jordanvn Sep 27, 2024
757226b
chore: setting more obviously fake bucket name as example
jordanvn Sep 30, 2024
f825df3
chore: adding link for setting up multi-bucket configuration to docs
jordanvn Sep 30, 2024
e96c3f3
chore: removing unnecessary type definitions
jordanvn Sep 30, 2024
a5c3d5e
chore: removing unnecessary type from Storage Image props
jordanvn Sep 30, 2024
68c0a84
Merge branch 'main' into multi-bucket-support
jordanvn Sep 30, 2024
21c0763
Merge branch 'main' into multi-bucket-support
jordanvn Oct 2, 2024
f6645f1
chore: adding bucket as omitted prop to gen1 props
jordanvn Oct 2, 2024
5787b6c
fix(tests): updating test data to fit expected behavior
jordanvn Oct 10, 2024
55a141a
Merge branch 'main' into multi-bucket-support
jordanvn Oct 10, 2024
2895e9b
Merge branch 'main' into multi-bucket-support
jordanvn Oct 11, 2024
cc79b45
Merge branch 'main' into multi-bucket-support
jordanvn Oct 11, 2024
23e7619
Merge branch 'main' into multi-bucket-support
jordanvn Dec 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/loud-walls-do.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@aws-amplify/ui-react-storage': minor
---

Support for multiple buckets added to storage image and file uploader
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { FileUploader } from '@aws-amplify/ui-react-storage';

export const App = () => {
return (
<FileUploader
acceptedFileTypes={['image/*']}
bucket={{
bucketName: 'my-bucket-xxxxxxxx',
region: 'us-west-2',
}}
path="public/"
maxFileCount={1}
/>
);
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { FileUploader } from '@aws-amplify/ui-react-storage';

export const App = () => {
return (
<FileUploader
acceptedFileTypes={['image/*']}
bucket="my-bucket"
path="public/"
maxFileCount={1}
/>
);
};
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,12 @@ export const FILE_UPLOADER = [
'Determines if the upload will automatically start after a file is selected. The default value is `true`',
type: 'boolean',
},
{
name: 'bucket?',
description:
'The S3 bucket which be will accessed. Allows either a string containing the user-assigned "friendly name" or an object containing a combination of the backend-assigned name on S3 and the S3 region.',
type: 'string | { bucketName: string, region: string }',
},
{
name: `maxFileCount`,
description: '',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,26 @@ You can limit what users upload with these 3 props:
</ExampleCode>
</Example>

## Setting a Bucket

If you have [configured your Amplify project to use multiple S3 buckets](https://docs.amplify.aws/react/build-a-backend/storage/set-up-storage/#configure-additional-storage-buckets), you can use the `bucket` prop to choose which of the buckets the component will use:

<Example>
<FileTypesExample />
<ExampleCode>
```jsx file=./examples/BucketFriendly.tsx
```
</ExampleCode>
</Example>

Alternatively, you can specify the bucket using the name and region it is assigned within S3:

<Example>
<ExampleCode>
```jsx file=./examples/BucketExact.tsx
```
</ExampleCode>
</Example>

## Pausable / Resumable Uploads

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ export const STORAGE_IMAGE = [
'The path to the image in Storage, representing a full S3 object key. See https://docs.amplify.aws/react/build-a-backend/storage/download-files/',
type: 'string | ((input: { identityId?: string }) => string);',
},
{
name: 'bucket?',
description:
'The S3 bucket which be will accessed. Allows either a string containing the user-assigned "friendly name" or an object containing a combination of the backend-assigned name on S3 and the S3 region.',
type: 'string | { bucketName: string, region: string }',
},
{
name: 'imgKey',
description:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
import amplifyOutputs from '@environments/storage/gen2/amplify_outputs';
export default amplifyOutputs;
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import * as React from 'react';

import { Amplify } from 'aws-amplify';
import { Text, Loader } from '@aws-amplify/ui-react';
import { StorageImage } from '@aws-amplify/ui-react-storage';
import '@aws-amplify/ui-react/styles.css';
import amplifyOutputs from './amplify_outputs';

Amplify.configure(amplifyOutputs);

export function StorageImageExample() {
const [isFirstImgLoaded, setIsFirstImgLoaded] = React.useState(false);
const [isSecondImgLoaded, setIsSecondImgLoaded] = React.useState(false);

return (
<>
<StorageImage
alt="public cat 1"
bucket="StorageEndToEnd"
path="public/public-e2e.jpeg"
onLoad={() => setIsFirstImgLoaded(true)}
/>
{isFirstImgLoaded ? (
<Text>The first public image is loaded.</Text>
) : (
<Loader testId="Loader1" />
)}
<StorageImage
alt="public cat 2"
bucket="StorageEndToEndSecondary"
path={() => 'public/public-e2e.jpeg'}
onLoad={() => setIsSecondImgLoaded(true)}
/>
{isSecondImgLoaded ? (
<Text>The second public image is loaded.</Text>
) : (
<Loader testId="Loader2" />
)}
</>
);
}
export default StorageImageExample;
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Feature: Load two images, each from a different S3 bucket with public access level settings

Background:
Given I'm running the example "ui/components/storage/storage-image/multi-bucket"

@react
Scenario: I successfully load two images from two buckets
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

public cat 1 is from the first bucket and public cat 2 is from the second bucket

Then I see "Loader1" element
Then I see "Loader2" element
Then I see the "public cat 1" image
Then I see the "public cat 2" image
Then I see "The first public image is loaded."
Then I see "The second public image is loaded."
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ const FileUploaderBase = React.forwardRef(function FileUploader(
acceptedFileTypes = [],
accessLevel,
autoUpload = true,
bucket,
components,
defaultFiles,
displayText: overrideDisplayText,
Expand Down Expand Up @@ -142,6 +143,7 @@ const FileUploaderBase = React.forwardRef(function FileUploader(

useUploadFiles({
accessLevel,
bucket,
files,
isResumable,
maxFileCount,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,40 @@ describe('FileUploader', () => {
});
});

it('passes a supplied bucket name to the options object', async () => {
const onUploadSuccess = jest.fn();
render(
<FileUploader
bucket="my-bucket"
path="my-path"
maxFileCount={100}
onUploadSuccess={onUploadSuccess}
/>
);
const hiddenInput = document.querySelector(
'input[type="file"]'
) as HTMLInputElement;

expect(hiddenInput).toBeInTheDocument();
const file = new File(['file content'], 'file.txt', { type: 'text/plain' });
fireEvent.change(hiddenInput, {
target: { files: [file] },
});

// Wait for the file to be uploaded
await waitFor(() => {
expect(uploadDataSpy).toHaveBeenCalledWith({
data: file,
options: {
bucket: 'my-bucket',
contentType: 'text/plain',
onProgress: expect.any(Function),
},
path: 'my-pathfile.txt',
});
});
});

it('calls onUploadStart callback when file starts uploading', async () => {
const onUploadStart = jest.fn();
render(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { isFunction } from '@aws-amplify/ui';

import { PathCallback, uploadFile } from '../../utils';
import { getInput } from '../../utils';
import { StorageBucket } from '../../types';
import { FileStatus } from '../../types';
import { FileUploaderProps } from '../../types';
import { UseFileUploader } from '../useFileUploader';
Expand All @@ -25,11 +26,13 @@ export interface UseUploadFilesProps
'setUploadingFile' | 'setUploadProgress' | 'setUploadSuccess' | 'files'
> {
accessLevel?: FileUploaderProps['accessLevel'];
bucket?: StorageBucket;
path?: string | PathCallback;
}

export function useUploadFiles({
accessLevel,
bucket,
files,
isResumable,
maxFileCount,
Expand Down Expand Up @@ -68,6 +71,7 @@ export function useUploadFiles({
if (file) {
const input = getInput({
accessLevel,
bucket,
file,
key,
onProgress,
Expand Down Expand Up @@ -105,6 +109,7 @@ export function useUploadFiles({
}, [
files,
accessLevel,
bucket,
isResumable,
setUploadProgress,
setUploadingFile,
Expand Down
13 changes: 12 additions & 1 deletion packages/react-storage/src/components/FileUploader/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,15 @@ import {
} from './ui';
import { FileUploaderDisplayText, PathCallback, UploadTask } from './utils';

export interface BucketInfo {
bucketName: string;
region: string;
}

// accepts either a 'friendly name' that the user has assigned
// or an object containing the region as well as the name generated by the backend
export type StorageBucket = string | BucketInfo;

export enum FileStatus {
ADDED = 'added',
QUEUED = 'queued',
Expand Down Expand Up @@ -71,6 +80,7 @@ export interface FileUploaderProps {
/**
* Component overrides
*/
bucket?: never;
components?: {
Container?: React.ComponentType<ContainerProps>;
DropZone?: React.ComponentType<DropZoneProps>;
Expand Down Expand Up @@ -132,7 +142,7 @@ export interface FileUploaderProps {
}

export interface FileUploaderPathProps
extends Omit<FileUploaderProps, 'accessLevel' | 'path'> {
extends Omit<FileUploaderProps, 'accessLevel' | 'bucket' | 'path'> {
/**
* S3 bucket key, allows either a `string` or a `PathCallback`:
* - `string`: `path` is prefixed to the file `key` for each file
Expand All @@ -141,5 +151,6 @@ export interface FileUploaderPathProps
*/
path: string | PathCallback;
accessLevel?: never;
bucket?: StorageBucket;
useAccelerateEndpoint?: boolean;
}
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ describe('getInput', () => {
const expected: UploadDataWithPathInput = {
data: file,
options: {
bucket: undefined,
contentType: file.type,
useAccelerateEndpoint: undefined,
onProgress,
Expand All @@ -78,6 +79,7 @@ describe('getInput', () => {
const expected: UploadDataWithPathInput = {
data: file,
options: {
bucket: undefined,
contentType: file.type,
useAccelerateEndpoint: undefined,
onProgress,
Expand All @@ -97,6 +99,7 @@ describe('getInput', () => {
data: file,
options: {
accessLevel,
bucket: undefined,
contentType: file.type,
useAccelerateEndpoint: undefined,
onProgress,
Expand All @@ -116,6 +119,7 @@ describe('getInput', () => {
data: file,
options: {
accessLevel,
bucket: undefined,
contentType: file.type,
useAccelerateEndpoint: undefined,
onProgress,
Expand All @@ -134,6 +138,7 @@ describe('getInput', () => {
const expected: UploadDataWithPathInput = {
data: file,
options: {
bucket: undefined,
contentType: file.type,
useAccelerateEndpoint: undefined,
onProgress,
Expand Down Expand Up @@ -218,6 +223,7 @@ describe('getInput', () => {
const expected: UploadDataWithPathInput = {
data,
options: {
bucket: undefined,
contentType: 'binary/octet-stream',
useAccelerateEndpoint: undefined,
onProgress,
Expand All @@ -237,6 +243,7 @@ describe('getInput', () => {
const expected: UploadDataWithPathInput = {
data,
options: {
bucket: undefined,
contentType: 'binary/octet-stream',
useAccelerateEndpoint: true,
onProgress,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,13 @@ import { UploadDataWithPathInput, UploadDataInput } from 'aws-amplify/storage';

import { isString, isFunction } from '@aws-amplify/ui';

import { ProcessFile, StorageAccessLevel } from '../types';
import { ProcessFile, StorageAccessLevel, StorageBucket } from '../types';
import { resolveFile } from './resolveFile';
import { PathCallback, PathInput } from './uploadFile';

export interface GetInputParams {
accessLevel: StorageAccessLevel | undefined;
bucket?: StorageBucket;
file: File;
key: string;
onProgress: NonNullable<UploadDataWithPathInput['options']>['onProgress'];
Expand All @@ -19,6 +20,7 @@ export interface GetInputParams {

export const getInput = ({
accessLevel,
bucket,
file,
key,
onProgress,
Expand All @@ -41,7 +43,13 @@ export const getInput = ({
const contentType = file.type || 'binary/octet-stream';

// IMPORTANT: always pass `...rest` here for backwards compatibility
const options = { contentType, onProgress, useAccelerateEndpoint, ...rest };
const options = {
bucket,
contentType,
onProgress,
useAccelerateEndpoint,
...rest,
};

let inputResult: PathInput | UploadDataInput;
if (hasKeyInput) {
Expand Down
Loading
Loading