The goal of this project is to provide a space for users under AWS S3 buckets (shared S3 buckets with a "folder" for every user), with a web interface to upload, browse, download and remove files.
This PoC has been done between asamo7 and davidayalas.
Features:
-
It's possible to setup "users" and "admins".
-
A user adds its username (email from saml response) to each key (uploading, browsing, deleting...) . An admin can see everything under a bucket or a folder. See permissions.csv.
- In our PoC, it works with SAML-JWT (https://samltest.id for demo purposes) deployed as a lambda (https://github.com/davidayalas/saml-jwt)
- You need a Custom Authorizer for your API Gateway to validate JWT Token (sample here backend/custom-auth/index.js).
- Our custom authorizer queries with S3 Select the CSV and retrieves the permissions for a user and adds the buckets, folders and roles to the context.
-
A user can upload folders (drag and drop) and the structure is recreated in S3
-
Files can be downloaded (not directories)
-
Folders can be deleted (included not empty)
-
A user can create a folder
-
Install serverless framework: https://www.serverless.com/framework/docs/getting-started/
-
Setup AWS credentials: https://www.serverless.com/framework/docs/providers/aws/cli-reference/config-credentials/
-
Update "serviceName" with your own in setup.demo.json
-
Execute first-deploy.sh (give it executable permissions)
$ chmod +x first-deploy.sh && sh first-deploy.sh -
It will modify some files in order to make all work. Go to https://samltest.id/upload.php and upload manually docs/sp-metadata.xml.
-
Run your server and try
$ npm run serve --prefix=frontend
-
Install all dependencies
$ npm install serverless-s3-sync $ npm --prefix ./backend/custom-auth install ./backend/custom-auth $ npm --prefix ./backend/login install ./backend/login $ npm --prefix ./frontend install ./frontend -
Update "serviceName" with your own in setup.demo.json
-
Basic env variables for SAML (your own, because demo is setup):
- SAML_CERT: you idp saml certificate as string
- IDP_HOST: your idp
- JWT_SECRET: to sign JWT from SAML and validate from custom authorizer
-
Generate frontend distribution
$ cd frontend $ npm run build -
Deploy demo
$ sls deploy $ sls info | grep GET -m 1 | awk -F[/:] '{printf "const endpoint={get(){return '\''https://"$4"/demo/'\'';}};export default endpoint;"}' > frontend/src/assets/js/endpoint.js $ sls s3sync -
Update your sp-metadata.xml with:
- your issuer
- your api gw endpoint
- your api gw stage
-
Upload this file to https://samltest.id/upload.php
-
Upload: this lambda generates the signature for valid uploads.
-
Browsing: this lambda retrieves the objects in a path
-
Delete: this lambda deletes recursively all objects in a path
-
Download: this lambda generate presigned urls for objects to be downloaded safely
-
Login: in this case SAML based to generate a JWT token.
-
Default setup points to samltest.id. Metadata for samltest.id is generated with https://www.samltool.com/sp_metadata.php:
-
In "Attribute Consume Service Endpoint (HTTP-POST)" you have to put your api endpoint:
https://${api gateway id}.execute-api.${region}.amazonaws.com/${stage}/login/callback -
In "entityID" update your issuer (same in environment variable for login handler in serverless.yml)
-
-
Custom authorizer: validates JWT token and adds extra permissions from CSV
-
New frontend is Vue based. Then follow the Getting Started
-
Create file "frontend/src/assets/js/endpoint.js" with the following content replacing values with aproppiate
const endpoint={ get(){ return 'https://${your api id}/${your stage}/'; } }; export default endpoint;
-
Or you can generate it from sls info output:
$ sls info | grep GET -m 1 | awk -F[/:] '{printf "const endpoint={get(){return '\''https://"$4"/demo/'\'';}};export default endpoint;"}' > frontend/src/assets/js/endpoint.js
- Quotas
- Sharing