-
Notifications
You must be signed in to change notification settings - Fork 16
Getting Started: 1. Google Services
This project works explicitly with multiple team drives to get around the 750GB/day upload limit and 400,000 files-per-team-drive limit. This may not be a concern for some, but to keep the complication to a minimum, team drives are currently the only supported type of Google Drive.
If you need to transfer your files from your "My Drive" to a "Team Drive", you can do so after they've been created within the Google Drive GUI. An 80tb move took around ~3 days, so keep that in mind.
This section will go over how to create your team drives, how to create service accounts that can access them, and how to configure them in Marauder.
Please note: All of the below steps are to be completed with the Google account that has the Drive that you wish to use with Marauder. Some of these steps can be done with other accounts, but for the sake of simplicity, this guide is written to assume that it will all be on the one account.
git clone https://github.com/Makeshift/Marauder.git
cd MarauderAs Docker no longer auto-creates mount points, Marauder needs some directories to put data in. These can be created automatically:
setup/init-dirs.sh- Make sure you can log in to the Google Cloud Console - You don't need to create a project, that's done for you later.
- Make sure you can log in to the Google Admin console.
Creating a group is an easy way to add permissions to a large group of service accounts at once, allowing them access to your team drives.
- Log in to the Google Admin console.
- Click on 'Groups'
- Click 'Create group'
- Enter the following information:
| Field Name | Value |
|---|---|
| Name | Marauders |
| Description | |
| Group email | marauders (It will automatically add your domain after it) |
| Group owner(s) | |
-
Make a note of the group email address! You will need it in the next step in the format
marauders@<your_domain.tld>. - Click 'Next'
- Enter the following information:
| Access Settings | Group Owners | Group Managers | Group Members | Entire Organisation | External |
|---|---|---|---|---|---|
| Contact Owners | ✔ | ||||
| View Members | ✔ | ||||
| View Topics | ✔ | ||||
| Publish Posts | |||||
| Membership Settings | |||||
| Manage Members | ✔ | ||||
| Who can join the group | Only invited users |
- Click 'Next'
Service accounts are used to access our Google Drives on behalf of a user. Marauder includes a container that assists with the job of creating a project and service accounts for you. It's configurable, but to get started quickly, the defaults are fine.
- Obtain the group email address from the previous step
- Run the following command, replacing the email with the one from above:
GROUP_EMAIL=marauders@<your-domain.tld> setup/init-gcloud.sh
- A docker container will download and start. Read what the script will do and type
yto continue. You will be told to go to a link in your browser and log in. Click 'Allow', then enter the verification code into your terminal. Wait for the container to exit.
At this point, the script will have made 100 service accounts for you, plus a .csv file containing the groups. This can be confirmed by checking the service_accounts folder.
Adding Service Accounts to the Group (#8)
- Log in to the Google Admin console.
- Click on 'Groups'
- Hover over the 'Marauders' group and click 'Manage members' on the right hand side
- Hover over the big '+' symbol and click the 'Bulk upload members' button that appears beneath it
- Click 'Attach CSV'
- Choose the CSV created by the init container,
service_accounts/marauder-<id>.csv. You may need to copy it from your host to your local machine. - Click "Upload"
You'll be able to see the progress at the top right. Once finished, you'll be able to refresh the page and see the new members in the list.
- Go to Google Drive
- On the left hand menu, expand the 'Shared drives' section
- Right click and create a new shared drive. Name it whatever you like.
- Click on the newly created team drive, and take note of the folder ID in the URL. It will look something like this: https://drive.google.com/drive/u/1/folders/**0foobargFoobarfooA**. This is your team drive ID.
- Right click in an empty space in the drive and select
New folder. Name the folderencrypted(#7) - At the top left under the name of the drive it will say
1 person - Add members. Click 'Add members'. - Add the group email
marauders@<your-domain.tld>
You can repeat this step as many times as you like. Each team drive can handle 400,000 files. I advise having 2-3, or more if you have a large collection. Keep a note of the team drive IDs.
At this point you have finished setting up the Google services, and can move onto the next step: Configuring Marauder.