Google Cloud setup

NOTE: This is a very simple setup

Setting up Virtual Machine

From a fresh VM...

Setup docker, you can do this quickly with:

$ curl https://get.docker.com | sudo sh
$ sudo usermod -aG docker $USER
# and then re-login

Checkout the repo on the google-cloud-storage branch and prepare configuration:

$ sudo apt-get install git python-pip -y
$ sudo pip install docker-compose
$ git clone https://github.com/codalab/codalab-competitions.git 
$ cd codalab-competitions
$ git checkout google-cloud-storage

# FOR PRODUCTION YOU SHOULD USE .env_production_sample instead of .env_sample !!!
$ cp .env_sample .env

Ports

Make sure your VM has the relevant ports open:

• NGINX_PORT
• SSL_PORT (if you are using SSL)
• RABBITMQ_PORT (if you will have remote workers connect to this queue)
• FLOWER_PORT (if you want to view Flower logs to debug queues)

Storage

Create 2 buckets, 1 public and 1 private, for example:

• coda-private
• coda-public

coda-private

Set bucket permissions so it is not publicly readable.

Set this CORS configuration via gsutil:

[
    {
        "origin": ["*"],
        "responseHeader": ["Content-Type"],
        "method": ["GET", "PUT", "OPTIONS", "HEAD"],
        "maxAgeSeconds": 3600
    }
]

coda-public

Set the public to publicly readable by all users:

Configure .env

Configure these lines in your .env, at least:

# Google Cloud Storage
DEFAULT_FILE_STORAGE=apps.web.storage.CodalabGoogleCloudStorage
GS_PUBLIC_BUCKET_NAME=coda-public
GS_PRIVATE_BUCKET_NAME=coda-private
GOOGLE_APPLICATION_CREDENTIALS=/app/certs/google-storage-api.json

# Your domain
CODALAB_SITE_DOMAIN=example.com

This assumes your google-storage-api.json api credentials are in the <project root>/certs directory.

More:

You'll want to set these up as well:

SSL

Email

Additional production settings -- at least more gunicorn workers and turn off local compute worker

Start the server

docker-compose up -d

You now have a compute worker and website all in one box. You probably want to disable the compute worker for production.

If there are any problems, try a restart:

docker-compose restart

Create a superuser

To administrate the server, you can create a superuser:

$ docker exec -it django python manage.py createsuperuser

Create a compute worker

See the Using your own compute workers section