Skip to content

User_Using your own compute workers

Jump to bottom
Eric Carmichael edited this page May 21, 2020 · 65 revisions

In these instructions, we guide you step-by-step to create your own "compute worker" (a server to which submissions of challenge participants are sent to be executed).

Contents

Hooking up a compute worker to a queue

Go to "Worker Queue Management":

Select "Add Queue":

Give the queue a name. Select whether you want is public or private (public queues can be used by everybody). For private queues, select which users you want to share them with. You can edit these settings later.

Get the queue key you needed in the previous section. Put in the .env configuration file as per instructions.

You can also set a hostname by adding a CODALAB_HOSTNAME env var.

Setting up a server as compute worker

To set up a server as compute worker, we provide instructions for creating Ubuntu Virtual machines (VM), but you may also use your own machine:

Then ssh into your machine and run the following commands:

  • Get docker (quick and dirty way; for a full clean installation see Docker Ubuntu instructions):

    $ curl https://get.docker.com | sudo sh

  • Add yourself to the docker group (so you don't nee to run as root):

    $ sudo usermod -aG docker ubuntu

  • Log out of your server and log back in. Then check your installation:

    $ docker run hello-world

If you want to run everything in one line:

Replace <queue broker url> in the following command with your BROKER_URL

Note: this will make a /tmp/codalab directory

mkdir -p /tmp/codalab && docker run \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /tmp/codalab:/tmp/codalab \
    --env BROKER_URL=<queue broker url> \
    --env BROKER_USE_SSL=True \
    -d \
    --restart unless-stopped \
    codalab/competitions-v1-compute-worker:1.1.5

If you want to run using .env configuration:

  • Create and edit the configuration file .env:

    $ vim .env

Note: vim is an editor coming with Ubuntu. If you rather use emacs, type first sudo apt-get install emacs.

  • Add in .env the line BROKER_URL= and paste your worker queue key:

    BROKER_URL=pyamqp://6ad9ac58-88e3-4a22-9be7-6ed5126ef388:40546f9d-0f2e-4413-a6b2-a1d86cad2b30@localhost/37324ab2-ee78-4e8d-a6ee-6089a159d253

To get your key, see next section. KNOWN BUG: Sometimes the server IP address or URL is replaced by localhost, like in the example above. If you get this, substitute localhost by the IP address or URL of your server.

  • Add BROKER_USE_SSL=True in .env.

  • Get your worker to start computing (it will start listening to the queue via BROKER_URL and pick up jobs):

Note: this will make a /tmp/codalab directory

docker run \
    --env-file .env \
    -d \
    --restart unless-stopped \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /tmp/codalab:/tmp/codalab \
    codalab/competitions-v1-compute-worker:1.1.5

  • Get the ID of the docker process

    $ docker ps

  • Make a submission to your competition and check logs with to confirm it is working:

    $ docker logs -f <DOCKER PS ID>

  • Should you need to change the queue that your compute worker listens to, edit again .env, then run:

    $ docker kill <DOCKER PS ID>

  • Then re-run the one line command above.

Assigning a queue to a competition

Go to your competition page and click "Edit":

Select the queue that you have created:

Cleaning up periodically

It is recommended to clean up your docker images and such, an example crontab:

@daily docker system prune -af
Clone this wiki locally