Background Jobs

Heliotrope uses Resque to process background jobs, such as characterizing files with FITS and creating derivatives (thumbnails, web-friendly video, etc.). In production we recommend resque-pool for managing resque workers. In development, you can use the relevant rake tasks. The job queue is stored in Redis.

How to install Redis

If you are using a mac, you can use homebrew to install redis:

brew install redis

If you're using a Vagrant box, redis server is probably already installed and running.
You can check with the following command: redis-cli ping, the server should respond with "pong".

Running background jobs

To start the redis server:

redis-server /usr/local/etc/redis.conf
# (or on ubuntu)
sudo redis-server /etc/redis/redis.conf

To start workers (3 in the example below) to run the jobs:

COUNT=3 QUEUE=* VERBOSE=1 bundle exec rake resque:work

Viewing background jobs

Resque-web is mounted onto the rails app, but is only available to platform-level admin users (superadmins). To see the status of recent jobs in the resque-web console, start up the server, log in as a platform admin, then navigate your browser to:

http://localhost:3000/resque

Checking Resque Pool Status on U-M deployment

On nectar.umdl.umichi.edu there is a script to check if heliotrope's resque pool is running at:

/hydra-dev/heliotrope-testing/bin/check_resque_pool

Installing FITS

If you haven't yet, you'll need to install FITS or some resque jobs like Characterize will fail. And if one job fails, the whole thing fails.

Download FITS and put it where Heliotrope expects it, /usr/local/bin/fits.sh.

1. Download fits and unzip it somewhere (I did ~/bin/fits-0.10.1/)

2. sudo ln -s /path/to/your/fits-0.10.1/fits.sh /usr/local/bin/fits.sh

3. Change the line in fits.sh from this:

    . "$(dirname $BASH_SOURCE)/fits-env.sh"
   

To this:

    . "/path/to/your/fits-env.sh"