BigJob on Hector

These are instructions for installing BigJob on Hector ( these instructions are for setup where Redis in installed on login node; if you want to use Redis running remotely please find required information at: http://saga-project.github.io/BigJob/sphinxdoc/install/index.html )

1.Bootstrap your Local Python Environment

since you probably don't have a root access on Hector you have to use the following script:
$curl --insecure -s https://raw.github.com/pypa/virtualenv/1.9.X/virtualenv.py | python - $HOME/.bigjob/python

2.Activate your Local Python Environment

$source $HOME/.bigjob/python/bin/activate

3.Install BigJob

$easy_install bigjob

4.Verify installation

$python
$import saga
$import bigjob
$quit()

if these instructions don't produce any errors BigJob and SAGA are successfully installed

5.Make agent directory

$mkdir agent

6.Install Redis (on Hector login node)

for this you probably want to connect to Hector from another terminal;
also it is better to run redis on the same node you run your BigJob scripts;
get list of available addresses:

$host login.hector.ac.uk

select from the list appropriate address and connect, e.g.:

$ssh [email protected]

install redis:

$wget http://download.redis.io/redis-stable.tar.gz
$tar xvzf redis-stable.tar.gz
$cd redis-stable

Switch to login node architecture:
$module swap craype-interlagos craype-istanbul

Switch to the GNU programming environment:
$module swap PrgEnv-cray PrgEnv-gnu

$make

start redis:
$cd src
$./redis-server

Now you are ready to run your BigJob scripts!

p.s. in your pilot_compute_description, please use "project" key instead of "allocation"; "allocation" key will not work on Hector; also you don’t have to specify queue on Hector, since scheduler will select an appropriate queue for you