ERROR: pygpu was configured but could not be imported or is too old (version 0.7 or higher required)

[sanjita-suresh]

I am trying the tutorial http://data.statmt.org/wmt17_systems/ to use the pretrained model de-en to translate wmt18 test dataset from de to en. I have this error and I am running on ubuntu and CPU only machine. Not sure if I have to change something in nematus for CPU only machine or it only works for GPU machines.

Thanks in advance.

[rsennrich]

have you set device=cpu in wmt17-scripts/vars ? This might stop theano from trying to use pygpu.

If you still have trouble, I suggest one of two options:

• debug your theano installation. This is no Nematus-specific issue, and you should be able to find other instructions to solve your problem.
• use Tensorflow Nematus. You can convert the models to be compatible with Tensorflow Nematus with the script nematus/nematus/theano_tf_convert.py

[sanjita-suresh]

Thank you. There was some issue with theano.

Do you know how long it might take to translate wmt18 news test dataset on CPU ?

[rsennrich]

depends on the model (ensembles are more expensive), but with the Theano codebase, it's probably in the range of hours on a single CPU core. Note that you can easily parallelize this across CPU cores using the '-p' option as an argument to translate.py.

[sanjita-suresh]

Thanks for the response.

I am using ensemble model and it has been running more than 12 hours with this warning- "WARNING: We did not find a dynamic library in the library_dir of the library we use for blas. If you use ATLAS, make sure to compile it with dynamics library."

Should I add some library or just wait?

[rsennrich]

if all you want is a baseline score, you could just wait. If you want to do more translation in the future, you should be able to substantially increase translation speed:

• for CPU Theano, the BLAS library used can have a big effect on speed. I recommend that you test your installation according to these instructions: http://deeplearning.net/software/theano/troubleshooting.html#test-blas
• most (all?) ensemble scripts use a single CPU process (look for -p 1 in the relevant translate-ensemble.sh). Depending on the number of CPU cores on your machine, you can use 4-32 processes and achieve an almost linear speedup.

either way, it's probably worthwhile to test speed on a smaller subset, or add the option '-v' to the translate.py command, to get a better estimate how quickly your translating.