- Install Keras with Apache MXNet backend
- Configure Keras backend and image_data_format
- Validate the Installation
- Train a simple handwritten digit recognition model
- Next Steps
Warning Not all Keras operators and functionalities are supported with MXNet backend. For more information, view the the list
of known issues and unsupported functionalities here.
Steps involved
- Install optional Keras dependencies
- CUDA and cuDNN (required if you plan on running Keras on GPU).
- HDF5 and h5py (required if you plan on saving Keras models to disk).
- graphviz and pydot (used by visualization utilities to plot model graphs).
- Install MXNet - https://github.com/apache/incubator-mxnet
- Install Keras with MXNet backend - https://github.com/awslabs/keras-apache-mxnet
NOTE
The following installation instructions are tested on Ubuntu 14.04/16.04 and Mac OS EL Capitan and Sierra.
# install python and pip if not already installed
$ sudo apt-get update
$ sudo apt-get install -y wget python
$ wget https://bootstrap.pypa.io/get-pip.py && sudo python get-pip.py
# install optional dependencies
$ pip install h5py --user
$ pip install graphviz --user
$ pip install pydot --user $ pip install mxnet --userIf you would like to use MXNet with MKL for high performance on CPU, install mxnet-mkl with below command.
$ pip install mxnet-mkl --user $ pip install keras-mxnet --user # install python and pip if not already installed
$ sudo apt-get update
$ sudo apt-get install -y wget python
$ wget https://bootstrap.pypa.io/get-pip.py && sudo python get-pip.py
# install optional dependencies
$ pip install h5py --user
$ pip install graphviz --user
$ pip install pydot --userNOTE
If you are running on AWS, you can launch an EC2 instance with AWS Deep Learning Base AMI to get CUDA, cuDNN and other required dependencies pre-configured. Follow the [tutorial here](https://docs.aws.amazon.com/dlami/latest/devguide/tutorial-base.html)
Install the following NVIDIA libraries to setup with GPU support:
- Install CUDA 9.1 following the NVIDIA's installation guide.
- Install cuDNN 7.1 for CUDA 9.1 following the NVIDIA's installation guide. You may need to register with NVIDIA for downloading the cuDNN library.
NOTE
Make sure to add the CUDA install path to `LD_LIBRARY_PATH`.
For example, on an Ubuntu machine, if you have downloaded CUDA debian package (cuda-repo-ubuntu1604-9-1-local_9.1.85-1_amd64.deb) and cuDNN 7.1 library (cudnn-9.1-linux-x64-v7.1.tgz), below are set of commands you run to setup CUDA and cuDNN.
# Setup CUDA 9.0.
$ sudo apt-get update
$ sudo apt-get install build-essential
$ sudo apt-get install linux-headers-$(uname -r)
# Assuming you have downloaded CUDA deb package from https://developer.nvidia.com/cuda-downloads
$ sudo dpkg -i cuda-repo-ubuntu1604-9-1-local_9.1.85-1_amd64.deb
$ sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
$ sudo apt-get update
$ sudo apt-get install cuda
$ export CUDA_HOME=/usr/local/cuda
$ export PATH=${CUDA_HOME}/bin${PATH:+:${PATH}}
$ export LD_LIBRARY_PATH=${CUDA_HOME}/lib64/:$LD_LIBRARY_PATH
# Setup cuDNN 7.1 for CUDA 9.1
# Assuming you have registered with NVIDA and downloaded cuDNN 7.1 for CUDA 9.1 from
# https://developer.nvidia.com/rdp/cudnn-download
$ tar -xvzf cudnn-9.1-linux-x64-v7.1.tgz
$ sudo cp cuda/include/cudnn.h /usr/local/cuda/include/
# Use cp -a to copy symlink and avoid copying duplicated files
$ sudo cp -a cuda/lib64/* /usr/local/cuda/lib64/
$ sudo chmod a+r /usr/local/cuda/include/cudnn.h
$ /usr/local/cuda/lib64/libcudnn*You can verify your CUDA setup with the following commands.
$ nvcc --version
$ nvidia-smi # If you use CUDA9. If CUDA8, use, mxnet-cu80
$ pip install mxnet-cu90 $ pip install keras-mxnet --userWhen you install the keras-mxnet, by default, the following values are set.
backend: mxnet
image_data_format: channels_lastWe strongly recommend changing the image_data_format to channels_first. MXNet is significantly faster on 'channels_first' data. Default is set to 'channels_last' with an objective to be compatible with majority of existing users of Keras. See performance tips guide for more details.
NOTE
If you cannot find ~/.keras/keras.json file, just load the Keras library once to get the config state created.
$ python
>>> import keras as k
Using mxnet backend
You can validate the installation by trying to import Keras in Python terminal and verifying that Keras is using mxnet backend.
$ python
>>> import keras as k
Using mxnet backendNext, get hands-on by training a simple Multi Layer Perceptron (MLP) model for handwritten digit recognition using MNIST dataset.
In this section you will verify the Keras-MXNet installation and try out training a simple Multi-Layer Perceptron (MLP) model.
awslabs/keras-apache-mxnet/examples already consists of code to train the model. For simplicity, we only submit the model training job as a black box to see the training in action and do not focus on teaching what the model does.
# Clone the awslabs/keras-apache-mxnet repository.
$ git clone --recursive https://github.com/awslabs/keras-apache-mxnet
# Launch the model training job.
$ python keras-apache-mxnet/examples/mnist_mlp.pyYour output should look something like below.
Using MXNet backend.
60000 train samples
10000 test samples
____________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
====================================================================================================
dense_1 (Dense) (None, 512) 401920 dense_input_1[0][0]
____________________________________________________________________________________________________
activation_1 (Activation) (None, 512) 0 dense_1[0][0]
____________________________________________________________________________________________________
dropout_1 (Dropout) (None, 512) 0 activation_1[0][0]
____________________________________________________________________________________________________
dense_2 (Dense) (None, 512) 262656 dropout_1[0][0]
____________________________________________________________________________________________________
activation_2 (Activation) (None, 512) 0 dense_2[0][0]
____________________________________________________________________________________________________
dropout_2 (Dropout) (None, 512) 0 activation_2[0][0]
____________________________________________________________________________________________________
dense_3 (Dense) (None, 10) 5130 dropout_2[0][0]
____________________________________________________________________________________________________
activation_3 (Activation) (None, 10) 0 dense_3[0][0]
====================================================================================================
Total params: 669,706
Trainable params: 669,706
Non-trainable params: 0
____________________________________________________________________________________________________
Train on 60000 samples, validate on 10000 samples
Epoch 1/20
/home/ubuntu/.local/lib/python2.7/site-packages/mxnet/module/bucketing_module.py:368: UserWarning: Optimizer created manually outside Module but rescale_grad is not normalized to 1.0/batch_size/num_workers (1.0 vs. 0.0078125). Is this intended?
force_init=force_init)
60000/60000 [==============================] - 98s - loss: 1.2175 - acc: 0.6823 - val_loss: 0.5459 - val_acc: 0.8675
Epoch 2/20
43136/60000 [====================>.........] - ETA: 25s - loss: 0.5554 - acc: 0.8458
Congratulations! You have successfully installed Apache MXNet, Keras with MXNet backend and trained your first model!
- Read the Keras documentation at Keras.io.
- For more examples explore keras/examples directory.
- Tutorial on how to use multiple GPUs with MXNet backend - Multi-GPU Distributed Training with Keras and Apache MXNet
- Tutorial on how to save MXNet model from Keras-MXNet - Save MXNet model from Keras-MXNet
- Tips for high performance with Keras-MXNet - Performance Guide for MXNet Backend