Implementation of Deep Homography in Keras

This repository implements Deep Homography in Keras. It contains both the code to generate dataset from any large image dataset (MSCOCO, CIFAR 10, etc.) and the code to train them. The trained models for the dataset MSCOCO 2014 are stored in the models directory.

Processed data can be downloaded here

Uses of functions:

In generate-dataset.py

• load_random_image(path_source, size=128) - loads a random image from a directory
• path_source - directory where to get a random image of file format .jpg
• size - resize the image
• save_to_file(images, offsets, path_dest) - saves an image pair in npz format
• images - array of image pairs to be stored
• offsets - offset relationship of the two images
• path_dest = path where to store the saved array as npz
• generate_dataset(path_source, path_dest, rho, height, width, data, box) - generate a dataset from a large group of images
• path_source - source of the images
• path_dest - path where to save the converted images as arrays
• rho - range of possible offsets
• height - height of the resized image
• width - width of the resized image
• data - number of image pairs to be generated
• box - size of the final image pair
• group_dataset (path, new_path, box=128, size=64) - group a dataset by batches. This is generally recommended since training is significantly slower when the data is ungrouped
• path - path of the ungrouped dataset
• new_path - where to save the grouped dataset
• box - size of the image box
• size - size of grouped dataset. Default is at 64 which is the batch size in training

In deep-homography.py

• data_loader(path) - loads the npz files
• path - path where to load the data
• common(model) - the common architecture to both models
• model - model to be used for the network
• regression_network_last(model, lr) - the last layer of the regression network
• model - model to be used for the nework
• lr - learning rate of the model
• classification_network_last(model, lr, batch_size=64) - the last layer of the classification network
• model - model to be used for the nework
• lr - learning rate of the model
• batch_size - batch size of training
• regression_network(train_data_path, val_data_path, total_iterations, batch_size, itr, steps_per_epoch, val_steps) - training the entire regression network
• train_data_path - path of the data for training in npz format
• val_data_path - path of the data for validation in npz format
• total_iterations - total iterations for training
• batch_size - batch_size of training
• itr - number of iterations before changing the learning rate
• steps_per_epoch - number of steps per epoch for training
• val_steps - number of steps for validation
• classification_network(train_data_path, val_data_path, total_iterations, batch_size, itr, steps_per_epoch, val_steps) - training the entire classification network
• train_data_path - path of the data for training in npz format. Ensure that the data are grouped by batches. If not use the group_dataset function from generate-dataset.py
• val_data_path - path of the data for validation in npz format. Ensure that the data are grouped by batches. If not use the group_dataset function from generate-dataset.py
• total_iterations - total iterations for training
• batch_size - batch_size of training
• itr - number of iterations before changing the learning rate
• steps_per_epoch - number of steps per epoch for training
• val_steps - number of steps for validation
• test_model(model_save_path, test_data_path, test_size=5000, batch_size=64) - testing a saved model
• model_save_path - path where the models are saved
• test_data_path - path of the data for testing in npz format
• test_size - size of the testing data
• batch_size - batch size to be accepted by the network. Same as batch_size during training

Current Performance

• Regression Network MACE - 49.425491601
• Classification Network MACE - 71.5220545317