Low-rank highway networks
Highway Networks with Low-rank matrix factorization
Paper: http://arxiv.org/abs/1603.03116
A reparametrization of Highway Networks (Srivastava et al. 2015, http://arxiv.org/abs/1505.00387) where the hidden layer matrices are constrained to be low-rank. Reduces data complexity and memory usage.
This code is based on and depends on Deepy (Shu 2015). In order to run dowload Deepy from https://github.com/zomux/deepy and copy these files in the deepy/experiments/highway_networks/ directory.