Paper: MOAB Fusion Network Experiment

[Sukh-P]

Arxiv/Blog/Paper Link

Paper from Queen Mary's University (@omniaalwazzan) can be found here and code related to the paper here

Detailed Description

This paper from Queen Mary's University DERI group presents a new type of fusion architecture (Multi-Modal Outer Arithmetic Block, MOAB) which is fusion network which uses outer arithmetic (e.g. outer product, addition etc) to create interaction vectors, this is applied to a problem which has similarities to PVNet e.g. multimodal inputs and separate encodings before fusion of the 1d vectors.

Would be an interesting experiment to implement and train a MOAB style fusion network instead of the concatenation fusion network we currently use to compare forecasting results (would be good to compare other fusion networks too).