ECE 535 Course Project: Multimodal Federated Learning (FL) on Internet of Things (IoT) Data

Team Members:

Samuel Almeida
Arjun Viswanathan

Why is your group suitable for the project?

We would like to learn more about how multimodal data is learned and a model is built from that, as well as its applications to IoT. We both have a solid understanding of ML models, and would like to see how we can use that knowledge in this project.

Motivation

Wanting to learn more about FL

Design Goal:

Understand and benchmark different multimodal datasets in a federated setting

Deliverables:

• Understand multimodal FL • Use given datasets to reproduce the results in the paper • Perform a per-class accuracy analysis of the results and observe the effect of skewed data distribution on the per-class accuracy • Evaluate the system on a multimodal dataset that is relatively balanced in class distribution

Responsibilities:

Setup: Arjun Viswanathan
Software: Both
Networking: Samuel Almeida
Writing: Arjun Viswanathan
Research: Samuel Almeida
Algorithm Design: Both

When we say both of us are assigned a responsibility, we will take equal parts to complete that responsibility.

System Blocks

HW/SW Requirements

A Linux computer that has Python, and optionally CUDA-enabled capabilities, and a GPU.

Project Timeline:

The due date is TBD, but we are hoping we will work 1-4 hours on this project per week to advance towards the finish line and create our final presentation and demos

References:

Multimodal Federated Learning for IoT Data
• Code:
https://github.com/yuchenzhao/iotdi22-mmfl

• Datasets:
https://drive.google.com/drive/folders/1rWJYkfMavGs1F-H0jykJ5V0fIiwrQdJV