You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Project Name:
Ising Spin-Based Expert Selector for Mixed Audio and Visual Classification
Team Name:
Petrichoir
Which challenges would you like to submit your project for?
(you can choose more than one if it fits the topic)
Hybrid Quantum-Classical Computing Challenge
Power-Up plan:
We did this to have a framework for further research, not to win prizes. (Though we would not say no!)
Project Link:
(Here will be the link to your repository. Be sure to link to the corresponding commit as explained in the Readme)
Example: https://github.com/KetpuntoG/project_MyTeam/tree/8007fec6786402adc3ac155ad7de19bae674d10c
Remember not to modify this issue after the due date or it will not be evaluated
Project Description:
This model uses a mix of classical CNNs and a Quantum Ising model which work together to solve multiple classification tasks. We chose 8 separate datasets and used 4 experts, though the framework should be flexible enough to work on any data following the same schema.
The overall premise is that an ICA encoding of each datum is used as an input for the Quantum Ising model, which then outputs either it’s probabilities (in training) or a sample (in testing) to select which experts’ contribution should be counted towards the final answer (and also which expert models should be backpropagated in the case of training).
The novelty of this expert selector lies in our leveraging entanglement to select a combination of models for each task. Furthermore, our mixed dataset combines fourier transformed audio data as well as a few standard image classification tasks, meaning that the quantum part of the model can tell apart different types of tasks.
Our Datasets include: MNIST digit, MNIST fashion, CIFAR, OpenMic, SVHN, FMA Small, IRMAS
Our experts use 4 reparameterized convolution layers with 5x5, 3x3, 3x3, and 5x5 kernels, respectively. The first output is passed through at the same time as the second two, then these two branches get added together. The fourth acts on these outputs and then feeds into two reparameterized dense layers with 256 and 10 nodes each. The reparameterized layers come from tensorflow probability and are used to learn a more generalized feature-space.
The convolutional and dense layers all use sigmoid activation besides the last, which uses softmax for classification.
The Quantum model has four wires, with each corresponding to an expert. We encode the wires with the ICA weights from a 12 feature ICA model trained on the full data by applying Rot gates on the | + > state. We then learn weights for a series of IsingXY, IsingZZ, and IsingXX gates operating each in a ring, as well as a final set of Rotation weights to set the measurement basis for each wire. The only other gates are controlled Z gates operating on every other wire to share information on the wires to more than just their neighbors. In our testing, we used 4 experts, thus we wound up with 24 total weights in the Quantum expert selector. (4 each per type of Ising gate, and 4 sets of three for the final Rotations).
We defined these models and a custom training loop in a single tensorflow class and trained as much as we could in the given time, though with the Ising model, each epoch of 7694 samples takes around 30 minutes on an RTX 3060. The final accuracy we report may not be conclusive, though it is quite promising, and I hope you keep your eyes open for more work on this problem from the members of this team.
MNIST
Deng, L. (2012). The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6), 141–142.
MNIST Fashion
Xiao, Han; Rasul, Kashif; Vollgraf, Roland Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. https://arxiv.org/abs/1708.07747
Spoken MNIST
Humphrey, Eric, Simon Durand, and Brian McFee. "OpenMIC-2018: An Open Data-set for Multiple Instrument Recognition." ISMIR. 2018.
FMA
Micha ̈el Defferrard et al. “FMA: A Dataset for Music Analysis”. In:18th International Society for Music Information Retrieval Confer-ence (ISMIR). 2017. arXiv: 1612.01840. url: https://arxiv.org/abs/1612.01840.
CIFAR-10
Krizhevsky, Alex, and Geoffrey Hinton. "Learning multiple layers of features from tiny images." (2009): 7.
SVHN
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y. Ng Reading Digits in Natural Images with Unsupervised Feature Learning NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011. (PDF)
IRMAS
Bosch, J. J., Janer, J., Fuhrmann, F., & Herrera, P. “A Comparison of Sound Segregation Techniques for Predominant Instrument Recognition in Musical Audio Signals”, in Proc. ISMIR (pp. 559-564), 2012
FMA-Small
Defferrard, Michaël, et al. "FMA: A dataset for music analysis." arXiv preprint arXiv:1612.01840 (2016).
COMMIT: BAdams999/QHackathon2023@071a790
Project Name:
Ising Spin-Based Expert Selector for Mixed Audio and Visual Classification
Team Name:
Petrichoir
Which challenges would you like to submit your project for?
(you can choose more than one if it fits the topic)
Hybrid Quantum-Classical Computing Challenge
Power-Up plan:
We did this to have a framework for further research, not to win prizes. (Though we would not say no!)
Project Link:
(Here will be the link to your repository. Be sure to link to the corresponding commit as explained in the Readme)
Example:
https://github.com/KetpuntoG/project_MyTeam/tree/8007fec6786402adc3ac155ad7de19bae674d10c
Remember not to modify this issue after the due date or it will not be evaluated
Project Description:
This model uses a mix of classical CNNs and a Quantum Ising model which work together to solve multiple classification tasks. We chose 8 separate datasets and used 4 experts, though the framework should be flexible enough to work on any data following the same schema.
The overall premise is that an ICA encoding of each datum is used as an input for the Quantum Ising model, which then outputs either it’s probabilities (in training) or a sample (in testing) to select which experts’ contribution should be counted towards the final answer (and also which expert models should be backpropagated in the case of training).
The novelty of this expert selector lies in our leveraging entanglement to select a combination of models for each task. Furthermore, our mixed dataset combines fourier transformed audio data as well as a few standard image classification tasks, meaning that the quantum part of the model can tell apart different types of tasks.
Our Datasets include: MNIST digit, MNIST fashion, CIFAR, OpenMic, SVHN, FMA Small, IRMAS
Our experts use 4 reparameterized convolution layers with 5x5, 3x3, 3x3, and 5x5 kernels, respectively. The first output is passed through at the same time as the second two, then these two branches get added together. The fourth acts on these outputs and then feeds into two reparameterized dense layers with 256 and 10 nodes each. The reparameterized layers come from tensorflow probability and are used to learn a more generalized feature-space.
The convolutional and dense layers all use sigmoid activation besides the last, which uses softmax for classification.
The Quantum model has four wires, with each corresponding to an expert. We encode the wires with the ICA weights from a 12 feature ICA model trained on the full data by applying Rot gates on the | + > state. We then learn weights for a series of IsingXY, IsingZZ, and IsingXX gates operating each in a ring, as well as a final set of Rotation weights to set the measurement basis for each wire. The only other gates are controlled Z gates operating on every other wire to share information on the wires to more than just their neighbors. In our testing, we used 4 experts, thus we wound up with 24 total weights in the Quantum expert selector. (4 each per type of Ising gate, and 4 sets of three for the final Rotations).
We defined these models and a custom training loop in a single tensorflow class and trained as much as we could in the given time, though with the Ising model, each epoch of 7694 samples takes around 30 minutes on an RTX 3060. The final accuracy we report may not be conclusive, though it is quite promising, and I hope you keep your eyes open for more work on this problem from the members of this team.
Below you can find citations for the Datasets we used. Our final dataset file was too large to add to GitHub, but it can be found here: https://drive.google.com/file/d/1INRrcG-Isfs2Ri0KiXRS9GfcL9PM-LVz/view?usp=sharing
MNIST
Deng, L. (2012). The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6), 141–142.
MNIST Fashion
Xiao, Han; Rasul, Kashif; Vollgraf, Roland Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. https://arxiv.org/abs/1708.07747
Spoken MNIST
Humphrey, Eric, Simon Durand, and Brian McFee. "OpenMIC-2018: An Open Data-set for Multiple Instrument Recognition." ISMIR. 2018.
FMA
Micha ̈el Defferrard et al. “FMA: A Dataset for Music Analysis”. In:18th International Society for Music Information Retrieval Confer-ence (ISMIR). 2017. arXiv: 1612.01840. url: https://arxiv.org/abs/1612.01840.
CIFAR-10
Krizhevsky, Alex, and Geoffrey Hinton. "Learning multiple layers of features from tiny images." (2009): 7.
SVHN
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y. Ng Reading Digits in Natural Images with Unsupervised Feature Learning NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011. (PDF)
IRMAS
Bosch, J. J., Janer, J., Fuhrmann, F., & Herrera, P. “A Comparison of Sound Segregation Techniques for Predominant Instrument Recognition in Musical Audio Signals”, in Proc. ISMIR (pp. 559-564), 2012
FMA-Small
Defferrard, Michaël, et al. "FMA: A dataset for music analysis." arXiv preprint arXiv:1612.01840 (2016).
Open-Mic
Humphrey, Eric J., Durand, Simon, & McFee, Brian. (2018). OpenMIC-2018 (v1.0.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.1432913
The text was updated successfully, but these errors were encountered: