Links for music + machine learning. This is a curated list of links for people who want to explore generating music with deep learning.
Check https://www.meetup.com/Perth-Machine-Learning-Group/events for updates on in-person meetups
| Recommended | ||||||
|---|---|---|---|---|---|---|
| Name | Description | Demo | Code | Paper | Suitable for | Released |
| Foundation Models for Music: A Survey | Review paper on the state-of-the-art foundation models for music | Paper | August 2024 | |||
| Generating Audio(.wav or .mp3 files) | ||||||
| MusicFX (Google) | Generate music from text | Blog Audio Youtube |
Paper | Non-coders | December 2023 | |
| Stable Audio (Stability AI) | Generate music and sound effects from text. | Research Blog Github |
Non-coders | September 2023 | ||
| MusicGen (Meta AI) | Generate music from text (controllable) | Hugging Face Spaces Audio |
Google Colab | Paper | Non-coders | June 2023 |
| Make An Audio (ByteDance) | Github Pages Hugging Face Spaces |
Github | Paper | June 2023 | ||
| Harmonai (Stability AI) | Generate novel audio from pretrained models or fine-tune with your own audio tracks | Hugging Face Spaces 24/7 Youtube stream |
Dance Diffusion Fine-tuning Dance Diffusion Github |
Guide WandB article |
Non-coders (Hugging Face), coders (Google Colab) | Oct 2022 |
| Jukebox (OpenAI) | Choose artist, genre, lyrics, and generate audio | Audio | Google Colab | Paper | Coders (Google Colab) | April 2020 |
| Generating MIDI(.mid files) | ||||||
| MuseNet (OpenAI) | Generate new melodies(MIDI file). Scroll down to "Compose in the style of Chopin", generate a continuation of that song, and download the MIDI | Audio | Non-coders | Apr 2019 | ||
| Piano Transformer (Google) | Generate piano MIDI notes from scratch or from a starting MIDI file | Audio | Google Colab | Non-coders | Sep 2019 | |
| Other resources | ||||||
| MIDI(.mid files) | ||||||
| Basic Pitch (Spotify) | Audio-to-MIDI converter with pitch bend detection. Record with your device, or upload an audio file to obtain a MIDI transcription | Non-coders | June 2022 | |||
| Magenta Studio (Google) | Compose multitrack melodies/chords in Ableton Live (music production software) | Github | Ableton users, music producers | Feb 2019 | ||
| MIDI DDSP (Google Magenta) | MIDI-to-Audio synthesis (expression control) | Github Pages Blog |
Github | Paper | Jan 2022 | |
| Hello Magenta (Google) | Introduction to Google Magenta - includes MelodyRNN and MusicVAE | Google Colab Magenta.js |
Coders | |||
| Music Autobot | Generate MIDI - song generation, harmonization, generating melodies, and remixing existing songs. | Non-coders,non-musicians | ||||
| FreeMIDI | Download free MIDI files here | Non-coders,non-musicians | ||||
| Online Sequencer | Make your own MIDI file from scratch | Non-coders,non-musicians | ||||
| Audio(.wav or .mp3 files) | ||||||
| DDSP-VST (Google Magenta) | DDSP morphs audio into a range of different instruments. | Blog Page |
Github | Paper | Music producers | Jan 2020 |
| Spleeter by Deezer | Music source separation library. Upload an audio file, isolate vocals or instrumental | Github Google Colab |
Non-coders, coders | Nov 2019 | ||
| GANSynth | Upload a MIDI file,download an audio file of the interpolation between different instruments | Audio | Google Colab | Paper | Non-coders | Feb 2019 |
| MusicVAE (Google) | Generate audio, interpolate between 2 different melodies or drum beats | Audio | Google Colab | Paper | Non-coders | March 2018 |
| Introduction to Music | ||||||
| Chrome Music Lab | ||||||
| Learning Music with Ableton | ||||||
| Learning Synths (Ableton) | ||||||
| Music Theory | ||||||
| Miscellaneous | ||||||
| Lyria (Google DeepMind) | AI Music Generation model used in Youtube's Dream Track for Shorts and AI Music Tools | Youtube Blog DeepMind Blog |
TBA | |||
| Microsoft - Muzic | Research project on AI music that empowers music understanding and generation with deep learning and artificial intelligence | Github Github Pages |
||||
| NVIDIA - Maxine Audio Effects SDK | API Source Code and Sample Applications. Various audio effects for broadcast use cases with real-time audio processing | Github NVIDIA Broadcast SDK NVIDIA NGC |
||||
| Google Magenta | Open source research project exploring the role of machine learning as a tool in the creative process | Blog Talks Research |
Github Magenta Github |
|||
| PapersWithCode - Music Generation | Papers, code, evaluation papers, datasets | |||||
| PapersWithCode - Music Source Separation | Papers, code, evaluation papers, datasets | |||||
| Python In Music | List of music software written in Python,music software supporting Python,music programming in Python, etc. | |||||
| Github - Deep Learning for Music Generation by Carlos Hernández-Oliván | List of articles related to deep learning applied to music generation | |||||
| Github - Deep Learning papers in music by Yann Bayle | List of articles related to deep learning applied to music | |||||
| Github - Audio Development Tools by Yuan Man | List of sound, audio and music development tools | |||||
| TorchAudio | Library for audio and signal processing with PyTorch | |||||
| Deep Learning with audio thread - Fast.AI forum | List of links related to deep learning applied to audio | |||||
| International Society for Music Information Retrieval(ISMIR) | Links to datasets, educational material, tutorials, software related to music information retrieval | |||||
| NLP4MusA Youtube channel | Natural Language Processing for Music and Audio - Zoom talks by NLP+music researchers | |||||
| SigSep | Open resources for music source separation. Datasets, code, literature, tutorials | Github | ||||
| Google Experiments - Music |