NSynth
 | This article, NSynth, has recently been created via the Articles for creation process. Please check to see if the reviewer has accidentally left this template after accepting the draft and take appropriate action as necessary.
Reviewer tools: Inform author |
| Original author(s) | Google Brain, Deep Mind, Magenta |
|---|---|
| Initial release | 6 April 2017 |
| Repository | github |
| Written in | Python |
| Type | Software synthesizer |
| License | Apache 2.0 |
| Website | magenta |
NSynth (a portmanteau of "Neural Synthesizer") is a software algorithm, outlined in a paper in April 2017[1], that generates new sounds through a neural network based synthesis, employing a WaveNet-style autoencoder to learn its own temporal embeddings from four different sounds.[2]. Google then released an open source hardware interface for the algorithm called NSynth Super[3], used by musicians such as Grimes[4] and YACHT [5] to generate experimental music using AI. The research and development of the algorithm was part of a collaboration between Google Brain, Magenta and DeepMind.[6]
Development
Dataset
The NSynth dataset is composed of 305,979 one-shot instrumental notes featuring a unique pitch, timbre, and envelope, sampled from 1,006 instruments from commercial sample libraries.[7] For each instrument the dataset contains four-second 16kHz audio snippets by ranging over every pitch of a standard MIDI piano, as well as five different velocities [8]. The dataset is made available under a Creative Commons Attribution 4.0 International (CC BY 4.0) license. [9]
Model
A spectral autoencoder model and a WaveNet autoencoder model are publicly available on GitHub [10]. The baseline model uses a spectrogram with fft_size 1024 and hop_size 256, MSE loss on the magnitudes, and the Griffin-Lim algorithm for reconstruction. The WaveNet model trains on mu-law encoded waveform chunks of size 6144. It learns embeddings with 16 dimensions that are downsampled by 512 in time[1].
| NSynth Super | |
|---|---|
 NSynth Super Front Panel | |
| Manufacturer | Google Brain, Google Creative Lab |
| Dates | 2018 |
| Technical specifications | |
| Synthesis type | Neural Network Sample-based synthesis |
| Input/output | |
| Left-hand control | Pitch bend, ADSR |
| External control | MIDI |
NSynth Super
Later in 2018 Google released a hardware interface for the NSynth algorithm, called NSynth Super[12], designed to provide an accessible physical interface to the algorithm for musicians to use in their artistic production[13].
Design files, source code and internal components are released under an open source Apache License 2.0[14], enabling hobbyists and musicians to freely build and use the instrument[15].
Hardware
I'm baby four dollar toast deep v subway tile small batch affogato. Celiac bitters cray post-ironic, DSA coloring book sustainable whatever. Cronut trust fund lo-fi, ugh flexitarian dreamcatcher paleo. XOXO mumblecore listicle man braid lomo poke blog. Photo booth cornhole mukbang edison bulb, put a bird on it 3 wolf moon
Influence
The instrument includes features from notable artists, such as Grimes and Yacht, using Nsynth Super in their music productions.[16]
google IO / Grimes etc baby four dollar toast deep v subway tile small batch affogato. Celiac bitters cray post-ironic, DSA coloring book sustainable whatever. Cronut trust fund lo-fi, ugh flexitarian dreamcatcher paleo. XOXO mumblecore listicle man braid lomo poke blog. Photo booth cornhole mukbang edison bulb, put a bird on it 3 wolf moon
References
- ^ a b Engel, Jesse; Resnick, Cinjon; Roberts, Adam; Dieleman, Sander; Eck, Douglas; Simonyan, Karen; Norouzi, Mohammad (2017). "Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders". arXiv:1704.01279 [cs.LG].
- ^ "Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders". research.google.
- ^ "Google's open-source neural synth is creating totally new sounds". wired.co.uk.
- ^ "73 | Grimes (c) on Music, Creativity, and Digital Personae – Sean Carroll". www.preposterousuniverse.com.
- ^ "Music and Machine Learning (Google I/O'19)". youtube.com.
- ^ "NSynth: Neural Audio Synthesis". Magenta.
- ^ "NSynth Dataset". activeloop.ai.
- ^ A bot will complete this citation soon. Click here to jump the queue arXiv:1907.08520.
- ^ "The NSynth Dataset". tensorflow.org.
- ^ "NSynth: Neural Audio Synthesis". GitHub.
- ^ "NSYNTH SUPER". NSYNTH SUPER.
- ^ "Google built a musical instrument that uses AI and released the plans so you can make your own". CNBC. 13 March 2018.
- ^ "NSynth Super is an AI-backed touchscreen synth". The Verge.
- ^ "googlecreativelab/open-nsynth-super". April 1, 2021 – via GitHub.
- ^ "Open NSynth Super". hackaday.io.
- ^ "73 | Grimes (c) on Music, Creativity, and Digital Personae – Sean Carroll". www.preposterousuniverse.com.
Further reading
Engel, Jesse; Resnick, Cinjon; Roberts, Adam; Dieleman, Sander; Eck, Douglas; Simonyan, Karen; Norouzi, Mohammad (2017). "Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders". arXiv:1704.01279 [cs.LG].
External links