User:Younghazi/Music and artificial intelligence

Intro

Artificial intelligence is the development of music software programs which use AI to generate music. As with applications in other fields, AI in music also simulates mental tasks. A prominent feature is the capability of an AI algorithm to learn based on past data, such as in computer accompaniment technology, wherein the AI is capable of listening to a human performer and performing accompaniment. Artificial intelligence also drives interactive composition technology, wherein a computer composes music in response to a live performance. There are other AI applications in music that cover not only music composition, production, and performance but also how music is marketed and consumed. Several music player programs have also been developed to use voice recognition and natural language processing technology for music voice control.

Erwin Panofksy proposed that in all art, there existed 3 levels of meaning: primary meaning, or the natural subject; secondary meaning, or the conventional subject; and tertiary meaning, the intrinsic content of the subject. AI music explores the foremost of these, creating music without the "intention" which is usually behind it, leaving composers who listen to machine-generated pieces feeling unsettled by the lack of apparent meaning.

History

Artificial intelligence finds its beginnings in music with the transcription problem: accurately recording a performance into musical notation as it is played. Père Engramelle's schematic of a "piano roll," a mode of automatically recording note timing and duration in a way which could be easily transcribed to proper musical notation by hand, was first implemented by German engineers J.F. Unger and J. Hohlfield in 1752.

Software Applications

Various forms of software applications exist within artificial intelligence and they are used and applied in the music production process. Software applications are used in music in order to allow musicians, producers, and engineers to mix and master recordings at a faster and more efficient rate. With AI, songwriters and musicians can quickly create “synthesized sound-alike vocals, separate elements on the same track and much more”.

Software Applications in Music:

Jukedeck

Main article: Jukedeck

Jukedeck was a website that let people use artificial intelligence to generate original, royalty-free music for use in videos. The team started building the music generation technology in 2010, formed a company around it in 2012, and launched the website publicly in 2015. The technology used was originally a rule-based algorithmic composition system, which was later replaced with artificial neural networks. The website was used to create over 1 million pieces of music, and brands that used it included Coca-Cola, Google, UKTV, and the Natural History Museum, London. In 2019, the company was acquired by ByteDance.

AIVA

Main article: AIVA

Created in February 2016, in Luxembourg, AIVA is a program that produces soundtracks for any type of media. The algorithms behind AIVA are based on deep learning architectures AIVA has also been used to compose a Rock track called On the Edge, as well as a pop tune Love Sick in collaboration with singer Taryn Southern, for the creation of her 2018 album "I am AI".

Google Magenta

Google's Magenta team has published several AI music applications and technical papers since their launch in 2016. In 2017 they released the NSynth algorithm and dataset, and an open source hardware musical instrument, designed to facilitate musicians in using the algorithm. The instrument was used by notable artists such as Grimes and YACHT in their albums. In 2018, they released a piano improvisation app called Piano Genie. This was later followed by Magenta Studio, a suite of 5 MIDI plugins that allow music producers to elaborate on existing music in their DAW. In 2023, their machine learning team published a technical paper on GitHub that described MusicLM, a private text-to-music generator which they'd developed.

Riffusion

This section is an excerpt from Riffusion.

Generated spectrogram from the prompt "bossa nova with electric guitar" (top), and the resulting audio after conversion (bottom)

Riffusion is a neural network, designed by Seth Forsgren and Hayk Martiros, that generates music using images of sound rather than audio. It was created as a fine-tuning of Stable Diffusion, an existing open-source model for generating images from text prompts, on spectrograms. This results in a model which uses text prompts to generate image files, which can be put through an inverse Fourier transform and converted into audio files. While these files are only several seconds long, the model can also use latent space between outputs to interpolate different files together. This is accomplished using a functionality of the Stable Diffusion model known as img2img.

The resulting music has been described as "de otro mundo" (otherworldly), although unlikely to replace man-made music. The model was made available on December 15, 2022, with the code also freely available on GitHub. It is one of many models derived from Stable Diffusion. Riffusion is classified within a subset of AI text-to-music generators. In December 2022, Mubert similarly used Stable Diffusion to turn descriptive text into music loops. In January 2023, Google published a paper on their own text-to-music generator called MusicLM. Research Projects:

MorpheuS

MorpheuS is a research project by Dorien Herremans and Elaine Chew at Queen Mary University of London, funded by a Marie Skłodowská-Curie EU project. The system uses an optimization approach based on a variable neighborhood search algorithm to morph existing template pieces into novel pieces with a set level of tonal tension that changes dynamically throughout the piece. This optimization approach allows for the integration of a pattern detection technique in order to enforce long term structure and recurring themes in the generated music. Pieces composed by MorpheuS have been performed at concerts in both Stanford and London.

Computer Accompaniment (Carnegie Mellon University)[edit]

The Computer Music Project at Carnegie Mellon University develops computer music and interactive performance technology to enhance human musical experience and creativity. This interdisciplinary effort draws on music theory, cognitive science, artificial intelligence and machine learning, human computer interaction, real-time systems, computer graphics and animation, multimedia, programming languages, and signal processing.

ChucK

Main article: ChucK

Developed at Princeton University by Ge Wang and Perry Cook, ChucK is a text-based, cross-platform language. By extracting and classifying the theoretical techniques it finds in musical pieces, the software is able to synthesize entirely new pieces from the techniques it has learned. The technology is used by SLOrk (Stanford Laptop Orchestra) and PLOrk (Princeton Laptop Orchestra).

Musical Applications

Artificial Intelligence has the opportunity to impact how producers create music by giving reiterations of a track that follow a prompt given by the creator. These prompts allow the AI to follow a certain style that the artist is trying to go for.

AI has also been seen in musical analysis where it has been used for feature extraction, pattern recognition, and musical recommendations.

(Needs to be edited and compacted) Composition:

Objective of Music and AI is to automate musical tasks and to develop autonomously creative systems to compose music within a specific musical style.

With the development of Ai new compositional affordance will give way to new concepts and practices that can used within the industry

Through collaboration, a producer can use AI to help with creativity and bringing their ideas to life by prompting the AI to follow a specific style. By reconceptualizing an artist can have multiple variations of the same track in a shorter amount of time .

Generative models have been able to create music based on a given dataset by learning the structures and patterns in the music to compose a piece based on the learned style.

AI has the potential to impact the composition landscape through new Automated analysis, generative capabilities, collaborative composition, broadened musical vocabulary, style emulation and fusion, enhanced revision and refinement, and accessibility and democratization

Copyright

The question of who owns the copyright to AI music outputs remain uncertain. When AI is used as a collaborative tool as a function of the human creative process, current US copyright laws are likely to apply. However, music outputs solely generated by AI are not granted copyright protection. In the Compendium of U.S. Copyright Office Practices, the Copyright Office has stated that it would not grant copyrights to “works that lack human authorship” and “the Office will not register works produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author.” In February 2022, the Copyright Review Board rejected an application to copyright AI-generated artwork on the basis that it "lacked the required human authorship necessary to sustain a claim in copyright."

The recent advancements in artificial intelligence made by groups such as Stability AI, OpenAI, and Google has incurred an enormous sum of copyright claims leveled against generative technology, including AI music. Should these lawsuits succeed, the machine learning models behind these technologies would have their datasets restricted to the public domain.

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4072806

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3864922

Musical Deepfakes

A more nascent development of AI in music is the application of audio deepfakes to cast the lyrics or musical style of a preexisting song to the voice or style of another artist. This has raised many concerns regarding the legality of technology, as well as the ethics of employing it, particularly in the context of artistic identity. Furthermore, it has also raised the question of to whom the authorship of these works is attributed. As AI cannot hold authorship of its own, current speculation suggests that there will be no clear answer until further rulings are made regarding machine learning technologies as a whole.

Heart on My Sleeve

In 2023 an artist known as “Ghostwriter” used a musical deepfake that copied the voices of Drake and The Weeknd  and prompted an AI to create the track for them.The producer of this track remained anonymous to the public. The track was submitted for Grammy consideration as the best rap song and song of the year .  It went viral and gained traction on TikTok and received a positive response from the audience leading to its official release on Apple Music, Spotify, and YouTube in April of 2023. Many believed the track was fully composed by an AI software, but the producer claimed the songwriting, production, and voice were still done him. It would later be rescinded from any Grammy considerations due to it not following the guidelines necessary to be considered for a grammy award.  Ultimately, the track would end up being removed from all music platforms by Universal Music Group (UMG).

Health Impacts of AI music

AI not only enhances the selection of therapeutic music but also supports mental health intervention by suggesting personalized and data-based recommendations for dealing with depression, anxiety, and stress-related disorder.

A machine learning model has been developed to predict the therapeutic effects of specific songs with 94% accuracy, considering emotional (happiness, sadness) and musical characteristics (tempo, rhythm, harmony and melody). The development process involved collecting and annotating of a diverse dataset comprising various music genres, each tagged with emotional valences and physiological responses observed in participants. ​These inputs were processed by computer programs designed to identify complex relationships between certain music and emotional qualities and their effects on therapy.

TensorFlow Recurrent Neural Network was employed to create music, with nurses’ preferences using Geneva Emotional Music Scales-9 (GEMS-9 and focus group interviews) applied. The peaceful slow tempos, natural sounds, and the lack of human vocals found in classical piano were found among the preferences that were applied.

Similarly, another research study on the application of machine learning to study emotions was trained on the  Million Song Dataset, which can be used  to analyze solfeggio frequencies for the stimulation of certain emotions.