Hands-On Music Generation with Magenta

Explore the role of deep learning in music generation and assisted music composition

Alexandre DuBreuil

• Editore: Packt Publishing
• Anno: 2020
• ISBN:
  9781838825768
• Edizione: 1
• Argomento: Computer Science
• Categoria: Computer Science General

Design and use machine learning models for music generation using Magenta and make them interact with existing music creation tools

Key Features

• Learn how machine learning, deep learning, and reinforcement learning are used in music generation
• Generate new content by manipulating the source data using Magenta utilities, and train machine learning models with it
• Explore various Magenta projects such as Magenta Studio, MusicVAE, and NSynth

Book Description

The importance of machine learning (ML) in art is growing at a rapid pace due to recent advancements in the field, and Magenta is at the forefront of this innovation. With this book, you'll follow a hands-on approach to using ML models for music generation, learning how to integrate them into an existing music production workflow. Complete with practical examples and explanations of the theoretical background required to understand the underlying technologies, this book is the perfect starting point to begin exploring music generation.

The book will help you learn how to use the models in Magenta for generating percussion sequences, monophonic and polyphonic melodies in MIDI, and instrument sounds in raw audio. Through practical examples and in-depth explanations, you'll understand ML models such as RNNs, VAEs, and GANs. Using this knowledge, you'll create and train your own models for advanced music generation use cases, along with preparing new datasets. Finally, you'll get to grips with integrating Magenta with other technologies, such as digital audio workstations (DAWs), and using Magenta.js to distribute music generation apps in the browser.

By the end of this book, you'll be well-versed with Magenta and have developed the skills you need to use ML models for music generation in your own style.

What you will learn

• Use RNN models in Magenta to generate MIDI percussion, and monophonic and polyphonic sequences
• Use WaveNet and GAN models to generate instrument notes in the form of raw audio
• Employ Variational Autoencoder models like MusicVAE and GrooVAE to sample, interpolate, and humanize existing sequences
• Prepare and create your dataset on specific styles and instruments
• Train your network on your personal datasets and fix problems when training networks
• Apply MIDI to synchronize Magenta with existing music production tools like DAWs

Who this book is for

This book is for technically inclined artists and musically inclined computer scientists. Readers who want to get hands-on with building generative music applications that use deep learning will also find this book useful. Although prior musical or technical competence is not required, basic knowledge of the Python programming language is assumed.

Section 1: Introduction to Artwork Generation

This section consists of an introduction to artwork generation and the use of machine learning in the field, with a comprehensive overview of Magenta and TensorFlow. We'll go through the different models used in music generation and explain why those models are important.

This section contains the following chapter:

• Chapter 1, Introduction to Magenta and Generative Art

Introduction to Magenta and Generative Art

In this chapter, you'll learn the basics of generative music and what already exists. You'll learn about the new techniques of artwork generation, such as machine learning, and how those techniques can be applied to produce music and art. Google's Magenta open source research platform will be introduced, along with Google's open source machine learning platform TensorFlow, along with an overview of its different parts and the installation of the required software for this book. We'll finish the installation by generating a simple MIDI file on the command line.

The following topics will be covered in this chapter:

• Overview of generative artwork
• New techniques with machine learning
• Magenta and TensorFlow in music generation
• Installing Magenta
• Installing the music software and synthesizers
• Installing the code editing software
• Generating a basic MIDI file

Technical requirements

In this chapter, we'll use the following tools:

• Python, Conda, and pip, to install and execute the Magenta environment
• Magenta, to test our setup by performing music generation
  
• Magenta GPU (optional), CUDA drivers, and cuDNN drivers, to make Magenta run on the GPU
• FluidSynth, to listen to the generated music sample using a software synthesizer
• Other optional software we might use throughout this book, such as Audacity for audio editing, MuseScore for sheet music editing, and Jupyter Notebook for code editing.

It is recommended that you follow this book's source code when you read the chapters in this book. The source code also provides useful scripts and tips. Follow these steps to check out the code in your user directory (you can use another location if you want):

1. First, you need to install Git, which can be installed on any platform by downloading and executing the installer at git-scm.com/downloads. Then, follow the prompts and make sure you add the program to your PATH so that it is available on the command line.
2. Then, clone the source code repository by opening a new Terminal and executing the following command:

> git clone https://github.com/PacktPublishing/hands-on-music-generation-with-magenta
> cd hands-on-music-generation-with-magenta

Each chapter has its own folder; Chapter01, Chapter02, and so on. For example, the code for this chapter is located at https://github.com/PacktPublishing/hands-on-music-generation-with-magenta/tree/master/Chapter01. The examples and code snippets will be located in this chapter's folder. For this chapter, you should open cd Chapter01 before you start.

We won't be using a lot of Git commands except git clone, which duplicates a code repository to your machine, but if you are unfamiliar with Git and want to learn more, a good place to start is the excellent Git Book (git-scm.com/book/en/v2), which is available in multiple languages.

Check out the following video to see the Code in Action:
http://bit.ly/2O847tW

Overview of generative art

The term generative art has been coined with the advent of the computer, and since the very beginning of computer science, artists and scientists used technology as a tool to produce art. Interestingly, generative art predates computers, because generative systems can be derived by hand.

In this section, we'll provide an overview of generative music by showing you interesting examples from art history going back to the 18th century. This will help you understand the different types of generative music by looking at specific examples and prepare the groundwork for later chapters.

Pen and paper generative music

There's a lot of examples of generative art in the history of mankind. A popular example dates back to the 18th century, where a game called Musikalisches Würfelspiel (German for musical dice game) grew popular in Europe. The concept of the game was attributed to Mozart by Nikolaus Simrock in 1792, though it was never confirmed to be his creation.

The players of the game throw a dice and from the result, select one of the predefined 272 musical measures from it. Throwing the dice over and over again allows the players to compose a full minute (the musical genre that is generated by the game) that respects the rules of the genre because it was composed in such a way that the possible arrangements sound pretty.

In the following table and the image that follows, a small part of a musical dice game can be seen. In the table, the y-axis represents the dice throw outcome while the x-axis represents the measure of the score you are currently generating. The players will throw two dices 16 times:

1. On the first throw of two dices, we read the first column. A total of two will output the measure 96 (first row), a total of two will output the measure 32 (second row), and so on.

2. On the second throw of two dices, we read the second column. A total of two will output the measure 22 (first row), a total of three will output the measure 6 (second row), and so on.

After 16 throws, the game will have output 16 measures for the index:

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16
2	96	22	141	41	105	122	11	30	70	121	26	9	112	49	109	14
3	32	6	128	63	146	46	134	81	117	39	126	56	174	18	116	83
4	69	95	158	13	153	55	110	24	66	139	15	132	73	58	145	79
5	40	17	113	85	161	2	159	100	90	176	7	34	67	160	52	170
6	148	74	163	45	80	97	36	107	25	143	64	125	76	136	1	93
7	104	157	27	167	154	68	118	91	138	71	150	29	101	162	23	151
8	152	60	171	53	99	133	21	127	16	155	57	175	43	168	89	172
9	119	84	114	50	140	86	169	94	120	88	48	166	51	115	72	111
10	98	142	42	156	75	129	62	123	65	77	19	82	137	38	149	8
11	3	87	165	61	135	47	147	33	102	4	31	164	144	59	173	78
12	54	130	10	103	28	37	106	5	35	20	108	92	12	124	44	131

The preceding table shows a small part of the whole score, with each measure annotated with an index. For each of the generated 16 indexes, we take the corresponding measure in order, which constitutes our minuet (the minuet is the style that's generated by this game – basically, it's a music score with specific rules).

There are different types of generative properties:

• Chance or randomness, which the dice game is a good example of, where the outcome of the generated art is partially or totally defined by chance. Interestingly, adding randomness to a process in art is often seen as humanizing the process, since an und...

Indice dei contenuti

• Title Page
• Copyright and Credits
• About Packt
• Contributors
• Preface
• Section 1: Introduction to Artwork Generation
• Introduction to Magenta and Generative Art
• Section 2: Music Generation with Machine Learning
• Generating Drum Sequences with the Drums RNN
• Generating Polyphonic Melodies
• Latent Space Interpolation with MusicVAE
• Audio Generation with NSynth and GANSynth
• Section 3: Training, Learning, and Generating a Specific Style
• Data Preparation for Training
• Training Magenta Models
• Section 4: Making Your Models Interact with Other Applications
• Magenta in the Browser with Magenta.js
• Making Magenta Interact with Music Applications
• Assessments
• Other Books You May Enjoy