Intro: I’m a M.S. student in Computer Science, with my research advisor in the Philosophy department. My thesis work uses deep learning to answer questions in philosophy of aesthetics concerning sound. My main interest is in a course project combining machine learning with the humanities, with a special focus in musical applications. My technical experience is in machine learning, deep learning, statistics, and natural language processing.
Ideas: Ideally, I would like to extend my thesis project into a tool that artists could utilize. Briefly, my research is an adaptation of Google’s DeepDream (https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html) for audio data. It seeks to identify the aesthetic building blocks of “interesting” sound based on the tastes of actual composers. From these components, we can generate entirely new sounds that meet the aesthetic interest of the composers on which the network was trained. My first idea would be to design an interface and community around this project that would allow additional musicians to both add their preferences to a collection of networks, as well as enable the construction of new sounds tuned to specific artists’ preferences.
For my second project idea, I propose a tool to assist in practicing musical improvisation for groups (e.g., jazz improvisation). This is a skill that, in my opinion, constantly requires practice in context to master. One needs to learn the relation of sounds by experimentation. Unfortunately, a backing band on call is not a luxury that many have. Especially for someone beginning their study in improvisation, the pressure of performing a trial and error process in front of other musicians can distract from the actual reflection. Though backing tracks exist, they don’t offer the dynamic give-and-take that an actual band can offer. Though I don’t have any concrete plans for implementation at the moment, I would like to explore the benefits that machine learning can offer to a project like this. I have been interested in exploring the application of cooperative reinforcement learning techniques to musical improvisation.
I am pretty sure that you know all the wavenet things (ai generated sample-level audio)? https://deepmind.com/blog/wavenet-generative-model-raw-audio/
Who is this for? composers? or listeners? How does it change people making music (or listening to music)?
Understanding the needs would be helpful in shaping the first project. For, example, I saw the blog post above, listened to the audio, got really impressed and that was it. What will be the reason why they would come back for their own creative practice (or would I go back to purely listen to the music again?)
Re: the 2nd project, I believe the Shimon – improvisation marimba player from GaTech would be of your interests.
Playing the backing track and give-and-take seems somewhat different. Maybe you can find some mid-point in which it is somewhat prepared but interactive. One recent example that I found was pretty interesting.
https://usdivad.itch.io/seasons
This was more about listening but I felt like I was somewhat performing the piece.
By the way, I would highly recommend you to take the computational creativity presenter or discussant and read the reading assignments immediately.
the readings are:
Can computer create arts? part 1, 2, 3
(https://medium.com/@aaronhertzmann/how-photography-became-an-art-form-7b74da777c63, https://medium.com/@aaronhertzmann/why-computers-do-not-make-art-6c7f9bff6b04, https://medium.com/@aaronhertzmann/will-a-computer-ever-be-an-artist-6f861aa1349)
DreamSketch: Early Stage 3D Design Explorations with Sketching and Generative Design
Rico: A Mobile App Dataset for Building Data-Driven Design Applications
It sounds like a lot but the first three are just a short blog posts