|
Human meets AI in music | |
| A project of the CREA network |
[ CREA Home ]
This project develops concrete case study in collaborative human-machine music performance. This is a special case of the more general problem of anticipation and creative processes in mixed human-robot, or anthrobotic, systems.
The concept explored in this project is illustrated on the right. A human musician plays freely, and an AI system controls the parameters of a robotic performer. At the same time, the human musician hears what the robot does and may adjust to it.
We use the term "robot" here in a broad sense to mean any agent that generates physical actions: this could be a dancing robot, a virtual drummer, or a sound agent that spatializes music in the concert hall. In our first experiments, reported below, the musician is a jazz pianist and the robot performer is a commercial virtual drummer (Strike2).
|
Title:
"Anticipation in collaborative music performance using fuzzy systems: a case study"
Authors: Oscar Thörn, Peter Fogel, Peter Knudsen, Luis de Miranda, Alessandro Saffiotti Abstract: In order to collaborate and co-create with humans, an AI system must be capable of both reactive and anticipatory behavior. We present a case study in jazz improvisation, in which a human pianist is accompained by a synthetic drummer controlled by an AI system. Presented at: SAIS-2019: 31th workshop of the Swedish AI Society. Umeå, Sweden, June 18-19, 2019. Full text: 
|
|
|
Synopsis: A hybrid duo of a human pianist and an AI drummer
perform jazz music live. This was the first public concert that
demonstrated our concept of Human-AI harmonious collaboration. It was
given in the occasion of a visit of more than 50 ambassadors to
Örebro University, Sweden, on May 28, 2019. The video includes
an introduction and comments by the authors.
Artists: Peter Knudsen (human pianist), Oscar Thörn (AI programmer), Alessandro Saffiotti (AI expert), Peter Fogel (musicologist) |
|
Synopsis:
A duo of a human pianist and an AI drummer perform jazz tunes. This
performance was part of the
"public evening on AI"
organized in the framework of the First
International CREA Symposium on "Anticipation and Anticipatory
Systems: Humans Meet Artificial Intelligence". The experiment was
aimed at showing how humans and AI can understand each other and
collaborate in a harmonious way. It was performed on June 12, 2019,
and was attended by more than 100 people.
Introduction: Luis de Miranda Artists: Peter Knudsen (human pianist), Oscar Thörn (AI programmer), Alessandro Saffiotti (AI expert), Peter Fogel (musicologist) |
|
Synopsis: In October 2019, our CREA team participated in the
Music Tech Fest
(MTF) hosted by Örebro University. We joined forces with the
"Accents in Motion" team, who had previously researched the use of
body movements to control sound. We modified our system to take
input from a Vicon traking system instead of a piano, so that our AI
drummer follows the movements of two dancers. A simple performance
was recorded on October 19, 2019, and shown at the MTF closing
night. The clip on the right is a short excerpt.
Artists (CREA): Oscar Thorn (AI programming), MD Furkanul Islam (ROS programming), Alessandro Saffiotti (supervision) Artists (Accents in Motion): Andreas Bergsland (music technology), Joseph Wilk (music programmer), Kirsis Mustalahti (dancer), Lilian Jap (dancer) |
|
Synopsis: A human pianist, a virtual drummer and a dancing
robot, all coordinated by AI. This unusual team entertained the
audience of the "Festkonsert", an official yearly celebration of
Örebro University with the participation of about 250 people.
The performance was meant to show the use of AI to orchestrate the
collaboration among human and artificial agents in a creative
process. The music performed by the pianist was improvised, and an
AI system decided the moods, patterns and motions to be performed by
the virtual drummer and by the dancing robot.
The clip on the right is a recording of the public event. You can also watch an excerpt from one of the laboratory tests here. Artists: Peter Knudsen (human pianist), Pepper (the robot), Oscar Thörn (AI programmer), Alessandro Saffiotti (AI expert) |
|
|
Left to right in the photo:
Peter Fogel (musicologist) Alessandro Saffiotti (AI expert) Peter Knudsen (human pianist) Oscar Thörn (AI programmer) Luis de Miranda (philosopher) |
|
[ CREA Home ]
This project was supported by Örebro University through its 20-year jubileum program, and by AI4EU, the EC H2020 landmark project to develop a European AI platform and ecosystem.
CREA is part of Örebro University's larger effort to promote multidisciplinary research in AI.
Website hosted by
Last updated on February 7, 2020 by A. Saffiotti