Musicians in the New World Symphony in Miami perform below a digital projection designed by students from Parsons The New School for Design. Photo courtesy of the New World Symphony.

On November 10, brass and woodwind trios and dueling violinists took to the stage of Miami’s New World Symphony. Despite standing center stage, the musicians, clad all in black, were not what the audience was looking at. As they played, dazzling images were projected over their heads, moving and changing along with the music. These projections were not pre-recorded; they were generated by the movements of the musicians below them, turning a classical concert into a breathtaking multimedia experience.

This visually stunning experience was the result of months of hard work by teams of students in the School of Art, Media, and Technology at Parsons The New School for Design. In the Expanded Cinema and Sound-Image course, students spent a semester working with faculty members Edward Keller, Joe Saavedra, and Leif Percifield to create hardware and software to be used during orchestral performances to enhance concertgoers’ experience.

Students devised a variety of sensors that are attached to orchestra members during a performance to monitor movement, biometric data, and more. These technologies included wearable motion sensors using accelerometers and breath rate monitors, both of which communicate over Wi-Fi. Student-designed software translates this data into moving projections that complement the music. After first being tested on their fellow students from Mannes College The New School for Music, the devices were used in November’s public performance at the New World Symphony. Attendees at the performance would have seen the delicate movements of a violinist’s hand produce a beautiful splash of color. A sharp intake of breath by a drummer makes digital shapes spin, grow, or shrink. The tempo and melody of the music change the speed and rotation of colorful patterns.

The New World Symphony is no stranger to this kind of groundbreaking work. As a laboratory for musical education and expression, the orchestra tests the boundaries of the classical music world, frequently incorporating new technology into performances at its concert hall, designed by Frank Gehry.

“We are excited that our students have the opportunity to work with a prestigious organization like the New World Symphony,” says Keller, the director of the Center for Transformative Media at Parsons. “The work they’ve done is extremely impressive, showing the uniquely creative design one can make by going beyond the status quo and approaching an aesthetic challenge—merging technology, image, computer art, and classical music—from a fresh perspective.”

A full list of participating students, as well as the names of their projects, is below.

Poulenc – Sonata – Horn, Trumpet and Trombone
Quincy Bock
Francesca Castelli
Matt Griffis
Michael Kahane
Mauricio Sanchez Duque