Abstract painting featuring various musical instruments and speakers in vibrant colors and dynamic shapes.

Soundscape image

This image was generated using AI.

Transdisciplinary RFP: Soundspace

April 15, 2025|11 min|Matt Cook|

An institute leverages a unique set of strengths to make waves at the intersection of art, science, and technology.

TL;DR

Combining Rochester’s strengths in music, engineering, and science, Soundspace is an institute that will extend the technical and artistic abilities of music and sound. The team will focus on developing a best-in-class hub for research, education, performance, and public engagement where music and technology meet. Jump to: their big idea; the why and why us; implications for Rochester’s reputation; the last word.

Music and technology have always been linked—from the acoustical-mechanical innovations embodied in the grand church organs of the 18th century, through the 19th century when Edison’s phonograph came along, and into electronic revolution of the 20th century. The convergence of music and technology has accelerated over the last several decades. What once was the domain of audio engineers has become the realm of digital audio workstation (DAW) warriors, brought on in a big way by the COVID-19 pandemic. Proficiency in DAWs—used to produce, edit, and record audio—is now an almost necessary skill for anyone creating and sharing their music on social media platforms. Similarly, computer programming has also taken the stage. Basic coding skills and software knowledge are common needs for those making electroacoustic music—which uses electronic devices and techniques to manipulate and create sounds.

But this merging of sound and audio technology is transforming our world beyond how we create and enjoy music. One major challenge today is making sure what we hear is authentic. Artificial intelligence (AI) is getting scary good at generating fake images, videos, and voices that are nearly indistinguishable from the real thing. There is also a growing threat of AI-generated misinformation. As advancements in generative AI continue, it’s increasingly important to develop better ways to confirm whether a recording is genuine, including techniques such as data-hiding and watermarking. Twenty years ago, Rochester researchers in audio and music engineering (AME) created and patented an audio watermarking method widely employed by the motion picture industry to protect Blu-ray discs, saving them from the kind of piracy that plagued the music industry throughout the 1990s. More recently, a National Institute of Justice fellowship allowed a University graduate student to develop new audio deepfake detection techniques.

Another important area is how emerging technologies can determine how sound interacts with constructed and natural environments. We’re talking about noise control. There are places that have a high propensity for noise where noise can be detrimental to their purpose, such as hospitals and schools. It’s also a concern for cities, where noise can affect public health and wildlife. In collaboration with the University of Rochester Medical Center’s Patient Experience Office, Rochester AME researchers conducted a small pilot project to assess the effect of sound and noise in the care environment on healthcare outcomes and recommend noise mitigation strategies.

Finally, advances in speech recognition, smart speakers, augmented and virtual reality (AR/VR), and AI-driven audio have revolutionized human-computer interaction. It’s yet another area that contains the fingerprints of the University AME researchers, who have developed smart acoustic surfaces and spatial sound technologies that could redefine how we experience digital and physical spaces.

Soundspace aims to push these ideas and work further. In the spirit of the Bauhaus—a German art school that combined fine arts with crafts, design, and manufacturing—the proposed institute will be a hub for technical creativity, research, scholarship, education, performance, and outreach, dedicated to exploring the myriad applications of sound in all environments.

The team*

Lead:

Mark Bocko
Distinguished professor of electrical and computer engineering
Director, audio and music engineering program

Co-lead

Mikel Kuehn ’93E (MA), ’95E (PhD)
Professor, composition
Director, Electroacoustic Music Studios at Eastman School of Music

*The work of this proposed center is supported and supplemented by 11 other faculty members from biomedical engineering, chemical engineering, composition, digital media studies, electrical and computer engineering, musicology, music theory, and music teaching and learning.

Okay—Picture this…

What’s the main idea behind the center?

Have you heard the joke that goes, “a digital animator, a composer, a dancer, and a saxophonist walk into a room…”? Just kidding. We don’t know any jokes that start that way, but that is the start of a pertinent thought exercise. Imagine these people coming together to create a unique multimedia experience, each adding a technological flair to their performance.

The saxophonist modifies their sax with electronic devices, allowing them to change the instrument’s sound in real time.
The dancer wears motion sensors, giving them the ability to control parts of the music and visuals with their movements—so spinning might create a strobe light effect.
The composer dives into machine learning algorithms so that they can turn raw information into music for a more immersive experience.
The animator adds to the immersion by turning the physical space into a dynamic canvas for various imagery with projection mapping

Would this not be a wild show?

The very specific thought exercise we just ran through represents the collaboration Soundspace seeks to encourage and facilitate among musicians, visual artists, engineers, and scientists from acoustics, auditory neuroscience, health sciences, linguistics, computer science, and geophysics. The institute’s vision is to create an environment that fosters opportunities for artists, technologists, and scientists to work together and produce results that exceed the sum of their individual efforts. Furthermore, the work’s collaborative nature would inspire new technology development and avenues of scientific and artistic exploration.

Soundspace will focus on sound technologies for music and media, including electroacoustic music, music for film, games, and mixed media, multimedia and technology-enabled performance, sound and music capture and informatics (essentially, recording and analyzing audio), and technologies for learning music. Another focus is the applications of AI in music and sound, which includes the generation and transformation of music and speech, music and sound analysis, media security, and audio forensics. The team also plans to explore audio interfaces for human-computer interactions, interactive “smart” spaces, spatial sound for AR/VR, and the monitoring and control of sound in built and natural environments.

From the broadest perspective, Soundspace aspires to develop shared facilities that encourage and support University-wide collaboration and create a more closely knit community of musicians, scientists, and technologists that ultimately showcase innovation in sound and music.

Success is in the air

Why and how is Rochester poised to take this on, and what strengths is this center bringing to the table?

Soundspace is confident about its potential for success due to the Eastman School of Music’s (ESM) reputation as a leading international music school, the industry-recognized quality of research and educational opportunities of the AME program, and the University’s extensive resources in allied fields, such as data science, auditory neuroscience, and computer science. Here are some specific strengths they would build on:

ESM

Time-tested leadership in electroacoustic music. The 1983 International Computer Music Conference was hosted by the Eastman Computer Music Center, which was recently rebranded as the Electroacoustic Music Studios @ Eastman (EMuSE). The studios are currently being expanded and renovated with state-of-the-art audio equipment. If you’re really curious about the conference, you can read a report on it, courtesy of the University libraries.
The Beal Institute for Film Music and Contemporary Media.
A notable history of computer programming for music research (starting in the early 1980s), particularly in computer-assisted research in composition, theory, and musicology.
Longstanding collaborations between the Departments of Music Theory and Brain and Cognitive Science, the AME, electrical and computer engineering (ECE), and computer science programs.
The Music Cognition Symposia, which is held several times per year.

AME

Interdisciplinary research, including a grant from the National Science Foundation Future of Work and the Human-Technology Frontier program and multiple grants from the National Institutes of Health with the School of Nursing for asthma symptom monitoring.
The creation of a significant patent portfolio (25+ patents) in new audio technologies.
Research collaboration with the Departments of Biomedical Engineering and Brain and Cognitive Sciences
Rochester’s AME program is unique among peer programs in music technology as the only one in the country that offers an ABET-accredited BS degree in addition to MS and PhD degrees in ECE with a concentration in audio and acoustic signal processing.
“Fundamentals of Audio and Music Engineering” course (through Coursera.org) with nearly 90,000 learners to date.

ESM and AME

Strong connections with leading research centers, most notably Stanford University’s Center for Computer Research in Music and Acoustics (CCRMA). Recently, AME and EMuSE partnered in joint residency to host composer John Chowning, inventor of FM Synthesis and founder of CCRMA.
Research collaborations with AME and music theory faculty at ESM.
The creation of a University of Rochester 3D Audio YouTube channel showcasing groundbreaking immersive audio recordings of ESM performances.
Shared research between ESM faculty members and AME researchers. In addition to those mentioned in music cognition, Ming-Lun Lee is researching Ambisonic immersive audio through the lens of live concert recording and technical implementation. Soundspace co-lead Mikel Kuehn is researching Ambisonics from the perspective of digital synthesis algorithms in musical composition.

The Soundspace team is keenly aware that the University presents vast opportunities for collaboration. They will continue to seek the broadest possible range of contributors and aspire to find more ways to work with researchers and scientists from across the University.

Reputation. Reputation. Reputation.

How will the success of the center enhance Rochester’s prominence and prestige?

In the early 20th century, some well-known powerhouses helped make the city of Rochester a major hub of communication innovation. Eastman Kodak was revolutionizing photography and imaging. Xerox was pioneering photocopying. Bausch & Lomb was pushing the frontiers of optics. And then there was The Stromberg-Carlson Telephone Manufacturing Company.

Stromberg-Carlson was a significant national player as a supplier to competitors of the Bell Telephone Company (AT&T), but eventually, the company began producing radios and sound equipment, contributing to the rise of public broadcasting. That early Rochester expertise is how Eastman Theatre hosted some of the country’s first live music broadcasts. (Fun facts: You can still find the building for Stromberg-Carlson’s headquarters on University Ave., and its warehouse is now the local recreation space Radio Social—which is why they have a display of radios in their lobby area.)

Soundspace is an attempt to put Rochester back at the frontier of music and technology by using its expertise in music and AME to help produce musical experiences that engage three or four of our senses as opposed to one.

The music industry has undergone enormous changes in just the last 20 years. Rapid advances in technology have enabled amateur musicians to produce studio-quality recordings and professional-level performances. The music institutes that adapt to and embrace these changes will be the future’s leaders. Soundspace would give ESM a potent focus on technology, modernizing its mission and helping it secure its position as one of the leading international music schools for the next 100 years.

The last word

When accounting for all the applications across media and entertainment and consumer electronics and audio, music and audio technologies are behind more than $2 trillion of global economic activity annually. Not only is it a robust industry in which Rochester has the ability to be a dominant force, but it is also one that is intrinsically engaging.

People like music!

So, one of the more aspirational aspects of Soundspace would be to create a fully functioning research lab equipped with “the works” that is also—in spirit—part Rochester Museum and Science Center and part Strong National Museum of Play. On the research side, it would be a space for University artists and engineers to experiment and conduct research that drives innovation and collaborate with industry partners. The play side would be more of a public learning lab, where the community could take classes, play instruments, engage with various music technologies, and otherwise explore. It’s a vision that makes the University and its work more accessible to the community while enabling the work that feeds the University’s reputation.

Speaking of reputation, let’s go back to Rochester’s ability to be a dominant force in the music technology industry. Soundspace co-lead Mark Bocko briefly shared time at the University with Bob Freeman, who led ESM from 1973 to 1996. Mark recalls Bob was fond of saying something along the lines of “Stanford University has a great engineering program, but we have the Ying Quartet!” The anecdote touches on a larger point about the viability of Soundspace’s endeavor: while there are places across the country that work in the music technology space, none have a music school with the bona fides of ESM and the deep and comprehensive programs in engineering and science found at the University of Rochester.

“We have advantages no one else has,” Mark says. “That’s critical because we’re not huge. We don’t have the size endowment you find at considerably bigger universities. So, we have to pick our battles very carefully in areas where we can absolutely dominate. I think this institute offers the University a really good chance at distinguishing itself.”