What AI is doing in Tod Machover’s new work for the Boston Symphony

“AI Joins the Boston Symphony” sounds as if it came straight from a supermarket tabloid.

If you remember the Weekly World News, it belongs somewhere near “Bat Boy Found in West Virginia Cave!” Bat Boy first appeared on the tabloid’s cover in 1992: a frightening face, a spectacular claim and just enough scientific language to make fiction look momentarily like fact. Five years later, that manufactured monster became “Bat Boy: The Musical”. Material created to provoke fear was taken apart, reimagined and turned into art.

Something similar, and importantly different, is happening around Tod Machover’s “already and not yet”, a new work commissioned by the BSO from Waltham-based composer Tod Machover.

Pairing “orchestra” with “artificial intelligence” leads people to assume AI composed the music and is performing work musicians would otherwise do. These concerns are not imaginary. Machover shares many of them. But that is not how he has designed this piece for human musicians and AI to perform in collaboration.

The AI controversy contains more than one kind of AI

Machover does not defend the commercial uses of generative AI that have caused much of the anger in the music industry. He condemns them in very direct terms.

“The two things that people are mostly doing with AI in music, which I think are terrible, really bad things to do, honestly,” he told me, begin with systems that use enormous databases to produce music that sounds like existing work while differing just enough to be sold as something new.

He called the resulting flood “infinite music, which is just slightly variant or slightly parasitically living off of existing music.”

He was equally blunt about systems that turn a few verbal prompts into a completed song: “I also don’t believe that being creative means giving a few prompts and getting something sent back to you. That’s the opposite of creativity.”

That distinction matters because the phrase artificial intelligence now covers systems performing radically different functions. Some generate finished outputs from prompts. Some classify or predict. Some detect patterns in the material supplied to them. Some operate as responsive instruments. Some help people find relationships they might not otherwise perceive.

A debate treating all of these systems the same can create confusion and fail to distinguish between human creations and the role AI is actually playing.

I have often described artificial intelligence as human intelligence electrified and operating at scale. Machover’s composition gives that idea a literal form. The intelligence that matters begins with the composer and the musicians. The AI system listens for selected qualities in their playing, changes or connects what it hears, and returns the result to people who may respond again. The circuit contains a machine, but it is completed by human attention to the machine output and vice versa.

What the system does

The artificial intelligence in “already and not yet” does not arrive with a finished composition and ask the Boston Symphony Orchestra to perform it. Machover wrote the music. The musicians create the live sound. The system analyzes features in that sound and works with the material as it is performed.

Machover described two capabilities that matter to the piece.

The first is variation. “They can take something you feed it and do something with that material,” he said.

The degree of transformation depends on the constraints. A system told to attend only to duration, loudness, or another limited feature can return something close to the original. Changing those constraints can produce a more complex transformation.

The second capability is connection. The system can make hybrids between sounds and find relationships among musicians who would ordinarily be heard as members of orchestral sections rather than as distinct individuals.

“The other thing these systems do pretty well is to make connections between things and to make hybrids and to say, a little bit of this and a little bit that,” Machover explained.

The point is not to surrender the composition to unrestricted machine generation.

“What you want to make is a system that is going to stay in the kind of boundaries that you want, but is going to do something lively and interesting and surprising in a nice way,” he said.

The system performs several roles during the piece. In some passages, Machover said, “the AI could be thought of like an accompanist.” It adds an aura or extension to what the musician plays.

In others, the performers are told to listen to the transformation and answer it. There are also call-and-response sections: the orchestra plays, the conductor stops and cues the AI, the system answers, and the orchestra responds to what it hears.

That feedback loop changes the next human decision. It is the difference between an output machine and a live relationship.

The orchestra becomes a model of democracy

The technology is not an accessory added to the composition after the fact. Its ability to emphasize individuals, form unexpected connections and produce a collective sound serves the political structure of the piece.

Machover began thinking about the commission during a period when, he said, institutions and values he had long taken for granted appeared to be under direct threat.

The orchestra offered him a way to examine a society. A conventional orchestral score may place one hundred people onstage while asking entire sections to play one line together. Machover wondered what would happen if those one hundred musicians were first treated as individuals and then asked to form different kinds of groups.

“Maybe it’s a way of looking at where we are with democracy and where it might go,” he said.

The three movements enact that progression.

In “Smash Up”, louder instruments gradually prevent softer ones from being heard.

In “Wake Up”, each player receives a small solo. Individual voices emerge and pass melodies among themselves.

In “Stand Up”, musicians are asked to listen and negotiate new relationships, sometimes matching someone else, sometimes doing what no one else is doing.

Machover described the destination as a balance “where it’s not chaotic and not Fascistic, or they’re not all doing the same thing, and they’re not all going their own way.”

That may be the most precise statement of the democratic idea in the piece. Democracy is not represented by everyone playing the same notes. Nor is it represented by complete individual freedom without mutual obligation.

It exists in the difficult work between those extremes: remaining identifiable while listening, responding and building something with other people.

The ethical concern is real

The distinction between this performance’s system and commercial music generators should not be used to dismiss musicians’ broader concerns. Machover shares many of them.

Performers, he said, “don’t want anything they do to be put into a model and then come out on the other end used by somebody in some way they didn’t intend.”

He added, “I don’t think any of us want data that we care about to be taken and reused in some way.”

That is not technophobia. It is a question of authorship, consent and power. The mistake is not raising the concern. The mistake is assuming that every use of AI necessarily performs the same extraction.

This is not actually a single-species or single-technology issue. It is a governance issue.

Humans can appropriate another person’s work, hide how it is being used and profit without meaningful consent. Automated systems can accelerate those practices and make them harder to see. The relevant questions are what enters a system, who authorized it, what the system is allowed to do and who remains accountable for the result.

Creativity requires the ability to go deeper

Machover’s objection to prompt-generated music is not a defense of artistic gatekeeping. He rejects the idea that meaningful creativity belongs only to a tiny number of elite artists. Much of his career has involved building instruments and interfaces that allow more people to make consequential musical choices.

But access to creativity is different from receiving a completed artifact after issuing a command.

“The tools should be clear and easy,” he said, “but the process should allow you to go deep and go deep into what you care about and who you are and what it means to connect with somebody else.”

That idea reaches beyond music.

Language is my instrument, and I have spent much of my career examining the parts of human communication that cannot be reduced neatly to math: the parts that are art, the parts that are squishy, the parts that bleed all over the pages.

As AI enters those spaces, humans and machines do not automatically share the same concepts, associations or ways of assigning meaning. We are beginning a cross-cultural exchange before we have fully developed the language needed to describe it.

Machover encounters the same problem in sound. A system can identify features without knowing what an instrument or note means. It can be instructed to make a sound sharper or more sparkling, yet still have no human understanding of beauty, smoothness or emotional significance.

Its value comes from the structure people build around that capability and the human interpretation that follows.

The final test is still the music

Machover does not want the technology to become the reason the piece matters.

When the BSO approached him, he initially considered writing an entirely acoustic work.

“I’d love to write a piece that’s acoustic and just where people will listen to the music because at the end of the day, that’s what I really care about,” he recalled telling them.

Even after AI became integral to the composition, his standard remained the same.

“It’s not the point of the piece,” he said. “The point of the piece, at the end of the day, is like all pieces of music, the sound and the musical experience.”

The work is meant to move from crisis and silencing toward individual awakening and collective resolve.

One melody runs through all three movements. It is smashed in the first, fragmented in the second and placed near the center of the final movement. The piece builds toward something resembling a call to action but ends, Machover said, on “a bit of a question mark.”

“It didn’t feel authentic to say, ‘We’re here,’” he explained. “That’s why it’s ‘not yet’. We have to keep trying.”

The headline tells us AI has joined the Boston Symphony.

The more accurate story is that one hundred musicians, a composer and a set of carefully bounded computational processes are testing what happens when individual expression can be extended without being erased.

The difference is not semantic. It is the difference between using human expression as raw material to replace its makers and building a system that allows human expression to travel farther, encounter other expression and return changed.

Additional information

The Boston Symphony Orchestra under conductor Kent Nagano will perform “already and not yet”, a world premiere by Tod Machover, at Symphony Hall in Boston. Performances on Thursday, Oct. 22, 7:30 p.m.; Friday, Oct. 23, 1:30 p.m.; and Saturday, Oct. 24, 8:00 p.m. Tickets and program details.

Disclosure: Tod Machover is married to Waltham Times president & publisher June Kinoshita.