Yale University

Making Music the New-Fashioned Way

May 8th, 2009 by Bennett Lovett-Graff

Paul Hudak, Professor of Computer Science and incoming Master of Saybrook College, received a McCredie Fellowship to further develop the set of compositional programming modules called Haskore for use by students in his class, Fundamentals of Computer Music: Algorithmic and Heuristic Composition.

Some time during the fifth century B.C., on the Greek island of Samos, legend has it that the Greek philosopher and mathematician Pythagoras, while passing a blacksmith’s shop, took note of the musical ring that accompanied each stroke of the smith’s hammer. Intrigued, he looked in the shop and noted the different anvils.  After careful measurement, he realized that the sizes of the anvils were in simple ratios to one another, which explained the musical harmonies emanating from so unlikely a venue.  This discovery inspired Pythagoras’ first formulations about the relationship between math and music, laying the groundwork for ancient Greek music.

Today, the connection between mathematics and music seems obvious. But in an age of ever more sophisticated applications of computing technology to music-from Guitar Hero to laser harps to iPhone wind instruments-opportunities to instruct students and advance the relationship between the two abound.  Or, at least, so contends Professor of Computer Science and incoming Master of Saybrook College, Paul Hudak.

Recipient of a McCredie Fellowship in Instructional Technology, Hudak is an expert in the functional programming language Haskell and an amateur jazz pianist, which explain his own role in formulating a computer language for composing music in Haskell, fittingly dubbed Haskore.

“I have worked in programming language design for graphics, animation, robotics, and computer music,” Hudak notes.  So when Hudak together with computer science faculty colleagues Julie Dorsey and Holly Rushmeier developed a new program of study at Yale for “Computing and the Arts,” Hudak saw an opportunity to combine his love of music and music theory with his work in the development of Haskore. The syllabus to “Fundamentals of Computer Music: Algorithmic and Heuristic Composition” pithily describes the course as follows: “This is not a course simply to learn how to use music technology….This is a course to learn about the mathematical and computational principles that underlie music technology.”

A screenshot of Haskore's Music Synchronizer interface. Click the image for a larger view.

As Hudak points out, a functional programming language such as Haskell is ideal for elegantly describing many musical structures.  Because musical notation reflects a set of mathematical relationships expressed through pitch, duration, harmony, and volume, Haskore permits programmers to adopt an explicitly mathematical and algorithmic approach to crafting musical compositions, from randomly generated, musically rich sound patterns to fully scored compositions.

However, the class’s goal was not merely to instruct students in how to write strings of code.  The class, after all, was open not only to computer science students but also, according to the course description, “music majors, computing and the arts majors, as well as any undergraduate student or graduate student with a keen interest in computer music concepts.”  This meant providing students with a way of transmitting musical information, such as through a keyboard, into Haskell and outputting results through computer speakers or a MIDI devices. It was at this point that Hudak requested and received a McCredie Fellowship for graduate student Paul Liu to design the necessary software to perform just those functions.

“Haskore is an example of ‘functional reactive programming,’” Liu explains, “which means it can focus on how values change over time-a critical aspect of music.” By way of example, he offers, consider the flight of a ball thrown on a baseball field. As it makes its way, its speed, its height as measured from the ground, and its distance from the thrower and receiver respectively change over the course of its three-second flight. We already know that mathematics can describes these relations, and well-crafted code in a robust computer language can recreate these factors.  Music, Liu notes, is no different. The musical factors of pitch, volume, harmony, and duration are all components that can change-recur, modulate or recombine in any number of ways-over time. Creating an application that allows students to compose music and let it be recorded not merely as notes on a page but as lines of mathematically adjustable computer code was critical to the class’s success.

A screenshot showing a virtual piano keyboard and guitar fretboard which was created by a student. Click on the image for a larger view.

Computers are a long way from composing great music, of course.  But that’s not really the purpose of Hudak’s work, who sees the computer as a tool to help the musician achieve his or her artistic goals.  Appropriate software can help augment the creative process (for example, suggesting new harmonic or melodic ideas, or transforming existing ideas in interesting ways), implement tedious tasks (such as transposition, analysis, accompaniment, etc.), or do things that are technically difficult or even impossible (such as playing notes faster than is humanly possible, or simulating the sound of a saxophone the size of a house).

The results, Liu points out, could be remarkable.  Hudak’s course required students to “either create some interesting music or create some interesting music technology,” with an in-class “concert” at the semester’s end. One such project was a tool that allowed the user to remove any one instrument in a MIDI composition and fill in manually by keyboard. This “karaoke” experience, however, is not what made this particular project unique.  What did was the fact that when the instrumentalist filled in for the missing instrument, if his tempo slowed, so did that of all of the other instruments; if he sped up, they did as well. But even this is only the tip of the iceberg since it is possible, for example, to have the entire piece transpose keys in response to a sudden key change by the instrumentalist or switch over from keeping tempo to syncopating, depending on the algorithm underlying the software. Another project involved the design of a virtual keyboard and guitar fret. Still another simulated a ball rolling on a fractal landscape, with the ball’s motion translated into musical sounds.

This integration of music, computation, and mathematics, Hudak notes, allows the students to deepen their understanding of all three areas of study and to consider the interdisciplinary nature of musical technology.  “My goal was to point out the mathematical and computational nature of music,” offers Hudak, “and then let the creative juices of the Yale students take over. The possibilities are endless.”

Jump to top.

The Educational Technologies Newsletter is published periodically to feature examples of how Yale faculty and students are using technology in teaching and learning. The examples will usually be activities involving our four units: the Instructional Technology Group, the Student Technology Collaborative, the Film Study Center and the Statlab.