Music Hack Day
Music Hack Day was an international 24-hour event where programmers, designers, and artists from all over the country came together in Philadelphia to conceptualize, build, and demo the future of music. Software, hardware, mobile, web, instruments, art - anything went as long as it was music related.
It was both a fun and challenging weekend with 24 straight hours of hacking fueled by lots of coffee, as well as an awesome performance by Philly's own Dirk Quinn Band that kept the momentum going through the night. There were twenty-five hacks that came out of the weekend, and even more that were not yet ready for prime time.
Below is a sampling of the projects from Music Hack Day, each with a description written by the team members who worked on them.
Multi-Voice Jazz Companion
The app is built on isolated multitrack recordings of a jazz ensemble. In order to display the audio using a piano roll visualization, audio to MIDI conversion can be done using a DAW such as Pro Tools for a monophonic instrument like a bass guitar, or using Melodyne for polyphonic interments such as chords for the guitar voice.
During Music Hack Day, we tried another way to generate the piano roll. We analyzed the acoustic features for each instrument voice in the composition directly from the audio stems using the Echonest API named "track". The analysis of the audio gives a list of segments of time within the song, and data is given for each segment. We used the information about pitch, start time of the segment, duration of the segment, and loudness in order to identify each note in the voice and then plot it on a piano roll. The pitch estimation is given by the API via a chroma analysis, so we only have notes in a single octave. Finally, the data is plotted based on the current section for visualization of the musical and acoustic properties during playback.
The next steps for this application are to create a drums visualization that will display rhythmic patterns using the start times of the segments and the loudness of the segment. Moreover, we would like to generalize the app and open it to any song that provides a multi-track recording. Instruments could be added to a library, the user could enter the different sections of the song, and comments on specific events in the song and the lead sheet could be generated digitally.
- Julie Borgeot, Undergradute, Electrical Engineering
The Melodic Twit
The Melodic Twit lets Twitter users hear their tweets sonified based on the sentiment of their comments. A positive tweet will create a happier sounding lyric and a negative tweet will generate a slower, sadder lyric. A melody is selected based on the analysis of the phrase sentiment and the words are sung using a text to speech engine. The audio is then posted on Soundcloud and a link to the audio is retweeted at the person who originally sent it.
- Jeff Scott, Ph.D. Electrical Engineering
I worked with a Philadelphia high school student, Frankie, to create a robot DJ. DARwIn, a mini-humanoid, mixes songs by pressing buttons on a digital controller pad. During the songs, he performs a choreographed dance until it's time to switch to the next song. I went into Music Hack Day knowing that I wanted to learn to use DARwIn, but with no other ideas on what to do. Frankie developed the choreography, staging, and video editing for the hack while I programmed the robot.
- Alyssa Batula, Ph.D. Electrical Engineering
Magnetic Resonator Piano Control Using the Microsoft Kinect
The Magnetic Resonator Piano (MRP) is currently configured to receive Open Sound Control (OSC) messages as a method of controlling audio synthesis parameters. This configuration and flexible messaging format makes possible a wide range of controllers that can actuate piano strings remotely, and independent of the piano keyboard. For example, we can do pitch tracking on other instruments and induce the piano strings to vibrate sympathetically with the performing instrument.
The system uses a Microsoft Kinect to track a user's joint positions for mapping to note allocation and real-time synthesis parameters. The final product divides the view into twelve regions, each corresponding to a note on a pentatonic scale. The position of the users feet in each region generates the notes with a two-note polyphony, while opening and closing the user's arms controls note volume. Additionally, the user can modify the brightness (harmonic weights) of a note by raising the right knee.
- Jeff Gregorio, Ph.D. Electrical Engineering
Sports Soundtrack DJ Interface
I worked with Simon Doolittle on developing a system that could synchronize an audio track to a live sporting event using acoustic data. This system listens to the sporting event and attempts to determine how 'interesting' the action on the field is. It then uses a phase vocoder to speed up or slow down a music track correspondingly. We chose table tennis as our sport because the sound of the ball bouncing on the table or paddles could be clearly picked up by the arena's microphones and so could be used as input to our system. I had a good time at Hack Day and feel that I learned a lot about rapidly prototyping new musical interfaces.
- David Grunberg, Ph.D. Electrical Engineering
Hubo Dance Choreographer
- William Hilton, Ph.D. Electrical Engineering
For Music Hack Day I worked on a single project aimed at extending the magnetic resonator piano. Old style player pianos required a physical paper roll that controlled a pneumatic system which drove the hammers, but with the MRP the strings have the option of being controlled programmatically. So my goal was to take advantage of this, and take the output from any audio source and encode that over a network so that the piano can recreate the the signal and in essence, act as a speaker.
The experience of Music Hack Day as a whole was great. There were so many people from all different fields of study and interests that there was always someone to talk to about something new and interesting. I met plenty of people that I otherwise would have never had a chance to talk to, and saw many hacks that I would have never even thought of. All-in-all it was great and I would love to participate again in the future.
- Matthem Zimmerman, BS/MS Electrical Engineering
Echo Nest presented two prizes. A fourth generation iPad and one of the coveted Echo Nest sweatseedos were presented to Multi-voice Jazz Companion App (Julie Borgeot, Ray Migneco). A second sweatseedo was presented to Are You Happy Now? (Benjamin Rose).
SoundCloud presented free airfare to the next Music in the US to Mend-a-break (Brian McFee).
Rdio presented one year subscriptions to All That Philly Jazz (Faye Anderson, Mark Headd, Mike Lamond) and Automatic Reception (Dave DesRoches)
SendGrid presented the Sphero ball robot to Internet Hubo (Will Hilton, Kunal Batra)
TokBox - PlaylisTable (Teja Vishwanadha, Johnny Megahan)
So far, Music Hack Day has taken place in over 12 cities around the world and hosted more than 2000 participants.
Sponsors included The Echo Nest, Spotify, SoundCloud, tokbox, Rdio, musiXmatch, Gracenote, Fame House, SendGrid, MailChimp. Participating companies included The Echo Nest, Spotify, SoundCloud, tokbox, Rdio, musiXmatch, Gracenote, and Free Music Archive.
Music Hack Day Homepage