The app is built on isolated multitrack recordings of a jazz ensemble. In order to display the audio using a piano roll visualization, audio to MIDI conversion can be done using a DAW such as Pro Tools for a monophonic instrument like a bass guitar, or using Melodyne for polyphonic interments such as chords for the guitar voice.

During Music Hack Day, we tried another way to generate the piano roll. We analyzed the acoustic features for each instrument voice in the composition directly from the audio stems using the Echonest API named "track". The analysis of the audio gives a list of segments of time within the song, and data is given for each segment. We used the information about pitch, start time of the segment, duration of the segment, and loudness in order to identify each note in the voice and then plot it on a piano roll. The pitch estimation is given by the API via a chroma analysis, so we only have notes in a single octave. Finally, the data is plotted based on the current section for visualization of the musical and acoustic properties during playback.

The next steps for this application are to create a drums visualization that will display rhythmic patterns using the start times of the segments and the loudness of the segment. Moreover, we would like to generalize the app and open it to any song that provides a multi-track recording. Instruments could be added to a library, the user could enter the different sections of the song, and comments on specific events in the song and the lead sheet could be generated digitally.

- Julie Borgeot, Undergradute, Electrical Engineering

The Melodic Twit lets Twitter users hear their tweets sonified based on the sentiment of their comments. A positive tweet will create a happier sounding lyric and a negative tweet will generate a slower, sadder lyric. A melody is selected based on the analysis of the phrase sentiment and the words are sung using a text to speech engine. The audio is then posted on Soundcloud and a link to the audio is retweeted at the person who originally sent it.

- Jeff Scott, Ph.D. Electrical Engineering

I worked with a Philadelphia high school student, Frankie, to create a robot DJ. DARwIn, a mini-humanoid, which mixes songs by pressing buttons on a digital controller pad. During the songs, he performs a choreographed dance until it's time to switch to the next song. I went into Music Hack Day knowing that I wanted to learn to use DARwIn, but with no other ideas on what to do. Frankie developed the choreography, staging, and video editing for the hack while I programmed the robot.

- Alyssa Batula, Ph.D. Electrical Engineering

The Magnetic Resonator Piano (MRP) is currently configured to receive Open Sound Control (OSC) messages as a method of controlling audio synthesis parameters. This configuration and flexible messaging format makes possible a wide range of controllers that can actuate piano strings remotely, and independent of the piano keyboard. For example, we can do pitch tracking on other instruments and induce the piano strings to vibrate sympathetically with the performing instrument.

The system uses a Microsoft Kinect to track a user's joint positions for mapping to note allocation and real-time synthesis parameters. The final product divides the view into twelve regions, each corresponding to a note on a pentatonic scale. The position of the users feet in each region generates the notes with a two-note polyphony, while opening and closing the user's arms controls note volume. Additionally, the user can modify the brightness (harmonic weights) of a note by raising the right knee.

- Jeff Gregorio, Ph.D. Electrical Engineering

I worked with Simon Doolittle on developing a system that could synchronize an audio track to a live sporting event using acoustic data. This system listens to the sporting event and attempts to determine how 'interesting' the action on the field is. It then uses a phase vocoder to speed up or slow down a music track correspondingly. We chose table tennis as our sport because the sound of the ball bouncing on the table or paddles could be clearly picked up by the arena's microphones and so could be used as input to our system. I had a good time at Hack Day and feel that I learned a lot about rapidly prototyping new musical interfaces.

- David Grunberg, Ph.D. Electrical Engineering

- William Hilton, Ph.D. Electrical Engineering

For Music Hack Day I worked on a single project aimed at extending the magnetic resonator piano. Old style player pianos required a physical paper roll that controlled a pneumatic system which drove the hammers, but with the MRP the strings have the option of being controlled programmatically. So my goal was to take advantage of this, and take the output from any audio source and encode that over a network so that the piano can recreate the the signal and in essence, act as a speaker.

The experience of Music Hack Day as a whole was great. There were so many people from all different fields of study and interests that there was always someone to talk to about something new and interesting. I met plenty of people that I otherwise would have never had a chance to talk to, and saw many hacks that I would have never even thought of. All-in-all it was great and I would love to participate again in the future.

- Matthew Zimmerman, BS/MS Electrical Engineering