An awesome HTML5 interactive music video

Wdd Logo.
December 09, 2011
Interactive Music VideoDesigners and developers are pushing the envelope with what's possible in interactive design on a continuous basis. And awesome new examples come out all the time. One of the newest examples is an interactive music video for Evelyn, by ABBY. It's a fantastic site that gives you the opportunity to mix different instruments and vocal styles together, while the song is playing, for a completely custom experience. We asked the developers how they created such an awesome interactive video, and got their tips for working on projects of this type.

1. Where did the idea for the video come from? What was the creative process?

Although we were not directly involved in the creative process we will give you a short summary on how it was created. We, Steffen & Dominik, just founded a web development agency in Berlin called bleech specializing in HTML5 and other modern web technologies. We have our office in a shared space together with a recording studio which is run by the members of the band ABBY. The space is completed by a booking agency and an iOS agency. Most of us have known each other since college and we have worked together since then on many different projects in many diverse and creative environments. The actual idea for the video was developed by a friend of us who studied design. He needed a topic for his thesis and had the vision of creating an interactive experience which lets the user experiment with the different components of a modern song.

2. Can you give a quick overview of the process for actually creating the video, the steps involved, etc.?

One of the biggest challenges for the band was to select the adequate instruments which differ in their sound and characteristics but at the same time have a pleasant sound and harmonize with each other. Additionally they had to ensure that the rhythms and harmonics of the newly integrated tracks did not, at any time, go against each other. We believe they did a really great job on this. The entire idea was developed without dealing with any technical requirements regarding web technology. So they started recording one video for each audio track in a historic sound studio in Berlin. This resulted in a total of 20 individual videos, which then had to be merged to make the idea become reality. For this task they asked a befriended Flash developer to make a website where you can control the different tracks and mix your own version of the song. Unfortunately, he faced some difficulties with the syncing process over a network connection, so that early draft never made it to a final release. That's when we came in. We felt challenged to prove that it was possible to bring their idea to life with the latest technologies that nobody has yet used in this exact way. We started building the foundation in a test-driven JavaScript environment to ensure a consistent communication between the videos, our mediacontrollers, and the global timeline module. The mediacontrollers take care of switching the videos and displaying only the currently selected track. The timeline module serves as a reference time for each mediacontroller and syncs them if necessary.

3. What unexpected challenges presented themselves during the project? What advice would you give a developer who wanted to create a project like this?

A tough part in the development process was to keep the videos in sync without doing to many computations and to ensure that even on older computers a good, responsive user experience is possible. The methods we found out to be most effective were an aggregation of several algorithms which keep the videos in sync and adapt to the machine's performance by increasing the threshold and the frequency of how often the syncing is triggered. The biggest challenge though was fine tuning in the millisecond area, as for a song with 120 beats per minute, an offset of 50ms of any of the tracks would clearly be noticed by the listener. Finally, we managed to get all of the audio and video tracks in sync by less than 10ms on high performance computers (like a 2011 MacBook Pro / Air). If you're planning on developing a media-driven HTML5 project, be prepared for sleepless nights optimizing small pieces of code, unforeseen browser bugs, and a million possible ways to implement a single function.

4. Where do you see this kind of content going in the next couple of years?

We hope that more developers start to experiment with media-related web projects and hope to see new frameworks being created for media rich applications. Up to this point Java (processing) and Flash still have some advantages for certain use cases. The most prominent developments in HTML5 at the moment are definitely the audio and video capabilities and we are looking forward to features like the mediacontroller or device element being implemented in new browsers. With modern server technologies such as websocket, we are waiting to see the delivery of real-time events to the user instead of only serving static, prerecorded content. Additionally, it would be great to see more large scale interactive content that integrates social experience with existing media channels. The usual stuff that everybody is talking about right now... We are glad to be part of this project and to work with great artists with various expertise. The project has become our little baby and we are really overwhelmed by the positive response. BTW, we're looking for people to support our growing team. What other awesome HTML5 projects have you seen recently? Let us know in the comments!

WDD Staff

WDD staff are proud to be able to bring you this daily blog about web design and development. If there's something you think we should be talking about let us know @DesignerDepot.

Read Next

15 Best New Fonts, June 2024

Welcome to our roundup of the best new fonts we’ve found online in the last month. This month, there are notably fewer…

20 Best New Websites, June 2024

Arranging content in an easily accessible way is the backbone of any user-friendly website. A good website will present…

Exciting New Tools for Designers, June 2024

In this month’s roundup of the best tools for web designers and developers, we’ll explore a range of new and noteworthy…

3 Essential Design Trends, June 2024

Summer is off to a fun start with some highly dramatic website design trends showing up in projects. Let's dive in!

15 Best New Fonts, May 2024

In this month’s edition, there are lots of historically-inspired typefaces, more of the growing trend for French…

How to Reduce The Carbon Footprint of Your Website

On average, a web page produces 4.61 grams of CO2 for every page view; for whole sites, that amounts to hundreds of KG…

20 Best New Websites, May 2024

Welcome to May’s compilation of the best sites on the web. This month we’re focused on color for younger humans,…

Has AI Killed User Testing?

Web designers employ user testing to evaluate a website’s functionality and overall UX (user experience). Various…

Exciting New Tools for Designers, May 2024

This year, we’ve seen a wave of groundbreaking apps and tools. AI is reshaping the industry, enhancing productivity,…

Using AI to Predict Design Trends

Design trends evolve at a blistering pace, especially in web design. On multi-month projects, you might work on a…

15 Best New Fonts, April 2024

Just like web design, type design follows trends. And while there’s always room for an exciting outsider, we tend to…

3 Essential Design Trends, May 2024

Integrated navigation elements, interactive typography, and digital overprints are three website design trends making…