An awesome HTML5 interactive music video

Default avatar.
December 09, 2011
Interactive Music Video

Designers and developers are pushing the envelope with what’s possible in interactive design on a continuous basis. And awesome new examples come out all the time. One of the newest examples is an interactive music video for Evelyn, by ABBY. It’s a fantastic site that gives you the opportunity to mix different instruments and vocal styles together, while the song is playing, for a completely custom experience. We asked the developers how they created such an awesome interactive video, and got their tips for working on projects of this type.

1. Where did the idea for the video come from? What was the creative process?

Although we were not directly involved in the creative process we will give you a short summary on how it was created. We, Steffen & Dominik, just founded a web development agency in Berlin called bleech specializing in HTML5 and other modern web technologies. We have our office in a shared space together with a recording studio which is run by the members of the band ABBY. The space is completed by a booking agency and an iOS agency. Most of us have known each other since college and we have worked together since then on many different projects in many diverse and creative environments. The actual idea for the video was developed by a friend of us who studied design. He needed a topic for his thesis and had the vision of creating an interactive experience which lets the user experiment with the different components of a modern song. 

2. Can you give a quick overview of the process for actually creating the video, the steps involved, etc.?

One of the biggest challenges for the band was to select the adequate instruments which differ in their sound and characteristics but at the same time have a pleasant sound and harmonize with each other. Additionally they had to ensure that the rhythms and harmonics of the newly integrated tracks did not, at any time, go against each other. We believe they did a really great job on this. The entire idea was developed without dealing with any technical requirements regarding web technology. So they started recording one video for each audio track in a historic sound studio in Berlin. This resulted in a total of 20 individual videos, which then had to be merged to make the idea become reality. For this task they asked a befriended Flash developer to make a website where you can control the different tracks and mix your own version of the song. Unfortunately, he faced some difficulties with the syncing process over a network connection, so that early draft never made it to a final release. That’s when we came in. We felt challenged to prove that it was possible to bring their idea to life with the latest technologies that nobody has yet used in this exact way. We started building the foundation in a test-driven JavaScript environment to ensure a consistent communication between the videos, our mediacontrollers, and the global timeline module. The mediacontrollers take care of switching the videos and displaying only the currently selected track. The timeline module serves as a reference time for each mediacontroller and syncs them if necessary.

3. What unexpected challenges presented themselves during the project? What advice would you give a developer who wanted to create a project like this?

A tough part in the development process was to keep the videos in sync without doing to many computations and to ensure that even on older computers a good, responsive user experience is possible. The methods we found out to be most effective were an aggregation of several algorithms which keep the videos in sync and adapt to the machine’s performance by increasing the threshold and the frequency of how often the syncing is triggered. The biggest challenge though was fine tuning in the millisecond area, as for a song with 120 beats per minute, an offset of 50ms of any of the tracks would clearly be noticed by the listener. Finally, we managed to get all of the audio and video tracks in sync by less than 10ms on high performance computers (like a 2011 MacBook Pro /​Air). If you’re planning on developing a media-driven HTML5 project, be prepared for sleepless nights optimizing small pieces of code, unforeseen browser bugs, and a million possible ways to implement a single function.

4. Where do you see this kind of content going in the next couple of years?

We hope that more developers start to experiment with media-related web projects and hope to see new frameworks being created for media rich applications. Up to this point Java (processing) and Flash still have some advantages for certain use cases. The most prominent developments in HTML5 at the moment are definitely the audio and video capabilities and we are looking forward to features like the mediacontroller or device element being implemented in new browsers. With modern server technologies such as websocket, we are waiting to see the delivery of real-time events to the user instead of only serving static, prerecorded content. Additionally, it would be great to see more large scale interactive content that integrates social experience with existing media channels. The usual stuff that everybody is talking about right now… We are glad to be part of this project and to work with great artists with various expertise. The project has become our little baby and we are really overwhelmed by the positive response. BTW, we’re looking for people to support our growing team. What other awesome HTML5 projects have you seen recently? Let us know in the comments!

WDD Staff

WDD staff are proud to be able to bring you this daily blog about web design and development. If there’s something you think we should be talking about let us know @DesignerDepot.

Read Next

15 Best New Fonts, May 2023

The choices you make when selecting a typeface have more impact on your design than almost any other decision, so it’s …

10+ Best Tools & Resources for Web Designers and Agencies (2023 updated)

Having the ability to envision a tastefully designed website (i.e., the role creativity plays) is important. But being …

20 Best New Websites, May 2023

This month, there are tons of great new agency websites to get excited about. 3D animated prisms are a popular theme, a…

How to Find the Right White Label Website Builder for Your Agency

Web design agencies face a lot of obstacles in closing the deal with new clients. One of the most common ones is the ar…

Exciting New Tools For Designers, May 2023

There are hundreds of new tools for designers and developers released each month. We sift through them all to bring you…

3 Essential Design Trends, May 2023

All three of the website design trends here mimic something bigger going on in the tech space, from a desire to have mo…

10 Best AI Tools for Web Designers (2023)

It’s time to stop worrying if AI is going to take your job and instead start using AI to expand the services you can of…

10 Best Marketing Agency Websites (Examples, Inspo, and Templates!)

Marketers are skilled in developing strategies, producing visual assets, writing text with high impact, and optimizing …

15 Best New Fonts, April 2023

Fonts are a designer’s best friend. They add personality to our designs and enable fine typography to elevate the quali…

20 Best New Websites, April 2023

In April’s edition, there’s a whole heap of large-scale, and even full-screen, video. Drone footage is back with a veng…

Exciting New Tools For Designers, April 2023

The AI revolution is having a huge impact on the types of products that are hitting the market, with almost every app b…

3 Essential Design Trends, March 2023

One thing that we often think about design trends is that they are probably good to make a list. That’s not always true…