Helping developers create smarter online music-streaming services

Jul 10, 2013 by Rob Matheson

If you ever use Spotify, or a similar music-streaming service, there's a good chance your song recommendations, and other personalized features, are powered by novel technology developed and marketed by two MIT alumni entrepreneurs.

Brian Whitman PhD '05 and Tristan Jehan SM '01, PhD '05 are co-founders of Echo Nest, whose technology—based on their MIT research—mines data from millions of songs streaming online. Sometimes called "the big data of music," the company has compiled about a trillion data points from 35 million songs by 2.5 million artists.

Its music-intelligence platform—recently praised in publications such as Fast Company, Wired and Business Insider, among others—then translates this data into information for music-app developers, who use the information to build smarter, more personalized music apps.

Now, as a leader in the music-intelligence industry, Echo Nest has dozens of big-name clients, including MTV, BBC, Rdio, VEVO, Foursquare, Nokia, Sirius XM, Clear Channel's iHeartRadio, Univision Radio and Intel.

The company also provides third-party developers with access to this data via an (API) that has become the technological blueprint for more than 400 apps, including iHeartRadio and eMusic.

"Early on, we always wanted an API for developers, instead of being this closed company, where only people who paid us could use it," Whitman says. "The point of that is to see what people can build on top of our data. And there's been some amazing things."

The co-founders say the company's success is due, in part, to technology that predicted the growth of today's booming online-—which ushered in a host of music-streaming sites and saw the growth of Internet radio. "When all that technology was rising around us, we were ready," Whitman says.

Combining music content and cultural analysis

The foundations of Echo Nest's technology trace back to the MIT Media Lab, where the co-founders, then doctoral students, decided to combine their dissertations on music-data mining.

Jehan's dissertation, which he conducted in the Hyperinstruments Group, focused on the "content analysis" of music, extracting data on musical elements such as tempo, key and time signature. Whitman's work—conducted under the tutelage of professor emeritus Barry Vercoe—looked at a "cultural analysis" of music, focusing on what different types of people were saying about music online.

Seeing technological and commercial potential in combining the two projects, the co-founders mixed and tweaked their studies—a content-based and cultural analysis of music—and created what Whitman calls "a big database of what music sounds like to a computer, and what it means to people."

Now, when someone uses a music-streaming app that utilizes Echo Nest's platform to, say, generate a playlist, Whitman says, "the site accesses both parts of the combined technology and says, 'Here are the songs you should be listening to based on what we know about you and the music.' At the end of the day, we tell people what music they should hear."

Most of the developer clients, for instance, use Echo Nest's data to better understand listeners' tastes and behaviors and create smarter music-streaming features, such as , playlist generation, taste profiling, acoustic analysis, acoustic fingerprinting (an audio sample used to identify songs) and data feeds.

But an additional perk of Echo Nest's massive database, the co-founders say, is that it can help increase the visibility of rising Internet musicians who may have slipped through the song-recommendation cracks of earlier music-streaming services. For instance, MTV's music- is using it to help listeners discover artists who may be popular on the Web, but who don't get radio play.

"We're both musicians, and it's frustrating knowing that an independent artist may not get noticed in music-streaming sites. We wanted to change that," says Whitman, who recorded as an electronica musician before starting Echo Nest. Jehan is a keyboardist and guitar player who used to play in a Boston-based Brazilian band.

From scientists to entrepreneurs

In the company's early days, the co-founders say they found support through MIT's Venture Mentoring Service (VMS) and the MIT Media Lab, which helped turn them from scientists to entrepreneurs.

Meeting regularly with business mentors such as Roman Lubynsky, VMS's senior venture advisor, the two learned the basics for growing a company and were introduced to a variety of contacts, including lawyers, accountants and investors. "It was a very connected culture," Jehan says.

The co-founders say the MIT Media Lab also helped them make their technology accessible to investors—something foreign to some scientists, Jehan says. "Technology is not a product in itself," Jehan says. "Some people don't get that. The technology can be artistic, but you have to create artifacts people can grasp. We learned how to make it accessible to investors, or 'productize' it."

Whitman agrees, adding that the MIT Media Lab helped with patents and other legal issues. The experience taught him how to pitch ideas to the business community—something that helped Echo Nest acquire its initial investors.

"As a scientist, being forced to explain your work to someone who's not a scientist was a valuable lesson," Whitman says.

As scientists who freely accessed data for their dissertations, Whitman and Jehan have made sure to pay it forward, making some of Echo Nest's data and technologies readily available for research purposes. In 2011, the company released a million-song dataset to academic institutions and released Echoprint, an open source music-identification system.

"We come from the research world, and having access to data was really important," Whitman says. "So, we're trying to make sure that stays alive in our world."

Explore further: Modeling the ripples of health care information

add to favorites email to friend print save as pdf

Related Stories

Streaming music gains in US market

Apr 02, 2013

Streaming music services like Pandora online radio are gaining fast among US listeners under age 35, a survey showed Tuesday.

In the mood for music

Jun 27, 2013

Could a computer distinguish between the moods of a mournful classical movement or an angst-ridden emo rock song? Research to be published in the International Journal of Computational Intelligence Studies, suggests that i ...

Google's music plan part of fresh wave of upgrades

May 16, 2013

Google Inc. unveiled a streaming music service called All Access that blends songs users have already uploaded to their online libraries with millions of other tracks for a $10 monthly fee. The service puts the Internet ...

New study points to increased incomes from music streaming

Jun 04, 2013

The issue of intellectual property rights in the music industry remains a hot topic, and the debate seems to intensify every time technological advances are made. A new doctoral thesis from the School of Business, Economics ...

Recommended for you

Forging a photo is easy, but how do you spot a fake?

Nov 21, 2014

Faking photographs is not a new phenomenon. The Cottingley Fairies seemed convincing to some in 1917, just as the images recently broadcast on Russian television, purporting to be satellite images showin ...

Algorithm, not live committee, performs author ranking

Nov 21, 2014

Thousands of authors' works enter the public domain each year, but only a small number of them end up being widely available. So how to choose the ones taking center-stage? And how well can a machine-learning ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.