The social network giant Facebook released an update to its mobile app on Wednesday, May 21st, 2014, that allow users to automatically identify the song or TV show they're watching, and put it on their Facebook posts. The new feature is the company's latest move to give prominence to media and entertainment on its social network.
It's nothing new for Facebook to replicate popular apps and use it for its own - Poke, for example, is Snapchat's and Follow is Twitter's. According to a post on the official Facebook blog, the social network is releasing a listening feature to enhance its Wall with its unique audio-recognition algorithm built entirely in-house from scratch.
The optional feature uses audio-recognition technology, similar to the way Shazam and SoundHound, that works by "listening" to the user's phone microphone to match the title, album and artists of millions of songs and up to 160 channels of TV programming. Users can select the option to turn on the audio listening feature when they compose a status on their iOS or Android app.
When the feature is turned off by default. Users need to turn on manually to enable the feature before using it. When it's turned on, Facebook will automatically listen to what the user is listening or watching, and provide the result when the user compose a post. The action when Facebook is listening is seen by an animated blue sound bar icon in the upper-right hand corner of the screen, or over the smiley icon at the bottom.
When the app detects a registrable song, it'll attach its album artwork, along with the artist and the title. A play button on the album artwork allows others to listen to a 30-second preview of the song within their feed using one of three music streaming services: Spotify, Rdio, or Deezer. If it’s a TV show, it'll include the title, season, and episode number, or other relevant details including the show's art, and a link to the show's Facebook page.
Aryeh Selekman, Facebook's Product Manager, has been working on this project since 2013. He said that he wanted to help add context for people who are posting about popular television like Game of Thrones, a show that Facebook CEO Mark Zuckerberg likes to watch.
"People are already sharing while they're watching Game of Thrones, whether it's through the activity button or just actually sharing it in a post," said Selekman. "It's unlikely that they're going to be like, 'I'm on episode four, season six,' but that context is actually really valuable. If you're a few episodes behind and you really wanted to have a conversation with your friends, you might be less likely to post. You don't want people to spoil future episodes in the comments. Here it's really clear where you are in the series."
The audio-recognition even works for live TV and new programs. Facebook said that it's able to recognize live sport events as well.
"This lets you add a soundtrack to your posts," said Selekman.
Selekman said that his team built this feature to work in lower volume scenarios that also copes with ambient noise like when people are talking in the background. Selekman said he tested the app on his recent trip to Vegas, and it seemed to hear through the sounds of the Casino's machines well.
"I think this is huge," said Selekman. "Especially for mobile, where you're kind of bound by your feed and you don't necessarily want to be taken out of context. If you're wondering if this is worth investing your time, you can just get a preview. And If you're digging it, you just tap anywhere else on the album art."
When asked how the technology worked, Selekman said that it functions at a millisecond level. "As audio comes into the device, it immediately gets converted into these unique codes that we can use to identify properties that's specific to the content... It then looks up the code in the database and finds a match."
The method is slightly different from Shazam. "For every Shazam, the application analyzes the audio and generates a tiny unique fingerprint based on the audio characteristics found within the sample," said Charles Henrich, Shazam's Executive Vice President of Engineering. "It then uses this fingerprint to search against our database of tens of millions of audio tracks until it finds a match."
Having an app that "listens" to your environment can be creepy. To obey users' privacy, Facebook assures users that its listening feature works as quickly as it can before turning off. The feature is only activated when the user opens the compose box for a post, and if it hasn't found anything after 15 seconds listening, it will automatically shuts off. The new data it takes from what it listens, and sounds beyond that 15 seconds, are not recorded or stored on Facebook's servers. Users also have full control over their posts and can decide whether to share what it detects, or not. Facebook also ensures that his new feature has little effect on the phone's battery life.
This update comes just over a year after the Feelings and Activities feature was released in April 2013. Selekman also said that the audio-recognition feature was influenced by it.
"We had been thinking about enabling the TV show and music matching," Selekman said. "Seeing how popular that feature was, we decided this seems like the right place to do it."
The new feature comes as Facebook is increasingly competing with Twitter to become the main online venue where consumers discuss television shows, sports and other entertainment in real time.