The internet has the luxury of providing businesses the ability to customize and personalize experiences to their customers. This is something no offline vendors, like brick-and-mortal venues can provide.
On the web, one of the very best that can provide such customization and personalization, is the social giant Facebook.
With billions of users with different tastes and interests to serve, the social media is able to provide unique experiences to each and everyone of them through a complex personalized ranking system.
What the social media did, is using its massive content posted on its platform by other users, run them through a series of complex algorithms, and deliver only those that it thinks is best for their tastes and interests.
This is proven to be a complex challenge.
But Facebook managed to tackle this, almost flawlessly and in real time, using its News Feed Ranking.
And this ranking system, is heavily reliant on machine learning technology, powered by AI.
"We publicly share many of the details and features of News Feed. But under the hood, the machine learning (ML) ranking system that powers News Feed is incredibly complex, with many layers. We are sharing new details about how our ranking system works and the challenges of building a system to personalize the content for more than 2 billion people and show each of them content that is relevant and meaningful for them, every time they come to Facebook."
Facebook tries to clearly explain how its algorithms work for its more than 2 billion users.
The company said that without using AI, its News Feed could be flooded with content users don’t find as relevant or interesting, including overly promotional content or content and others. Facebook News Feed Ranking exists to help solve these problems.
Using machine learning, Facebook can predict which content matters most to each person.
Here is how Facebook describes it in a Facebook Engineering post:
Because Juan is connected to or has chosen to follow the producers of this content, it’s all likely to be relevant or interesting to him. To rank some of these things higher than others in Juan’s News Feed, we need to learn what matters most to Juan and which content carries the highest value for him. In mathematical terms, we need to define an objective function for Juan and perform a single-objective optimization.
Take Saanvi’s running video, for example. On Facebook, one concrete observable signal that an item has value for someone is if they click the like button. Given various attributes we know about a post (who is tagged in a photo, when it was posted, etc.), we can use the characteristics of the post Xit toward viewer j at time t, and predict Yijt (whether Juan might like the post). Mathematically, for each post i, we estimate Yijt = f(xijt1;xijt2; … xijtC), where c represents a characteristic c (1..C) such as the type of post or the relationship between the viewer and the author of the post (e.g., whether they marked each other as family members) and the function f(.) combines the attributes into a single value.
For example, if Juan tends to interact with Saanvi a lot or share the content Saanvi posts, and the running video is very recent (e.g., from this morning), we might see a high probability that Juan likes content like this. On the other hand, perhaps Juan has previously engaged more with video content than photos, so the like prediction for Wei’s cocker spaniel photo might be lower. In this case, our ranking algorithm would rank Saanvi’s running video higher than Wei’s cocker spaniel photo because it predicts a higher probability that Juan will like that piece of content.
But things don't stop there.
Liking isn't the only way users can express their preferences. In other cases, Facebook can include other inputs, like the articles a person shared, the videos the person watched, the comments the person left on posts, and so forth.
"Things get more mathematically complicated when we need to optimize for multiple objectives that all contribute to our overarching objective," Facebook continued.
"To complicate things further, for each person on Facebook there are thousands of signals we need to evaluate to determine what that person might find most relevant, so the algorithm gets very complex in practice."
Facebook here, focuses on long-term value.
To do this, Facebook picks the metrics that are aligned with what people say is important to them.
With multiple prediction models, the algorithm will try to rank each of the pieces for each user.
And after finding the approximate ideal ranking function in a scalable ranking system, the next thing Facebook did, is determining how to build a system to optimize the flow.
Here, Facebook needs to score all of the posts available for more than 2 billion people with more than 1,000 posts per user, per day, on average.
To manage this huge feat in real time, Facebook's architecture uses a Web/PHP layer, which queries the feed aggregator.
When someone opens up Facebook, regardless of the front-end interface (e.g., iPhone, Android phone, web browser), every time, the interface will send a request to a Web/PHP (front-end) layer to then send the queries back to the feed aggregator (back-end layer).
The role of the feed aggregator is to collect all relevant information about a post and analyze all the features (e.g., how many people have Liked this post before) in order to predict the post’s value.
The ranking system then goes to work by first querying the inventory, to then score the posts for users for each prediction, and then calculate a single score out of many predictions
"Once we’ve completed these ranking steps, we have a scored News Feed for Juan (and all the people using Facebook) in real time, ready for him to consume and enjoy," said Facebook.
Unlike Google which has to crawl the wilderness of the web to rank websites, Facebook operates inside its walled-garden.
What Facebook is saying here is that, its ranking system isn't comprised of a single algorithm.
Instead, it uses multiple layers of machine learning models to predict what each individual user wants to see.
The ranking system collects as much information as it can, including from candidate posts for each user, those shared by their friends, Groups, or Pages since their last login, and so on.
The ranking system can then give each post a score based on a variety of factors, such as who shared the content and how it matches with what the user generally interacts with.
AI can then narrow the candidates to a shorter list, and other algorithm will work to give each remaining post a score to determine in which order they should appear in the News Feed.
Finally, the system adds contextual features like diversity rules to ensure that the News Feed has a variety of content.
The entire process is complete in the time it takes to open the Facebook app, no matter what device users use.
This feat is an accomplishment on its own, considering how huge Facebook is, how many users Facebook has to serve simultaneously, and how fast it's capable of delivering the contents in a very minimal time with no noticeable lag.
Facebook has indeed grown a lot larger than what it was back in Mark Zuckerberg's dorm room. Powered by sophisticated engineering, huge computing power and tons of resources, Facebook is capable of delivering the capacity it promises, but minus the privacy it lacks here and there.