Popular web apps should find it difficult to understand user behavior that changes frequently. For Facebook, this matter is more of a problem that just a headache.
With more than 2.6 billion users, testing a proposed code update or new feature is a complex and challenging task.
People's behavior evolves over time, and is different from one country or region to the next. This makes it difficult for traditional algorithms to understand or replicate, making it also difficult for Facebook to anticipate what users will respond to even the smallest change to the Facebook experience.
In a place where a traditional A/B test is not anymore sufficient, the researchers at the social giant have developed what they call a Web-Enabled Simulation (WES). This method, according to Facebook, is "the first highly realistic, large-scale simulations of complex social networks."
In a blog post, Facebook said that:
According to the research paper, WES is a simulation of the behavior. It has three important aspects:
- WES uses machine learning technology to train bots to realistically simulate the behaviors of real people. These bots are trained to mimic human behavior when interacting with Facebook, including the ability to send messages to other bots, comment on other bot's post, or even publish its own, as well as making friend requests.
These bots only engage with other bots, and not with real users. What this means, their behavior cannot have any impact on real users or their experiences on the platform.
- WES is able to automate interactions between thousands or even millions of bots.
Facebook uses a combination of online and offline simulation, to train the bots with anything from simple rules and supervised machine learning to more sophisticated reinforcement learning. This should provide Facebook with simulation data that includes efficiency and effectiveness of something, rather than the traditional engineering concerns, like speed, scale and realism.
- WES deploys these bots on the platform’s actual production code base. Here, the bots can interact with one another, but are isolated from real users.
Here, the company’s AI team has came up with a way to make sure that software updates and tweaks won't screw up its platform and cost it any users, by building a fake Facebook-scale social media network full of bots to test things out on.
In other words, Facebook has created a copy of Facebook that is filled with trained bots.
While the research's goal is to benefit Facebook when it wants to deliver new features or functionalities, creating WES is already a scientific challenge on its own.
"For example, to build bots that behave realistically and intelligently, we leveraged a combination of technologies and published research on a wide variety of topics, including search-based software engineering, machine learning, programming languages, multiagent systems, graph theory, game AI, and AI-assisted gameplay," wrote Facebook.
To speed up the progress, the company is seeking input from others in the research community, by inviting them to contribute new ideas to WES and the initial prototype called WW.
"We received 85 submissions from 17 countries around the world. The interest, breadth, and depth of the research proposals are a testament to the way in which the WES research agenda touches on so many important and exciting research topics. We can only hope to answer a fraction of these open scientific questions ourselves. This is why we are so keen to partner with the scientific research community to help advance our collective understanding of these topics," added Facebook.
Humans are characterized as social creatures that interact with what's in presented to them in different ways, affected by region or country, gender, age, religion, experience and knowledge, traits and many more.
In any large-scale systems, they will see that the different people will respond differently to different things. And that response will also evolve and adapt, meaning that things will continuously change.
This is a complex thing to understand, and with WES (and WW), Facebook hopes that at least some of those complex interactions could be predicted.
"With the WES method and our WW simulator, we’ve built a better way to address this important but difficult challenge. This will help us make our platforms safer, more stable, more robust, and performant. And by sharing our work and providing grants to academic researchers, we also hope WES will help others develop new ways to model these sorts of complex situations."