When we think about the internet and the World Wide Web, we often imagine an invisible world that is populated by humans that have their eyes glued to various screens sizes of desktop computers, laptops, smartphones and tablets.
But that isn't necessary true as humans are again losing to bots.
According to a report by the cloud-based application delivery platform Incapsula, bots were winning for years before humans caught up in 2015. That year, human population on the web was more than 50 percent. But in 2016, bots were again sitting at the top with its traffic increased to 51.8 percent, or only slightly higher than Incapsula's report for 2012.
Breaking those bots down, there are good bots and bad bots:
Good bots are those designed to enhance web browsing experience. They include: monitoring bots (1.2 percent), commercial crawlers (2.9 percent), search engine bots (6.6 percent) and feed fetchers (12.2 percent).
Bad bots on the other hand, are created to steam contents, overwhelm web servers, confuse web analytics and more. They include: impersonators (24.3 percent), scrapers (1.7 percent), spammers (0.3 percent) and hacker tools (2.6 percent).
It's estimated that the bad bots cost businesses an estimated $7 billion in lost of revenue per year due to ad fraud.
Bots are essentially software applications designed to do repetitive process, used for many tasks involving the web. They are used by search engines in doing their "crawling" business to spammers in doing their schemes; bots are the digital ecosystem that we can't avoid.
As they become better and better, they can perform automated tasks with many of them as good as humans.
The report from Incapsula was from its observation on traffic trend by analyzing 16.7+ billion visits to 100,000 randomly-selected domains on its network from August 9, 2016 to November 6, 2016.
The good news about the report is that, the increase in bot traffic was due to an increase in good bots' activity. In 2016, Incapsula tracked 504 unique good bots - 278 of which were active enough to generate at least 1,000 daily visits to its network. 57.2 percent of them had an increase in activity, while 29.4 percent saw their activity decreased year-over-year.
The dominant good bots were from Google and Facebook. Together, Google's spiders and Facebook's feed fetcher accounted to over 8 percent of all traffic.
While bad bot traffic remained nearly the same in 2016 if compared to 2015, they continued to become the most active offenders. For the past five years, one out of three visitors of a website was an attack bot. In 2016, they were responsible for 28.9 percent of all traffic.
Out of 100,000 domains Incapsula surveyed, 94.2 percent of them experienced at least one bot attack over the 90 day period.
The increase in bots traffic activity is largely the result of a shift in human behavior that are moving towards mobile browsing. According to Incapsula, the trend shift saw an increase of 10.6 percent in mobile user website traffic and a decrease of 16 percent from desktop.
Facebook's feed fetcher bots are the most active, followed by Android's framework using the agent of Dalvik and Android Runtime processes. Apple's CFNetwork bot used in iPhone mobile apps came third.
Overall, bots associated with mobile apps and services account for over 69 percent of all feed fetcher traffic.