Searching Beyond Google: When The Internet Is Too Big For A Single Search Engine

Billions of people are connected to the internet, and each day, many of them contribute to the ever-increasing number of information the web has.

Because the internet is overflowing with information, it has become one of the greatest place to find data and for research. However, since it has grown a lot bigger than the previous day, in each coming days, no matter how good you are with Google, you cannot find everything.

First of all, Google may not show the things you want, due to its ranking system and algorithms. In other cases, the information is hidden, and inaccessible by Google. Some websites have even opted to stay our of Google' sight, making many other information difficult to find.

If Google can't show you want you want, you need to take your search attempt to the next level.

What this means, you're digging deeper into the web using other resources that the internet has to offer.

Using Other Search Engines

Yes, Google is the most obvious search engine for most people. But the thing is, results show up differently on different search engines.

For example, Google is great in showing results based on popularity in general. But one method it's able to do that, is by collecting its user' information. If you're a Google user, you may find your results tailored to your specification or interest.

While this is a good thing, you may miss other information.

To get into more things that Google has hidden far from you (lower on its search results pages), you may try alternative search engines, such as Microsoft's Bing. Another option is using the privacy-conscious search engine DuckDuckGo.

You can also use more specific tools for specific searches. For example, you can use Wolfram Alpha if you want to search for bits of information. Alpha excels in comparison and mathematical formulas, for example.

If you're seeking for public posts, you can use Boardreader. The search tool crawls public forums and message boards differently than Google, making it able to surface some things that you may not find in Google.

There are many other search engines to choose, and all may show different results for the same query.

And for last, you can also try the Internet Archive. What it does, is caching web pages, including images, articles and software. So if the case the information has been deleted from a website, Internet Archive may have cached it.

Use Social Media

Other than search engines that crawl and index, social media is the source of a huge amount of information.

Search engines don't have the ability to crawl every post on social media. Since social media kept adding up contents by its billions of users, older and not popular posts may get buried far from search engines' reach. To search for those posts, you may as well use the social engine's search feature.

Notable social media, such as Facebook and Twitter, both have search tools with advanced search capability.

Using filters, you can narrow your search based on location, date or other factors. You can also use their API for deeper searches.

Using these, you should be able to refine your query to find more relevant results.

Use The Community

If you can't find the information using the above methods, you may extend you search even deeper by scouting to pages created by communities and the government.

For example, some pages, like forum pages, are locked for unregistered users. To access them, you need to register yourself to the forum, or do other things to unlock those pages, such as posting several posts or liking others' posts first.

Some contents can also be hidden behind a paywall, requiring you to pay to get access to them.

Other things you can do, is checking directories of websites of educational institutions and universities. You can also scout for public records that are stored in documents. You can also visit websites relating to public records or run by the government that is publicly available, bit aren't indexed by search engines.

You can also look for academic documents from journal websites. To find the journal that may host the content, you can use Google or other search engines, ScienceDirect or Wikipedia.

Use The Deep Web

Not every information is on the "surface web." While most of the things people need is available using all the methods above, others can't be accessible by any of them.

To get into that information, you need to venture even deeper into the web, and into the deep web. Here, you need the Tor browser which enables you to go anonymous on the internet. Because of its anonymous nature, Tor has been used by a wide variety of people for both licit and illicit purposes.

For example, users include criminal enterprises, hacktivism groups, law enforcement agencies and many more.

So using Tor, you can scout the inner part of the web for information regarding illegal activities, getting access to censored information, data about political activities, words about criticisms on heads of state, acts and thoughts of criminalism and more.

Summary

If it's in the internet, it is there somewhere and it is there to see.

But the thing is, the information you need might be hidden from the tools you use. Either because the webmasters wanted that to happen by blocking search engines' access, or inaccessible by search engine because they are behind paywalls or login forms, or lies inside the deep web.

To access them, you need more specific tools other than search engines, and look at the right place, with the right privilege.

If and only if you still can't find the information you're looking for, it's either not available on the internet, or doesn't exist in the first place.

Or, they could be hidden inside database, encrypted, lies deeper and accessible to only using certain IPs, classified methods or privilege. In this case, it's not publicly available though it is there. You need to be a hacker to get yourself into that, and that is certainly illegal by any means.