AI can be made to understand written text, see what's inside an image, and understand the context within. Google wants to go further.
Google has what it calls the Google Lens, which is an AI-powered recognition tool that uses advanced machine learning and computer vision to identify and analyze visual information. It allows users to search using images instead of text, offering useful results and actions based on what it "sees" through the camera.
With it, users can copy text from an image, translate it, or search for related content, compare products and find similar alternatives, get information for various landmarks, identify species of plants and animals, and more.
And thanks to generative AI, users can have actionable results.
This time, Google Lens is getting an AI upgrade, in which users can also search with videos.
The company is slowly rolling out what it calls the 'Search with Video' for Google Lens, a feature that it originally announced during the Google I/O event.
Google has updated its blog post about generative AI to highlight this feature.
Thanks to advancements in video understanding, we’re able to take visual search to a whole new level, with the ability to ask questions with video.
Maybe you bought a record player at a thrift shop, but it’s not working when you turn it on and the metal piece with the needle is drifting unexpectedly. Searching with video saves you the time and trouble of finding the right words to describe this issue, and you’ll get an AI Overview with steps and resources to troubleshoot.
Before this, Google Lens visual capabilities only worked with images.
Users could upload pictures from their phone's gallery or use your device's camera to do contextual searches. However, sometimes an image alone won’t be enough to describe what you want, so you’ll need to add context.
Google Lens is already a powerful tool, but since the AI boom that happens after OpenAI released ChatGPT, Google knows that it has to move fast.
The company has since been working hard to improve its reliability and capability, massively improved Google Lens since its original release in 2017.
Google Lens is getting a “Search with video” option to offer even more search possibilities.
For example, the recent introduction of generative AI now provides fully AI-generated responses to uploaded images.
Additionally, Google Lens has been integrated into the widely used Circle to Search, which is continuously adding new features. Now, Google has announced that the Search with Video feature has begun rolling out to Android devices.
This feature leverages AI to analyze the content of videos and employs generative AI to interpret users' intentions based on the context.