'Recall' Is Microsoft Windows 'Photographic' Memory That Gives Convenience And Privacy Concern


Computers are devices that compute users' input, and provide output. But things have evolved fast.

By giving computers a lot more sensors to churn in information from things around them, computers can become smarter. And by giving computers more and more processing power, and more and more storage, they can now become smart, and also capable of remembering everything.

In an attempt to push its AI venture to more users, Microsoft has included its generative AI product to its core business: Windows.

And this time, its AI computer is gaining one special feature, which is the ability to remember everything users do with their PC.

The "everything" here means pretty much everything.

Microsoft introduces 'Recall', a feature meant to help users find that webpage, document, or presentation they were working on more easily.

"With Recall, we’re going to leverage the power of AI and the new system performance to make it possible to access virtually anything you have ever seen on your PC," said Yusuf Mehdi of Microsoft.

To do this, Recalls takes continuous snapshots of users screen throughout the computer's usage, and make them searchable, either through a timeline or by a keyword.

"Based on research that we’ve done, people spend one hour a day at work just trying to find things," said Matt Barlow, who runs Surface marketing for Microsoft.

“Recall gives you basically a virtual photographic memory of everything you’ve ever done on your PC."

According to reports, Recall takes screenshot of every single thing users see on their screen, no matter which program they're running, or using.

The system doesn’t take screenshots at the same interval every time, but according to the engineers, it uses various signals to understand when users do something different and that triggers a snapshot.

And because snapshots are indexed, this allows Microsoft to use AI to make them searchable.

For example, users who browsed for an image of a brown bag, and later, totally forgot about it, can simply type "brown bag" and resurface a screenshot of that.

From the images to the URL, to the words on the page, Recall can recall everything.

By performing a Recall action, users can access a snapshot from a specific time period, providing context for the event or moment they are searching for.

It also allows users to search through teleconference meetings they've participated in, and videos watched using an AI-powered feature that transcribes and translates speech.

"Once you find the snapshot that you were looking for in Recall, it will be analysed and offer you options to interact with the content. What actions you can take depend on the content and the chat provider capabilities in Copilot in Windows. For example, you may highlight a block of text and decide to summarise it, translate it, or open it with a text editor like Word or Notepad. If you highlight an image, you will be able to edit it or use your chat provider in Copilot in Windows to find or create a similar image," Microsoft explains.

"Recall will also enable you to open the snapshot in the original application in which it was created, and, as Recall is refined over time, it will open the actual source document, website or email in a screenshot."

Long story short, Recall is like a time machine.

"It's not keyword search, it's semantic search over all your history. And it's not just about any document. We can recreate moments from the past essentially," said CEO Satya Nadella.

To use this feature, users must use CoPilot+ PC, and at this time at least, Microsoft suggests that the feature will not come to older systems.

This is where the feature has some serious privacy implications.

Finding that file, image, or long-lost document can be useful, Recall can be a privacy nightmare.

Recalls is not going to record snapshots when users are using private browsing session on Chromium-based browsers, and that users can exclude it from taking snapshots whole users use certain apps or websites completely, with some limitations.

While Recalls has the option to be paused or to be turned off completely, according to Microsoft, Recall is turned on by default.

The privacy implication here is that, users use their PC for various things, and that they may look at a sensitive information on their your screen and forget that it’s running.

According to Microsoft, the snapshots are encrypted by default, and everything is processed locally, meaning that nothing is sent to the cloud.

"We’ve got built-in privacy control so you can always pause your content. You can delete your snapshots," said Nicci Trovinger with Microsoft.

But the issue is not just that.

What if someone else has access to that computer, and somehow managed to use Recall to recall what the computer saw?


Regardless, Recall brings Microsoft close to its goal, which is to "build computers that understand us versus us having to understand computers,” CEO Satya Nadella said, during the company’s annual developer conference at its Redmond, Washington headquarters.

“I feel like we’re really close to that real breakthrough,” he added.

With AI that runs locally, users can expect Microsoft to not spy because the AI doesn't need the internet to do its thing.

"The richest AI experiences will harness the power of the cloud and the edge working together in concert. This in turn will lead to a new category of devices that turn the world itself into a prompt."

"For us, this vision starts with our most beloved and most widely used canvas: Windows."

It's said that Microsoft uses OpenAI's GPT-4o to power this Recall feature.