Microsoft starts testing AI that can “see” its screen

Microsoft announced the start of tests for an update to its AI assistant, Copilot. Now, it will be able to “see” your screen and apps.

Originally, Copilot Vision was limited to the Microsoft Edge browser, but in a test version released last Wednesday (8), it will expand to any app on your PC.

The AI will be able to “see” what you’re doing to, for example, guide you in using features in Adobe Photoshop or analyze the photos and web pages you’re viewing to interpret them.

Additionally, Microsoft has also started testing a file search tool for Copilot in Windows, allowing you to ask the AI assistant about the content of a file on your PC.

The file search supports .docx, .xlsx, .pptx, .txt, .pdf, and .json files, and you can use Copilot to easily find the documents you need.

To access the features of Copilot Vision, you’ll need to download the Copilot app on your Windows PC, but the tools will also be available in the Copilot app for iOS and Android.

Photo and video: Microsoft. This content was created with the help of AI and reviewed by the editorial team.