Microsoft Just Dropped a Bunch of New Copilot Features

After an underwhelming launch in June, Microsoft is retooling Copilot and Copilot+ to make it match up to its competitors a bit more. It's a mixed bag of an update though: It’s losing some of its initial, unique promises—there don’t seem to be any tools for adjusting your Windows settings included in the new version—even as it gains multimodal features (like ChatGPT) and AI generative fill (like Photoshop).

The Copilot chatbot gets a voice

In a series of videos and a blog post, Microsoft revealed that Copilot now comes with a new, more neutral look as well as the ability to chat over voice—and soon, it will also take input from video. It’s similar to what companies like Google are doing with Gemini Live, with a few Microsoft touches.

Most obvious are the cards that pop up when you open the new Copilot app or website, which resemble Microsoft’s live tiles UI from older versions of Windows and give you suggestions for what to do with Copilot based on your past conversations with the chatbot.

More interesting is the Copilot Daily feature, which presents a brief AI-generated summary of recent news headlines. These are taken from approved partners like Reuters or Financial Times, to prevent copyright issues, and are presented in a sort of TV news anchor voice. It could be helpful to quickly get caught up when waking up, although I’d caution against using a freshly developed robot as your only news source.

The new look and voice support are available for iOS, Android, and the web today, while Vision, which will let you point your camera at your surroundings in order to use them when asking the AI questions (say, "What breed of dog is this?"), is currently part of the experimental Copilot Labs program.

Also in Copilot Labs is “Think Deeper,” which will let you give the AI more time to answer your questions, hopefully improving your results. Copilot Labs features will be available to a limited number of Copilot Pro subscribers first—it’s not something you can just sign up for like a Windows Insider account.

At this point in the lifecycle of AI, none of these reveals are especially revolutionary, with even Copilot Daily having similar competition in apps like Perplexity. The unique but controversial Recall feature that will take continuous screenshots of your Copilot+ PC is still in testing, although Microsoft says it will come to testers with Qualcomm’s Copilot+ chips soon.

In the meantime, Copilot+ laptops are getting some additions to Paint and Search to hold them over. 

Copilot+ Laptops may soon finally be worth it

Credit: Microsoft

Building on the existing Cocreator in Paint feature, which uses generative AI to make images based on your sketches, Generative Fill and Generative Erase allow Copilot+ laptop owners to adjust their images without having to draw anything first. Fill will simply let you create and place AI imagery from text, while Erase works a bit like Google’s Magic Eraser, letting you paint over elements you want to remove from a photo with a brush, then watching as they get erased and replaced with imagery the AI thinks is appropriate.

Credit: Microsoft

There’s also AI upscaling in the Copilot+ version of the Photos app, with which on-device AI will fill in the blanks on low-res photos to increase their resolution as much as eightfold. Microsoft says you can get 4K results “within seconds,” although I’ll need to see more examples of its output before I can really recommend the tool.

Credit: Microsoft

Meanwhile, Windows Search on Copilot+ PCs will soon use AI to discern the content of your files, not just their names. That means you could search for “cat” and it could bring up photos of cats, even if “cat” is not in the file name. While Microsoft doesn't say that the feature is in testing, the company seems to think it’s not ready yet, saying it will be coming first to File Explorer next month, and to Windows Search and Settings “in the coming months.”

Perhaps this extra time is to address any privacy concerns that could come from Microsoft snooping through your files, as happened with Recall, although Microsoft does say the feature will work offline.

Credit: Microsoft

One feature that Microsoft does say is in testing is “Click to do.” Click to Do seems sort of like an AI-powered right click, with Copilot analyzing what you’re currently doing on your PC and suggesting appropriate actions in a context menu. It’s a bit more involved, since you’ll need to access the Snipping Tool or Print Screen to do it, but it’s the one new feature that hearkens back to Copilot’s initial promises of, well, helping you pilot through Windows. Click to Do will be available to Windows Insiders on Copilot+ laptops next month.

Overall, today’s changes and announcements are hefty, in the sense that there’s a lot to dig into, but it might be a while until they actually impact your workflows, as some are in testing and you’ll need a Copilot+ laptop to access most of them. Whether that'll make Copilot+ worth the investment remains to be seen, but in a best-case scenario, features like Generative Fill could potentially save you the cost of a Photoshop subscription.

Source: View source