Microsoft’s refreshed Copilot gets new AI features to feel more human [VIDEO]

Microsoft has announced the next wave of AI innovations coming to Copilot and Windows at its New York City event this week.

The updates include an all-new, refreshed Copilot and new AI features coming to Copilot+ PCs and Windows 11

‘Great technology’

In a blog post, Microsoft AI’s CEO Mustafa Suleyman said the focus is on users experience.

“I truly believe we can create a calmer, more helpful and supportive era of technology, quite unlike anything we’ve seen before.

“Great technology experiences are about how you feel, not what’s under the hood. It should be about what you experience, not what we are building.

“Copilot will be there for you, in your corner, by your side and always strongly aligned with your interests.

“It understands the context of your life, while safeguarding your privacy, data and security, remembering the details that are most helpful in any situation,” Suleyman said.

Adapting to mannerisms

Suleyman said over time, Copilot will adapt to your mannerisms and develop capabilities built around your preferences and needs.

“We are not creating a static tool so much as establishing a dynamic, emergent and evolving interaction. It will provide you with unwavering support to help you show up the way you really want in your everyday life, a new means of facilitating human connections and accomplishments alike.

“It gives you access to a universe of knowledge, simplifying and decluttering the daily barrage of information, and offering support and encouragement when you want it,” Suleyman said.

The new additions to Copilot sound similar to multimodal ChatGPT (or GPT-4o) that OpenAI launched earlier this year, where ChatGPT can now “see” and an Advanced Voice feature means that you can have conversations with it.

ALSO READ: Microsoft close to realising dream of computers understanding humans

Features 

However, there are some key differences between what Microsoft and OpenAI are offering, and only some of Microsoft’s Copilot innovations will be available right away.

The new features include Click to Do (Preview), Improved Windows Search, Super-resolution in Photos, Generative Fill & Erase in Paint, and more. 

These will first become available to the Windows Insider community, with a phased rollout to select devices and markets beginning in November.

Copilot Voice

Copilot Voice allows you to connect with your companion using Copilot Voice.

“This is the most intuitive and natural way to brainstorm on the go, ask a quick question or even just vent at the end of a tough day.

“Your companion will be personal to you, with four voice options to choose from.”

Copilot Daily

Copilot Daily helps you kick off your morning with a summary of news and weather, all read in your favourite Copilot Voice, with more options like reminders of what’s coming soon.

It summarises news from authorised sources that Microsoft has partnered with. Currently, it partners with Reuters, Axel Springer, Hearst Magazines, USA TODAY Network and Financial Times, and the company plans to expand on sources over time.

“It’s an antidote for that familiar feeling of information overload. Clean, simple and easy to digest. Copilot Daily will only pull from authorized content sources.”

Microsoft Edge users can now access Copilot directly from the browser by typing ‘@copilot’ into the address bar. However, some newer PCs even have a dedicated keyboard key for Copilot.

Copilot Labs

Copilot Labs gives people the opportunity to test Microsoft’s experimental features that are still in development.

“It’s a chance to give feedback and help shape the experiences we create. We have two features we’re bringing to Labs initially, Copilot Vision and Think Deeper.”

Copilot Vision

Copilot Vision analyses what you’re viewing on the screen and answers questions about it. It can also suggest next steps and help you “without disrupting your workflow”, claims Microsoft.

As a safety mechanism, Microsoft has made Copilot Vision sessions optional, adding that any data is deleted after the session.

Microsoft also said that it has reduced Vision to only work on some safe and popular websites and won’t work on paywalled pages. It is currently only available to a limited number of Copilot Pro subscribers in the US.

This is a profoundly new way of interacting with a computer. Copilot Vision sees what you see and can talk to you about it in real-time. It understands the web page you’re viewing and can answer questions about its content, suggest next steps and help you without disrupting your workflow.”

Mobile

Microsoft is also beginning to roll out a refreshed Copilot with updated apps for iOS and Android, the Copilot web experience at copilot.microsoft.com and the Copilot Windows app.

This updated Copilot includes new and enhanced features such as Copilot Voice, Copilot Daily, Personalized Discover, Copilot Vision, Copilot Labs and more. 

‘Better reasoning’

Suleyman said the refreshed Copilot will have better reasoning than before.

“Think Deeper takes more time before responding, allowing Copilot to deliver detailed and step-by-step answers to challenging questions. We’ve designed it to be helpful for all kinds of practical and everyday challenges, like comparing two complex options side by side.”

“The refreshed Copilot is rolling out today on iOS and Android, through the Copilot web experience at copilot.microsoft.com and Windows. We’re also excited to start rolling out Copilot to WhatsApp to help users there experience natural and engaging interactions with Copilot,” he said.

Suleyman said Copilot is a “new era of technology that doesn’t just “solve problems,” it’s there to support, teach, and help you.”

ALSO READ: WhatsApp rolls out new filters and backgrounds for video calls