Summary
- Gemini is evolving rapidly, with new live video AI features resembling Project Astra being added to the mix.
- A person with a Google One subscription on a Xiaomi device could share their screen in real-time with Gemini overlays.
- Google’s initial release of the new features will focus on Gemini Advanced subscribers with a Google One AI Premium plan.
Gemini has been Google’s starchild for the past couple of years, with new capabilities added almost every week. What started as a chatbot has now taken over Assistant’s role, permeated into Google Workspace, and also rivals other Search tools like Lens and Circle to Search. With so many overlapping tools existing concurrently, it’s no surprise Google just added one more to the mix — live video AI features for Gemini resembling Project Astra.
Related
Project Astra: Everything you need to know about the Google Deepmind project
Google’s new vision for AI assistants
We’d forgive you if Project Astra is now forgotten since its unveiling nearly a year ago at Google’s annual developer conference in May 2024. It represented the company’s short-term vision for a real-time assistant capable of multimodal audio and video prompting. While Android XR for mixed reality glasses is still gaining traction, Gemini Live gave us the first taste of the future. Now, 9to5Google reports users are activating Gemini overlays for the Astra-powered camera and screen sharing in Live (via The Verge).
Gemini already supports Live discussions about a screenshot from your device, but a Reddit user with a Google One subscription on a Xiaomi device found they suddenly could share their screen in real time with the AI. They could tap the button called Share screen with Live just above the older Ask about screen prompt in the Gemini interface. Thereafter, you or anyone with this feature can ask about anything on screen.
Real-time audio video Astra was spotted too
Going beyond screenshots
Support for screen sharing rolling out widely isn’t a surprise per se, because it is one of the AI features Google promised to roll out in early March, and the same Redditor claims they had access for a limited time, but screen recording didn’t work, perhaps due to privacy-related permissions. However, the search giant’s initial release will focus on Gemini Advanced subscribers with a Google One AI Premium plan. Hopefully, limited demos will be available for free-tier users to drum up excitement for the subscription.
When prompting Astra, which uses real-time audio and video from your device’s camera and mic, Gemini switches to a phonecall-style UI. Here, you can pause the feed if interrupted, switch cameras, toggle video prompting on or off, switch to screen sharing, or end the session.
In our testing, neither Share screen with Live nor Astra were available, but the Reddit user claims they aren’t a beta tester, and they didn’t see an update for the Gemini app either. These signs suggest we are witnessing the early signs of a staggered worldwide release through server-side changes, provided Google keeps its word and follows through.
