Google has been working on Gemini’s agentic capabilities, enabling it to carry out actions on our behalf. It recently rolled out an AI-powered Auto Browse feature in Chrome for users on its AI Pro or Ultra tiers. The company is also working on bringing similar capabilities to Gemini on Android, letting it control apps directly on your phone. A new report now sheds more light on this feature.
Gemini has already made many complex tasks faster and easier. But this is just scratching the surface of what’s possible, as the AI will eventually evolve to take actions on our behalf.
A teardown of the latest Google app beta (v17.4) by 9to5Google details how the “Get tasks done with Gemini” feature will work. Codenamed bonobo, it will initially be available as a Labs feature, with the ability to book rides and place orders through screen automation.
In practice, this could let you ask Gemini to book an Uber to your office or order dinner from Uber Eats without ever opening the app yourself.
Reportedly, Gemini’s agentic capabilities will only work in “certain apps” initially. Given that app UIs change frequently, it makes sense for Google to initially limit Gemini’s screen automation feature on Android to a small set of apps. Beyond food delivery and ride-hailing, early support could be limited to first-party Google apps.
The feature will require at least Android 16 QPR3 to work, since Google has laid the groundwork for screen automation with this release.
You will always remain in full control
While you can ask Gemini to carry out tasks on your behalf, you’ll remain in full control and can manually stop or take over at any time.
The strings in the code provide more insight into the feature and the privacy implications. Google notes that, “When Gemini interacts with an app, screenshots are reviewed by trained reviewers and used to improve Google services if Keep Activity is on.”
Google will also warn users not to enter payment information into Gemini Chats and to use screen automation during emergencies.
While not mentioned, Gemini’s agentic capabilities on Android could be limited to users on the Pro and Ultra tiers. The company will most likely announce this feature in beta alongside the release of Android 16 QPR3 in March.
