Gemini Unlocks Proactive AI: New Beta Feature Anticipates Your Needs from Photos and Emails

Published 19 hours ago3 minute read
Uche Emeka
Uche Emeka
Gemini Unlocks Proactive AI: New Beta Feature Anticipates Your Needs from Photos and Emails

Google has unveiled a new beta feature for its Gemini AI assistant, termed "Personal Intelligence," which empowers Gemini to tailor responses by connecting and reasoning across a user's Google ecosystem. Initially integrating with Gmail, Google Photos, Search, and YouTube history, this advancement moves beyond mere information retrieval. Gemini can now understand context and provide proactive, intelligent results, such as linking an email thread to a previously watched video without explicit instructions on where to search.

This "Personal Intelligence" feature is designed with user control in mind, remaining off by default. Users have the option to actively choose if and when they wish to connect their Google applications to Gemini, acknowledging that not everyone may desire AI access to their personal photos or YouTube history. Google assures users that even when connected, Gemini will only leverage Personal Intelligence when it deems it genuinely helpful to the user's query or task.

According to Josh Woodward, VP of Gemini app, Google Labs, and AI Studio, Personal Intelligence boasts two primary strengths: its ability to reason across diverse and complex data sources and its capacity to retrieve precise details from specific applications like email or photos. Woodward highlighted how these strengths often combine, allowing Gemini to process text, images, and videos to generate uniquely customized answers. He provided an illustrative example where Gemini helped him determine his car's tire size and proactively suggested all-weather tires after analyzing family road trip photos in Google Photos. Similarly, Gemini could retrieve his license plate number from an image in his Photos library.

Beyond practical queries, Woodward noted that Gemini has been instrumental in offering personalized recommendations for books, shows, clothes, and travel. For instance, in planning a spring break, Gemini analyzed his family's interests and past trips in Gmail and Photos to suggest unique experiences, such as an overnight train journey and specific board games for the trip, effectively bypassing typical tourist traps.

Google has implemented guardrails to manage sensitive topics, ensuring Gemini avoids making proactive assumptions about private health data. However, the AI assistant is designed to discuss such sensitive information if directly prompted by the user. Importantly, Google clarifies that Gemini does not directly train on the content of user Gmail inboxes or Google Photos libraries. Instead, it trains on specific prompts within Gemini and the model's generated responses. The referenced photos or emails in the examples serve only to inform a particular response, not to train the underlying AI model.

The Personal Intelligence beta is currently rolling out to Google AI Pro and AI Ultra subscribers in the U.S., with plans for future expansion to more countries and eventually to Gemini's free tier. Google has provided examples of prompts that users can try, including requests to plan a weekend based on personal preferences, recommend documentaries aligned with past interests, or suggest YouTube cooking channels based on delivery receipts, search history, and YouTube watch history.

Loading...
Loading...
Loading...

You may also like...