Technology
Friday, December 8th, 2023 2:59 pm EDT
Key Points
- Project Ellmann’s AI Concept: Google is exploring the idea of using artificial intelligence (AI) technology, specifically large language models (LLMs) like Gemini, to create a comprehensive overview of users’ lives using mobile phone data such as photographs and searches. Dubbed “Project Ellmann” after the biographer Richard David Ellmann, the concept involves ingesting search results, identifying patterns in user photos, and creating a chatbot to answer complex questions, aiming to be the user’s “Life Story Teller.”
- Integration with Google Photos and Gemini: The proposal does not specify whether the capabilities of Project Ellmann will be incorporated into Google Photos or any other product. Google Photos, with over one billion users and four trillion photos and videos, is a potential platform for such features. Additionally, Google recently launched its advanced AI model, Gemini, which is multimodal, capable of processing information beyond text, including images, video, and audio. Gemini is expected to be licensed to a wide range of customers through Google Cloud.
- Functionality and Demonstrations: Project Ellmann aims to provide a bird’s-eye view of a user’s life story by analyzing biographies, past moments, and subsequent photos. Large language models could infer significant life events, such as a user’s child’s birth, by leveraging unstructured context from various sources. The presentation includes demonstrations of “Ellmann Chat,” showcasing a chatbot that knows everything about a user’s life. The technology could answer questions about pets, family visits, and even provide insights into a user’s eating habits, interests, work, travel plans, and more. Google emphasizes privacy as a top priority and describes Project Ellmann as a brainstorming concept in the early exploration stages. The proposed project aligns with the broader trend among tech giants to create personalized life memories using AI-driven features in photo apps.
A team at Google has proposed a project named “Ellmann” that utilizes artificial intelligence (AI) technology to create a comprehensive overview of users’ lives using mobile phone data, including photographs and searches. Named after biographer Richard David Ellmann, the project aims to leverage large language models (LLMs) like Gemini to analyze search results, identify patterns in user photos, and create a chatbot capable of answering complex questions. The ultimate goal is to be the user’s “Life Story Teller,” providing a bird’s-eye view of their life experiences.
The proposal suggests integrating Project Ellmann into Google Photos, a platform with over one billion users and a vast repository of four trillion photos and videos. The project was presented alongside Gemini, Google’s advanced AI model capable of processing multimodal information, including text, images, video, and audio. The teams spent months exploring the use of LLMs to make the bird’s-eye approach a reality.
Ellmann aims to go beyond traditional photo tagging, utilizing biographies, previous moments, and subsequent photos to offer a deeper understanding of user photos. The presentation suggests identifying meaningful moments, categorizing periods like university years, Bay Area years, and years as a parent. The team emphasizes the need for a holistic view of a user’s life to answer challenging questions and tell compelling stories.
The proposed AI system could infer details such as a user’s child’s birth by drawing on contextual information and knowledge from different points in the user’s life. The ability of LLMs to process unstructured context from various sources across a user’s life is highlighted as a key strength.
The presentation includes demonstrations of “Ellmann Chat,” envisioning a chatbot that already knows everything about a user’s life. Users could ask questions about their pets, recent events, or even seek recommendations for moving to similar towns. The AI system, by analyzing photos, could provide insights into users’ eating habits, preferences, potential purchases, interests, work, and travel plans.
While Google acknowledged the use of AI in Google Photos to enhance search capabilities, a company spokesperson stated that Project Ellmann is a brainstorming concept in the early exploration stages. Privacy protection is highlighted as a top priority, and Google intends to proceed responsibly.
The proposed project reflects the ongoing race among tech giants to create more personalized life memories. Google Photos and Apple Photos have long offered features like “memories” and album generation based on photo trends. However, the complexity of appropriately displaying and identifying images remains a challenge, as demonstrated by past issues related to image labeling and unintended memory resurfacing.
For the full original article on CNBC, please click here: https://www.cnbc.com/2023/12/08/google-weighing-project-ellmann-uses-gemini-ai-to-tell-life-stories.html
