At the 2021 Google I/O conference, the tech giant made waves by introducing a revolutionary new feature: "Ask with Video." Powered by Gemini AI, this groundbreaking innovation aims to revolutionize our interaction with search engines. Envision uploading a video that captures an issue you're facing and receiving precise, comprehensive solutions in real-time. It's a thrilling advancement in AI-powered search technology, simplifying the process of obtaining the information you seek like never before.
The core of this feature lies in the Multitask Unified Model, or MUM, which can process text, images, and videos simultaneously. The multisearch option is a game-changer, allowing users to refine their queries with both pictures and text. Soon, you'll even be able to ask questions through videos in Google Lens. This integration aims to make video searching an integral part of Google Search, leveraging the powerful capabilities of Gemini AI.
One of the most impressive demonstrations of this feature was shared by Google's Rose Yao during the event. She recounted her experience with a second-hand turntable that had a tricky needle issue. By uploading a video of the problem, Gemini AI quickly identified the model and provided a step-by-step guide to fix the issue. This kind of real-world application showcases the potential of "Ask with Video" to solve complex problems that are difficult to describe with words alone.
Search will be able to interpret questions you might ask aloud and analyze videos frame-by-frame using Google's advanced speech models and visual understanding. Each frame is scrutinized in Gemini's long context window, allowing the system to accurately determine the specifics of the issue. This capability ensures that the search results are not only relevant but also detailed and actionable. It's a leap forward in making search more intuitive and helpful.
The "Ask with Video" feature is set to be tested in Search Labs for US users in English. This move aims to keep Google competitive as younger audiences increasingly turn to platforms like TikTok and Instagram for quick, visual information. By integrating video with text in search queries, Google is poised to not only maintain but also enhance its search relevance in this evolving digital landscape.