7 biggest Google I/O 2024 announcements: AI and Gemini galore

Google I:O 2024 developer conference
Image: Google.

Google kicked off its I/O 2024 developer conference overnight, showing off the company’s latest software. Surprising no one, AI was the main theme.

Big changes to online search represent one of the most substantial shifts to Google’s core product in its history. Alongside the AI-heavy search experience, the mega-company also revealed updates to its existing features, like Circle to Search, in addition to its future vision for AI assistants.

Hardware took a back seat, although many devices will soon take advantage of the many tools shown, starting with the Pixel 8 phones. Expect to see more about Android 15 tomorrow. For now, here are seven of the biggest announcements from Google I/O 2024.

AI Overviews summarises Google search responses

“Google will do the Googling for you”, as the company in question puts it. After a lengthy trial phase, Google’s generative AI search feature is coming to the public under the name “AI Overviews”.

Coming to the US first, with other countries to follow, AI Overviews generates responses based on your search queries, like ChatGPT, Meta AI, and Microsoft Copilot. Using the latest Gemini AI model, the feature takes inputs – like multi-faceted questions – and produces a text summary along with any relevant links.

According to Google, people who tested the beta version “use search more, and are more satisfied with their results.” Recent updates to the search platform’s algorithm have caused concerns, particularly among independent publishers, due to reduced referral traffic.

When launching AI Overviews, Google also claimed that testers visited “a greater diversity of websites” and “links included in AI Overviews get more clicks” than traditional search results.

Considering Google expects more than a billion people to have access to AI Overviews by the end of the year, we’ll soon see whether this replicates at a wider scale.

Circle to Search becomes a teacher

Circle to Search, first seen on the Samsung Galaxy S24 phones, has been one of the more impressive software features in recent times. By circling a subject on your phone, you can instantly look it up online.

As shown at Google I/O, Circle to Search can now help kids with homework. Compatible with some physics and maths problems, you can circle a question to get a step-by-step explanation of how to solve it.

Google intends to expand the school-assisting functionality throughout the year. In the near future, Circle to Search will also be able to tackle problems based on graphs and diagrams.

Google I/O 2024 reveals smarter on-device AI

For now, only the Google Pixel 8 Pro has access to Gemini Nano, the company’s smaller AI model capable of running without an internet connection. Later this year, both the Pixel 8 and Pixel 8a will get Nano, enabling more local AI tasks, like transcriptions and text suggestions.

As announced at Google I/O, Gemini Nano will soon benefit from “full multimodal capabilities”. This essentially means gaining more complex processing beyond just text inputs. It’ll also factor in audio-visual contextual cues, like spoken words.

Google also suggested an accessibility use case, with Nano able to provide spoken alt text on images. Plus, there’s fraud detection potential, notifying you when the device notices any dodgy behaviour when on a phone call.

Again, this all happens locally on your device – no internet connection required.

Answer questions by uploading a video

Further to AI Overviews, Google is testing the ability to search via video. If you come across a specific problem that you can’t articulate, like a malfunctioning camera, you could record a video to get an AI-generated response.

It’s another experimental feature coming to testers soon. However, the video search feature has already raised eyebrows. As The Verge’s Nilay Patel pointed out, the demo shown at Google I/O highlighted a less-than-ideal response.

Ask Photos makes it easier to sort your saved snaps

Phones have let you search your saved files via keywords for a while now. Typing in “cat” brings up any saved images that look like a feline friend. Google’s “Ask Photos” aims to take things a step further, helping you find more specific images.

Labelled an “experimental feature” due out in the next few months, Ask Photos lets you search your photo library using more complex queries. More than just sorting specific snaps, it also provides responses based on an image’s – or series of images’ – contents.

Google Ask Photos demo
Image: Google.

One example provided was listing the themes a family had used for their daughter’s previous birthdays. Ask Photos then listed each theme based on the family’s Google Photos library, alongside the pictures it used to generate the response.

It’s a novel idea in theory that could make it easier to sift through large photo albums – not just by contents, but also by context.

Google I/O 2024 teases next-gen AI assistant Project Astra

Much of Google I/O revolved around Gemini, including a glimpse at Project Astra, an in-development AI assistant that responds to visuals and audio.

A pre-recorded demo showed someone pointing their phone’s camera at various objects while asking the device to answer various complex queries. The assistant provided programming advice, made up an alliterative sentence, and explained the components of a desk speaker.

It’s part of Google’s vision to make a reactive AI assistant capable of analysing context and responding naturally in real-time.

Want less cluttered search results? There’s a tab for that

All of the AI-powered features took the limelight at Google I/O 2024, but the back-to-basics feature might be the most useful. Among the tabs you can click on when searching on Google is a new “Web” option.

Click on this tab, and the search results suddenly look a lot cleaner. Instead of showing previews of shop listings and videos, it just shows a feed of web links, like the good old days. If you want to search for an article or a specific website page, this looks like the best way to do it.

Read more software news on GadgetGuy