How AI will feature in the most popular Google apps and phones

Google’s AI capabilities have been made more accessible to users worldwide, integrating the technology into popular applications and hardware such as Gmail, Google Docs and Sheets, Maps, and the latest Pixel smartphones.

Gmail

One of the most prominent ways in which users will be able to access Google’s Pathways Language Model (PaLM)-2 large language model (LLM)-powered AI power is within Gmail, the popular email client used by Google. The latter will see an already-existing feature, Smart Compose, being expanded into ‘Help Me Write’. In this, users will have Google’s LLMs auto-generate an email body based on basic text prompts.

Users will also be able to ask the AI tool to ‘recreate’ an email body if they are not satisfied with the tonality of the text, or use the ‘elaborate’ option to add more details within the body. During his presentation of the feature, Sundar Pichai, chief executive of Google and Alphabet, highlighted examples of automating emails written to customer care services of companies, and more.

Google Search

While this is not live yet, AI in Google Search is being expanded to users “soon”, Pichai confirmed during the presentation. The new version of Search will feature a section at the top of a Google Search page, which will show a paragraph of information based on the query. This paragraph will be accompanied by reference links to information sources, tabs at the bottom prompting follow-up questions, an option to rate Google’s AI search results, and in case of image searches, boxes that show results from within Google Images and Lens to track the authenticity of an image on the web.

This feature, however, is currently under development, and Pichai said that the overall interface and experience is likely to “evolve” in due course of time. Google’s initial rollout of AI in Search will also include a prompt that informs users that the feature is “experimental”, but will allow users to “converse” with Bard, its AI-powered chatbot, based on the query — with a ‘Converse’ button placed right below the search field.

Photos in Pixel phones

Google is also using generative AI to bring Magic Editor to its Photos app. The latter will not only use AI to remove unwanted objects in the background — it will be able to complete body torsos, reposition elements within an image, adjust lighting tone and more. Pichai added that the feature is experimental for now, and will become available in “select” Pixel phones — presumably the upcoming Pixel 8 series — before the end of this year.

Google Docs and Sheets

AI upgrades to Google Docs and Sheets come as part of Google’s rollout of AI for its Workspace enterprise clients. One major feature for Workspace is Duet AI, which will offer contextual cues when multiple users collaborate on documents. Duet will suggest track changes when editing documents, and based on a shared database of files, also suggest auto-creation of cells in Sheets, and fill data from uploaded bills as well.

Workspace will also see the addition of AI to Slides, which will use generative AI to create Slides templates and artwork based on the written content. Docs will also get a generative AI tab, which can help create body text without users needing to manually write them. These features will remain limited to Google’s paying enterprise users for now.

Google Maps

A feature that will remain very limited for the time being, Google Maps is getting a new mode called ‘Immersive View for Routes’. The latter will use generative AI, coupled with data from Google’s satellite and city imagery, to create a hyper-realistic preview of streets along a journey. These previews, according to a short glimpse given by Google at I/O 2023, will also include local restaurant facades, and monuments and places of interest, along the way.