Navigating the Evolution of AI-Powered Search

This article was originally published in The Drum.

Our SEM and SEO teams at PMG recently gained access to Google’s Search Generative Experience (SGE), the experimental search engine powered by artificial intelligence (AI). We dove in to understand the new search environment and how SGE handles concise, purpose-driven searches and complex, conversational queries with multiple layers of meaning.

We experimented with a wide range of search types, focusing on business-critical queries like brand terms, generic keywords, brand-generic combination searches, ‘near me’ searches, as well as searches around sensitive topics.

ChatGPT has already captured the public’s imagination by utilizing a static dataset to generate information that directly answered their queries (albeit with some personalized quirks). Meanwhile, search engines like Google and Bing strive to satisfy the broad spectrum of real-life questions and search behaviors posed by individuals from all walks of life – in real-time. This will take a massive feat of engineering capabilities and imagination.

Here’s what we discovered when we put SGE to the test.

Generative AI is good at summarizing content, answering factual queries, and providing well-written responses to more subjective questions. However, we encountered a mix of relevant and tangentially related results, including varying levels of visual presentation and ungrammatical, keyword-stuffed queries, requiring further refinement.

Implicit information, such as the importance of timeliness in fashion-related content, posed a challenge for SGE, with a search for trendy footwear yielding an article from 2021, neglecting the need for up-to-date information in fashion.

When we started to look at where SGE is the default option, searches combining a brand name with a specific product, particularly fashion searches, often automatically triggered the generative experience seamlessly.

For example, a search for ‘summer sandals Tory Burch’ produced a rich set of results, including articles about which sandals are best for certain types of feet, popular styles, and relevant follow-up questions—in contrast with the classic search results. However, when searching hotel brands, our team had to specifically request the generative experience.

Queries that could be both a brand name and a generic term, such as ‘economist,’ presented an interesting result: The initial search engine results page (SERP) primarily focused on the brand, The Economist, whereas the SGE experience emphasized the generic interpretation of the query, exposing the complexity underlying SGE’s algorithms and the searcher’s true intent.

While traditional search results reflected personalized outcomes, the generative experience seemed to discard or deprioritize this aspect. Personalized information carefully tailored to individual users often vanished within the generative experience. It also occasionally fixated on peripheral elements of a query, inadvertently neglecting the searcher’s intent altogether.

For instance, a search for high-quality coffee nearby, accompanied by an appreciation for Jamaican coffee and a preference for local stores over chains, led to results focused on countries renowned for coffee production – with no local listings provided.

Although subsequent clarifying searches improved the results, SGE needed help to grasp that Jamaican coffee was a suggestion, while the searcher's primary intent was obtaining high-quality coffee nearby. In another instance, a multidimensional query about ‘trendy men’s sandals for men with wide feet’ led to articles centered on knee health, nursing shoes, and women’s footwear.

Nevertheless, there were instances where SGE improved traditional search experiences. Simple statements like ‘I want to get my first credit card’ triggered the classic search experience, while more explicit queries, such as ‘What do I need to get approved for my first credit card’ yielded rich results.

The risk-reward calculus of AI goes far beyond conventional business problems, potentially ranging from medical breakthroughs to existential threats. Search engines must consider factors such as politics, public perception, and ethics.

Enhancing speed and speech understanding capabilities are obvious goals in providing users with seamless experiences; however, the real challenge lies in determining the prioritization of information within complex searches.

Building and maintaining trust through search remains crucial in an era where misinformation has eroded the notion of truth. User behavior shaped by ingrained habits will play a pivotal role in shaping the future of search as users have become accustomed to instantaneous results that precisely meet their expectations. It’s likely that users will only adjust these expectations if search engines face more intricate challenges.