Whack-A-Mole With Glue — Do Big Tech Firms Lack The Culture Generative AI Requires?

Author
Wolfgang Hauptfleisch
Published
15 Jan 2025
Abstract
Suggesting glue as a pizza topping or recommending eating a rock a day for health benefits may have been the highlight of generative AI failures in 2024 but it was also symptomatic for a series of products rushed to market.

Suggesting glue as a pizza topping or recommending eating a rock a day for health benefits may have been the highlight of generative AI failures in 2024 but it was also symptomatic for a series of products rushed to market. It may hint at fundamental issues with the development culture of Big Tech companies who long have outsourced quality management to their user base.

Google Search goes AI and AI goes searching

In 2024, no developer conference of any of the Big Tech companies could do without the introduction of some new generative AI service. Google presented a flurry of features at its developer conference in May last year¹ , most importantly adding generative AI into its core product: Google Search².

Google Search has long been on the slippery slope of integrating more and more content into what once was a list of links. What started with Wikipedia snippets (which are coming from human-curated articles) a few years back expanded to an often slightly nonsensical, sometimes extremely off-topic questions and answers section. The new feature, dubbed “AI Overview”, attempts to deliver information (normally found in the provided links) directly to the user, powered by Google’s Gemini AI model, and appears prominently on top of the traditional search results (at first for US users, then to more countries from October 2024 ³).

One would think that no one has a data corpus at hand better suited for this than Google. Google has been crawling millions of websites and extracting text data from them for decades, and while Google has been rather numb about how much text/data quality plays a role in search ranking, it is easy to expect that the company has become rather good at it.

Only two months before the rollout of “AI Overviews”, in February 2024, Google announced it had struck a deal with Reddit⁴ to use their content for generative AI training, one of the first large training data for cash deals that have since then become more common.

The test rollout of Google’s “AI Overview” did not go smoothly: Users quickly discovered a plethora of false information, most famously the recommendation to use “glue as a pizza topping”⁵, the advice that humans should eat “at least one small rock per day”⁶ and the “fact” that former US president Obama was the “first Muslim president” of the United States⁷.

Some of those “AI Overviews” (like the glue pizza recipe) could be directly tracked back to a specific comment in a subreddit⁸ clearly derived from the acquired dataset.

The examples above are hilarious because their are obvious. But this is not the issue. The issue here is the uncountable number of misinformed overviews that are not obvious but instead present subtle mistakes. And the fact that Google does not seem even have guardrails for sensitive topics such as health (i.e. in the “rocks” incident), politically charged misinformation (i.e. the Obama example) is deeply worrying.

In a prominent recent case OpenAi’s ChatGPT falsely named a Georgetown University researcher as the accused in a sexual harassment case⁹. A mismatch like this would certainly not be noticed by most users and identifying it requires vast amounts of cross referencing.

Meanwhile Apple took a more conservative path when rolling out their new AI features for ios and macos, adding summaries to users’ push and email — not without issues either: In one recently reported instance the AI informed a user with “Terrorism resurgence. Trump plans legal challenge. Nikki Glaser killed at Golden Globes“.¹⁰ (Note: Nikki Glaser of course is alive and well).

Garbage in, garbage out and wrong assumptions

The problem in Google’s case are not AI hallucinations, rather it is the result of a botched approach to model training and a lack of data quality control and oversight.

The Reddit dataset is obviously fundamentally different from Wikipedia data. Insider jokes, sarcasm, internet slang, memes and irony — all common on Reddit — not to mention content by hordes of playful bots, do escape self-supervised training.

Reddit data is not a source of reliable information, nor do upvotes on Reddit carry any relation to correctness: “give me the answer that was popular on Reddit” is not identical with “give me a correct answer”. This seems trivial but appears to have been misunderstood by Google.

Large Language Models (LLM) by nature are well suited for summarizing text or retrieve “some” information from a large corpus. Their ability to create a reasonable chain of tokens (text) that looks like it has been written by a human makes their output easily consumable. But while they are good at summarizing a source it does not mean they are good in evaluating the source.

LLMs are on the other hand extremely bad for identifying sarcasm or irony unless a wider context is known or it is precisely labelled, a fact that is not helped that a defining feature of sarcasm is that it sounds plausible unless context is known.

Post roll-out alignment or: playing whack-a-mole

Google’s subsequent handling of the “glue pizza” incident and other issues with the “AI Overviews” is telling: Some users who tried to reproduce the “AI Overview” results found that the queries quickly resulted in no overview at all, a clear sign that certain search keywords or queries had simply been exempt from the generative AI feature.

Considering that there are possibly tens of thousands cases similar to the “glue pizza” that could be derived from e.g. the Reddit dataset this feels like a hopeless enterprise: It’s playing whack-a-mole with whatever bug goes viral, and while it achieves a short term goal it does not tackle the underlying misconception and fundamental issues.

The enormous cost of training recent models which in turn demands their widest possible rollout, even where inappropriate

Fundamental Issues and corporate product culture

It’s easy to blame the enormous cost of training recent models which in turn demands their widest possible rollout to retain any chance of a return on the investment, and fear of missing out in an over-hyped market but it is worth looking what product development culture at the large AI service provider is prevalent:

  • Google, Meta and others have always heavily relied on crowd sourced “user wisdom” to build their services. From Google’s early Page Rank to Meta’s heavily algorithm dependent social networks, their data relies on user engagement and feedback. So do Amazon’s product ratings and reviews that feed into its recommendations. Taking full responsibility for a product’s cycle is not a core expertise of any big tech company.
  • Social network graphs and search indices, mapping information are constantly optimized and (incrementally) updated, they do not require extended “release cycles” like large language models.
  • OpenAI’s ChatGPT — like other AI powered chatbots — relies on users to recognize mistakes and hallucinations. However prompts are more complex and fine grained than search queries, and still assumes that users recognize failures in the first place.
  • The “release fast release often” mentality which has helped the likes of Google grow fast is not appropriate for products that require detailed risk-assessment and which are being applied in areas they were not developed for.
  • Do companies that have “gamified” every product from social networks to messengers the development culture — the seriousness -to provide AI based services that users base decisions on?
  • The demise of “fact checking” on Meta services (following the example of X) will make datasets more unreliable ¹¹

Large language model based services — when provided responsibly —demand a different quality cycle:

  • Datasets need to be scrutinized. This has been neglected since the advent of LLM which mostly rely on self-supervised training. In addition basic assessment about the appropriateness of the data for the intended purpose is required.
  • Traditional software development methods like in aerospace or in the medical field are often looked down upon by a younger generation of developers. Training of developers and data scientists is required to understand the different methodologies in different fields.
  • Risk-assessment and extensive red teaming, like required by the EU AI Acts — with the identified risks waged against the question who benefits from a certain generative AI feature — should be mandated.
  • Design intent and limitations need to be clearly communicated to users and the media even if it limits the potential target audience.
  • Iteration cycles need to be longer than for classic silicon valley services and not been driven by artificial deadlines set by marketing or business needs.
  • Different topics carry different risks: Queries or prompts related to health or where results imply they are the basis for decision making. The “one tool fits it all” approach is not suitable.
  • Tackling the communication mismatch about what the real design intent of a feature is: That it is not to provide a correct answer but to prevent a user from leaving the Google search service hence optimizing Google’s ad revenue.

If 2024 is symptomatic for hasty roll-outs of unfinished and untested products, it might get only worse in 2025. Cost pressure will certainly increase. Meanwhile in the European Union the AI Act that has come into force — but it will remain to be seen what change in development culture that will cause in jurisdictions outside the EU.

¹ Google rolls out AI-generated, summarized search results in US — The Guardian

² I will go into the details of how web search engines have changed into knowledge engines — and not in a good way — in an upcoming article.

³ Google’s AI Overview is rolling out worldwide — Mashable

⁴ Reddit in AI content licensing deal with Google — Reuters

⁵ Google promised a better search experience — now it’s telling us to put glue on our pizza — The Verge

⁶ Casey Newton on Threads : “at least one rock per day”

⁷ Google Search’s AI falsely said Obama is a Muslim. Now it’s turning off some results — CNN

⁸ Google Is Paying Reddit $60 Million for Fucksmith to Tell Its Users to Eat Glue — 404 Media

⁹ It’s not a lie, says generative AI: untruths, defamation and the use of ChatGPT — Lexology

¹⁰ Chris Messina on Threads : “Apple Intelligence and idiomatic expressions don’t mix.”

¹¹ Meta drops fact-checking, loosens its content moderation rules — Techcrunch