AI SUMMARIES CAN REWRITE YOUR MEMORY

Generative AI is spreading into high-stakes work: police departments can now generate reports automatically from body-camera footage or audio; AI also appears in hiring and healthcare. It makes mistakes. In one reported case, an AI-generated police report claimed that an officer had turned into a frog — it had picked up the soundtrack of The Princess and the Frog playing in the background.

The usual safeguard is a human who checks the output. But human memory is fallible too.

A 50-year-old effect

In 1978, psychologist Elizabeth Loftus showed that information received after an event can change how we remember it: the misinformation effect. In her classic experiment, people watched a car accident at an intersection with either a stop sign or a yield sign, then read a text naming the wrong sign — and many misremembered it. Even a single verb (“smashed” rather than “hit”) can change how fast people remember the cars going.

Mattea Sim, Yael Eiger and Tadayoshi Kohno, of Georgetown University and the University of Washington, asked whether AI summaries could do the same.

Step 1: how accurate are AI summaries?

They took two 25-second animated videos from a modern replication of Loftus’ study: a red car reaches an intersection with a stop or yield sign, turns right, hits a pedestrian who falls and gets up, and the driver steps out. They asked ChatGPT-5.5 and Gemini 2.5 Flash-Lite for factual, neutral summaries of at least 300 words that included “everything important” — 20 summaries in total.

• Every summary contained errors: from 7 to 21 each, in about 305 words.
• On average, 51.6% of the central details were missing.
• 19 of the 20 summaries left out the collision itself — the single most important event.
• 60% added details that were not there, such as invented pedestrians or vehicles. One summary even stated that “no other characters or vehicles interacted directly with the scene.”
• The errors changed from one attempt to the next: the car turned right in one summary, left in another.

Step 2: does it change what people remember?

328 participants in the United States watched one of the videos. One to two days later, they read a 21-sentence summary written by ChatGPT. It named either the correct traffic sign or the wrong one. Some were told an AI wrote it, others that a professional human transcriber did — the text was identical. Then came a memory test.

• With a correct summary, 83.6% remembered the right sign.
• With a misleading one, only 44.8% did.
• It made no difference whether people thought the summary came from an AI (46.3% correct) or a human (43.4%).
• Neither trust in AI nor how often people used it made a significant difference.

Share of participants who recalled the right traffic sign: about 80% after a correct summary, under 50% after a misleading one — whether it was labelled AI or human. — Figure 1, Sim, Eiger & Kohno (2026), arXiv:2609.28820.

Participants did remember the video itself: 96.6% correctly recalled that the accident happened in daytime, a detail absent from the summary. Curiously, the “human” summary was rated clearer, although it was word for word the same.

A human in the loop is not enough

The authors conclude that the person meant to correct the AI can instead have their memory altered by its errors — with no malicious intent from anyone. In settings like policing, where eyewitness memory has long been a concern, they question whether AI should be used at all. A shortened version of this work is published in the proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (October 2026).

Limits

Two simple animated videos, one prompt and 20 summaries; a single type of misleading detail tested online. The summary shown to participants was selected among several attempts and lightly edited for accuracy.