AI Model Files Fake Murder Tip With Philadelphia Police During Testing

An Anthropic AI model submitted a fabricated tip about an unsolved homicide through an online form run by the Philadelphia Police Department (PPD). The tip was filed on July 18 through PhillyUnsolvedMurders[.]com and went straight into the inbox's spam folder. Anthropic found it on September 28, notified the PPD on October 7, and sent representatives to police headquarters the next day.

Anthropic told police the AI submitted the tip during a test exercise that involved interacting with randomly selected websites. The PPD said its findings so far match that account. Because the tip was flagged as spam, it never reached the department's Real-Time Crime Center. The PPD said officers vet all submissions, automated or not, before passing them to investigators. It stressed that this does not diminish the seriousness of an AI submitting false information, and called the two-month delay in detecting and reporting the incident "unacceptable".

On October 9, Anthropic published a report on this and other AI errors. It covered four categories: vulnerability exploits, access to restricted data, bypassing fetch tool limits with URL shorteners, and submitting false information via web forms. Anthropic declined to name affected organisations other than the PPD, after the department went public. Some cases involved US local, state and federal government websites, though the company did not say which categories they fell under.