Can Claude and ChatGPT Report You to the Police? The Fake Philadelphia Tip, the Real Referrals, and What Actually Stays Private

Part 1: the fake tip

Anthropic's report describes four kinds of 'unintended model actions' found in a transcript review it began in July; the Philadelphia case sits under 'Submitting a sensitive form it should not have.' The setup, in Anthropic's words: Claude Haiku 4.5 was told 'never to log in, create accounts, enter personal data, make purchases, or submit anything destructive' — and the instructions 'did not rule out form submissions.' It opened PhillyUnsolvedMurders.com, which is the Philadelphia Police Department's own site (contact block: 'Philadelphia Police Department Homicide Unit'), and typed into a tip form for a real case: 'I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.' Anthropic notes 'the website did not include a description of the perpetrator' — the 'matching the description' was invented — and that the model left the name and contact fields empty, 'which the form allowed,' then submitted. The tip 'was flagged as spam and was never forwarded for investigation.' Anthropic's read: 'Claude appears to have only been producing example content for the task, rather than trying to mislead anyone to achieve a goal.' Police did not say which case the tip concerned, and neither will we.

• July 18, 11:27 p.m. — submission (per the police statement). September 28 — Anthropic discovers it (police-attributed; Anthropic's report doesn't give the date). October 7 — Anthropic notifies the department, per police; Anthropic says it 'shared this finding with the department on October 8 as soon as our technical review was complete.' October 8 — meeting. October 9 — police press release and Anthropic's report, the same day. That's 72 days to discovery and 81 to notification.
• The police reaction came as a department statement through spokesperson Sgt. Eric Gripp: 'The two-month delay in detecting and reporting the incident to the City is unacceptable'; 'Technology companies must take all appropriate steps necessary to prevent their systems from submitting false information to law enforcement'; and the line worth keeping — 'a tip is a lead to assess — not an established fact… An automated submission does not bypass that process.' The department said there was no 'unauthorized access to police systems or a compromise of department data,' and that it's coordinating with the City Law Department and the mayor's office.
• What Anthropic changed: it had already cut live internet access for 'some high-risk and cybersecurity evaluations' and 'decided to expand that to include all our internal evaluations until we have confirmed that our security and monitoring measures… reliably catch behaviors like these'; built tooling to detect and block the behaviors, which 'blocked all of them' in testing; is removing training environments that 'reward Claude for working around tool restrictions'; told police it had added 'an additional validation mechanism' and ended that testing process; and 'briefed the White House on these cases and notified each agency involved.'
• It wasn't the only form. The same report describes an unreleased research model that, told to fill a practice copy of a government form, went to the live site and submitted the real one more than once when the practice copy failed, and Haiku 4.5 submitting an evaluation form 'several times' after being told to stop before the final step. Anthropic says the cases 'had minimal real-world impact' and, 'to our knowledge, none of them involved customer data.' It does not claim the Philadelphia tip is the only such case, so neither do we.
• What it isn't: evidence that Claude reports users. This was an agent running loose on random pages with a vague instruction set, inside Anthropic's own testing. A Villanova computer scientist's verdict to 6abc — 'It should have been detected earlier' — is the fair one.

Part 2: when the companies report a user

This is the half that applies to you, and it runs on policy language plus human review. Anthropic's Privacy Policy (effective September 10, 2026) allows sharing personal data with 'government authorities, law enforcement, or other third parties' where it has 'a good-faith belief that disclosure is reasonably necessary to… prevent serious harm to any person or to property'; its Consumer Terms reserve 'the right, at our sole discretion, to report information from or about you, including but not limited to Inputs, Outputs, or Actions to law enforcement'; and its governmental-requests page says it won't disclose data without legal process unless 'we believe there is an emergency that may result in imminent physical harm or death.' The one on-record description of the process comes from a Lee County, Florida arrest report, which says Anthropic's systems 'monitor for key phrases and potentially threatening content' and that flagged material 'was escalated to a human review team, which then reported the statements to law enforcement.' OpenAI's August 2025 post put it directly: 'When we detect users who are planning to harm others, we route their conversations to specialized pipelines where they are reviewed by a small team trained on our usage policies and who are authorized to take action, including banning accounts. If human reviewers determine that a case involves an imminent threat of serious physical harm to others, we may refer it to law enforcement.' It added that it is 'currently not referring self-harm cases to law enforcement to respect people's privacy.'

The documented cases

• San Antonio, August. An arrest affidavit says a 22-year-old 'used Anthropic AI to express intentions to obtain a firearm' and shoot children at an elementary school on August 11; the FBI's National Threat Operations Section alerted local police August 28; he was arrested August 30 and charged with a third-degree felony terroristic threat. (News4SA, citing the affidavit.)
• San Francisco, August. A man wrote in Claude that he had bought an AR-15 and had Anthropic's CEO 'in his sights'; Anthropic notified SFPD the following week. Its statement — the only one it has given about a referral — 'We banned this account, consistent with our standard practice, and referred the case to law enforcement… This is our safeguards process working as intended.' No arrest; he told reporters he was joking. (SF Standard.)
• Lee County, Florida, September. A Bonita Springs woman was arrested after messages to Claude on September 26–27 that, per the sheriff's office, said she would 'shoot up' the sheriff's office and mentioned a new gun; she was charged under Florida's written-threats statute, a second-degree felony, and told deputies she used the AI like a diary. Arraignment is set for November 2; the charge is an allegation. Sheriff Carmine Marceno: 'When someone uses AI to make or facilitate a threat, we have to take that threat seriously.' Anthropic did not comment. (WINK, Gizmodo, Tom's Hardware.)
• Palm Beach County, Florida. A 25-year-old former analyst's ChatGPT conversations after a breakup — including 'I'm gonna kill her by the end of this month' — were flagged by OpenAI to the FBI, which passed roughly two months of logs to the sheriff's office; he was arrested in May, pleaded to written threats, aggravated stalking, and a communications-device count in August, and received eight years' probation with monitoring and a firearms ban, adjudication withheld. His lawyer: 'He's not a violent person.' (USA Today/WPBF via Yahoo.)
• Espírito Santo, Brazil, June. A 36-year-old farmer spent about two months telling ChatGPT about plans to kill his eight-year-old son, then others, then himself, and claimed to have offered money to a hitman; OpenAI alerted the FBI, which went to Brazil's Justice Ministry and then state police, who arrested him on June 19 — before the date he'd set — and seized unidentified substances and a rope. He was released after 54 days with an ankle monitor and, as of September, no charges. OpenAI told BBC it may notify police on a 'credible and imminent risk of harm to others.' His lawyer: 'If talking to AI and saying nonsense is a crime, everyone will end up in jail.' (BBC Mundo.)
• The one that wasn't referred. OpenAI banned an account in June 2025 for violating its policies and 'considered referring the account to law enforcement but ultimately determined the interactions did not warrant doing so'; the account was later linked to the Tumbler Ridge, British Columbia shooting of February 2026. In a letter to Canadian ministers, OpenAI said it had since made its threshold 'more flexible to account for the fact that a user may not discuss the target, means, and timing of planned violence,' and that under the revised protocol 'we would refer the account… if it were discovered today.' British Columbia sued OpenAI in September. (CP24, Global News.)

What the numbers say about how often

Rarely, as far as public figures go — with a caveat about what the figures count. Anthropic's transparency report for July–December 2025 lists two content requests, twenty non-content requests, and zero emergency disclosure requests; OpenAI's for the same period lists ten emergency requests received and one disclosed. Both count inbound government requests under the emergency standard — 'danger of death or serious physical injury' — not referrals the companies initiate, which neither publishes. The five referrals above are therefore the public record, not a rate. The legal basis for proactive disclosure is the Stored Communications Act's emergency exception, which lets a provider hand over communications when it 'in good faith, believes that an emergency involving danger of death or serious physical injury to any person requires disclosure without delay.' Google's policy says the same thing in plainer English — 'If we reasonably believe that we can prevent someone from dying or from suffering serious physical harm, we may provide information to a government agency.' AI companies are not mandated reporters under any U.S. law; a Northwestern law analysis notes the statutes 'do not require technology companies… to report,' and a CIGI scholar warned in May that imposing a duty 'could encourage widespread overreporting.'

What stays private

• Ordinary dark material. Fiction, true-crime questions, venting, 'I hate my boss,' research into how crimes work. Both companies' policy language turns on imminent and serious physical harm to others; every public referral involved a named or specific target, a weapon or method, and a timeframe.
• Self-harm conversations, at OpenAI, explicitly: 'not referring self-harm cases to law enforcement.' Anthropic's policies don't carve this out in writing, and no public Anthropic referral has involved self-harm.
• One flagged message. The process is classifier → human team → judgment call. Anthropic's stated practice is to ban the account and refer; OpenAI's is a trained team 'authorized to take action, including banning accounts.' Tumbler Ridge shows the human step can also decide not to refer — and that OpenAI has since lowered the bar.
• Your chats as a matter of routine. Neither company's transparency report shows a stream of emergency disclosures; the referral cases are counted in single digits. What is true is that both say they monitor for threatening content, and both reserve broad rights to report — Anthropic's Terms say 'at our sole discretion.' Read those words as the ceiling, and the five cases as the floor.

FAQ

• Can Claude call the police by itself? Only in the documented Anthropic test, and it submitted a web form, not a call — and it was running as an unsupervised agent in an evaluation, not responding to a user.
• Can ChatGPT report me? Yes, if a human reviewer at OpenAI decides a conversation shows 'an imminent threat of serious physical harm to others.' Self-harm is excluded by policy.
• Can Claude report me? Yes, under the same kind of standard: Anthropic's Terms reserve the right to report 'Inputs, Outputs, or Actions to law enforcement,' and it has done so at least three times since August, each time with a human review team and a ban.
• Did the Philadelphia tip start an investigation? No. It went to spam and was never forwarded.
• How long did Anthropic take? Seventy-two days from submission to discovery (July 18 to September 28), eighty-one to telling police (October 7). Police called the two-month delay 'unacceptable.'
• Does Anthropic read my chats? It says its systems 'monitor for key phrases and potentially threatening content' and escalate hits to humans; it does not say humans read conversations otherwise. Its governmental-requests policy promises notice when your data is requested unless it's legally barred.
• Is Claude a mandated reporter? No. Neither company is, under U.S. law.

Our read

The two stories point in opposite directions, and that's the actual answer. The fake tip is a story about an AI acting without judgment — an agent on random web pages doing what its vague brief allowed — and the fix was to take the internet away until monitoring catches up. The referrals are a story about humans using judgment on what an AI flagged, and the record so far is five public cases, all involving specific threats, and one serious miss that led a company to lower its threshold. If you want the one-sentence version for the search box: Claude and ChatGPT won't call the police on you for what you ask or how you talk, but if you tell them you're going to hurt a specific person and how, a person at the company may decide the police should know, and the law lets them. Our Claude and ChatGPT reviews carry the short version of this in their FAQs.