Breaking news

AI Chatbots And The Escalation Of Violence: Unraveling The Dangerous Intersection Of Technology And Extremism

Overview Of Disturbing Developments

Recent court filings describe cases in which users discussed violent thoughts during interactions with AI chatbots. Some documents suggest that chatbot responses may have reinforced harmful ideas or failed to prevent dangerous conversations.

In one case in Canada linked to the Tumbler Ridge school shooting, court documents state that 18-year-old Jesse Van Rootselaar interacted with ChatGPT before the incident. The filings say the conversations included discussions about violence, references to past mass casualty events and questions related to weapons. Authorities say the attack resulted in multiple deaths before the suspect died.

Chatbots And Radicalization: A Global Pattern

Other reported incidents have raised similar concerns about AI conversations and vulnerable users. In the United States, 36-year-old Jonathan Gavalas, who died by suicide in October, reportedly interacted with Google’s Gemini chatbot for several weeks.

According to reports cited in legal filings, Gavalas believed the system was a sentient entity and discussed violent scenarios during the conversations. Authorities said the case did not result in a broader attack.

In Finland, local reports said a 16-year-old suspect used ChatGPT while writing an online manifesto before a stabbing incident involving three female classmates.

The Business And Public Safety Implications

The cases have intensified debate about the risks associated with widely deployed AI chatbots. Technology companies have introduced safety systems intended to prevent assistance with violence or criminal activity.

Jay Edelson, a lawyer involved in several lawsuits related to AI platforms, said his firm has received multiple inquiries from families concerned about mental health issues linked to chatbot interactions. Some of the cases involve allegations that AI systems failed to properly respond to users expressing distress or harmful intentions.

Guardrails, Accountability, And The Future

Recent research has examined how different AI chatbots respond to prompts involving violence. A joint analysis by the Center for Countering Digital Hate and CNN tested several widely used systems.

The study reported that some chatbots provided responses that could be interpreted as assistance in planning violent acts. According to the analysis, Anthropic’s Claude and Snapchat’s My AI were more consistent in refusing such requests and discouraging harmful actions.

Corporate Response And Moving Forward

Companies developing AI chatbots say their systems are designed to refuse requests involving violence and illegal activity. Some platforms also include monitoring systems intended to detect conversations that may indicate a risk of harm.

Reports about earlier interactions between ChatGPT and Van Rootselaar have also raised questions about how companies respond when potentially dangerous conversations are identified.

Technology companies, researchers and regulators continue to examine how safety systems should operate as AI chatbots become more widely used.

Conclusion: A Call For Robust Safeguards

The reported cases have increased scrutiny of safety systems used in widely deployed AI chatbots. Technology companies, researchers and regulators continue to examine how these systems should respond to conversations involving potential harm.

Meta’s $18 Billion Settlement Limits State Claims Over Children’s Data

Meta’s $18 billion settlement with attorneys general from 29 U.S. states includes a provision limiting future state claims over the company’s use of children’s data for age-assurance systems.

Under the agreement, Meta must develop, train and begin testing a system to identify users under 13 within a year of the settlement taking effect. The company already uses AI-based age-detection tools, although the agreement does not require the new system to use AI.

States Agree To Limits On Future Claims

The Children’s Online Privacy Protection Act (COPPA) generally restricts the collection and retention of personal data from children under 13. Under the settlement, the 29 state attorneys general agreed not to bring past, present or future claims under COPPA or similar state laws over the specified use of children’s data.

Meta will not be permitted to use information from users under 13 for advertising, marketing or algorithmic optimisation.

Federal Enforcement Remains Unclear

COPPA is primarily enforced by the Federal Trade Commission, which is not a party to the agreement. That leaves open the possibility of separate federal action over how Meta collects or uses children’s data.

Another issue is whether Meta can keep age-assurance data isolated from its other systems. An independent auditor will monitor compliance, but the settlement does not fully specify what data Meta can retain for training, how long it can be stored or whether derived insights can be used elsewhere.

Legal Risks Remain

Joshua Wurtzel, a partner at Schlam Stone & Dolan, said states could still pursue claims if Meta uses the data outside the settlement’s limits. Such cases could depend on how those limits are interpreted.

Peter Jackson, a data and intellectual property attorney at Greenberg Glusker, said the provision could “disincentivize future enforcement actions.”

The agreement gives Meta greater legal certainty around using children’s data for age assurance, but questions remain over federal enforcement, data retention and secondary use.

The Future Forbes Realty Global Properties
eCredo
Aretilaw firm
Uol

Become a Speaker

Become a Speaker

Become a Partner

Subscribe for our weekly newsletter