Cybersecurity researchers from SentinelOne and Censys are raising concerns about a growing risk in the AI ecosystem: open-source large language models that run without basic safety guardrails. As more models are modified or deployed with protections removed, researchers say they are becoming easier targets for criminal misuse.
The core issue is not open source itself, but what happens when safeguards are intentionally stripped away. According to the research, attackers could compromise systems running these models and then use them to automate spam campaigns, generate disinformation, or assist other malicious activities, all while bypassing traditional security controls.
The researchers analyzed hundreds of internet-accessible LLM instances and found that many were based on popular open models, particularly variants of Meta’s Llama and Google DeepMind’s Gemma. In a significant number of cases, the models had their guardrails fully disabled. By examining system prompts directly, the team gained rare visibility into how these models were configured and used in real-world environments.
That analysis revealed that around 7.5% of observed prompts had the potential to cause serious harm. The geographic distribution of these exposed systems also raised concerns. Roughly 30% of the hosts were operating out of China, while about 20% were based in the United States.
Juan Andres Guerrero-Saade, executive director for intelligence and security research at SentinelOne, warned that current industry discussions often overlook this problem. In his view, there is a growing surplus of powerful AI capacity that is being repurposed in ways that range from legitimate experimentation to clearly criminal activity.
The findings have reignited questions about responsibility in the open-source AI space. A spokesperson for Meta declined to comment on whether developers should take more responsibility for preventing downstream abuse of open models. Meanwhile, Ram Shankar Siva Kumar, Microsoft’s AI Red Team Lead at Microsoft, emphasized that while open models are driving innovation, the company is actively monitoring emerging threats and misuse patterns.
Researchers conclude that addressing these risks will require shared accountability. Safe innovation in AI, they argue, depends on coordinated efforts from model creators, deployers, researchers, and security teams alike, especially as open models continue to scale in capability and reach.
We have helped 20+ companies in industries like Finance, Transportation, Health, Tourism, Events, Education, Sports.
