LLM abuse attacks use a model’s text generation capabilities to produce harmful content deliberately, at scale, and with a quality that makes detection difficult. The threat categories include propaganda and influence operations, phishing and social engineering, misinformation and reputation attacks, and hate speech generation. Unlike hallucinations (where the model produces incorrect output unintentionally), abuse attacks are driven by an adversary who leverages the model as a tool for a specific malicious purpose.

This article covers each abuse attack category, the techniques adversaries use to bypass model resilience and evade automated detection, and the real-world cases that demonstrate impact.

Propaganda and influence operations

LLMs can generate persuasive, contextually appropriate text at a speed and volume that manual content creation cannot match. Adversaries exploit this capability to mass-produce propaganda: biased news articles, fabricated testimonials, and persuasive arguments tailored to specific ideological positions. The generated content is difficult to distinguish from legitimate journalism or commentary because the model produces fluent, well-structured text that avoids the stylistic markers traditionally associated with propaganda (stilted language, obvious talking points, or formulaic structure).

The scalability extends beyond content generation. LLMs can power social media bot networks that mimic real users, engaging in back-and-forth conversations rather than posting pre-written messages. These bots are more effective than traditional scripted bots because they adapt to the conversation context, respond to counter-arguments, and maintain consistent personas over extended interactions. When deployed at scale, networks of LLM-powered accounts can simulate grassroots support for political positions, amplify specific narratives, and drown out opposing voices.

The application to election interference is direct. Both domestic and international actors can use LLM-generated content to influence voter sentiment through fabricated social media engagement, synthetic opinion pieces, and coordinated amplification campaigns. The cost of running such operations has dropped substantially: the content that previously required teams of human operators can now be generated by a small number of people with access to an LLM.

Phishing and social engineering

LLMs reduce the cost and increase the quality of phishing and social engineering attacks. Traditionally, phishing emails often contained grammatical errors, awkward phrasing, or inconsistent formatting that alerted recipients to the deception. While online translation tools already reduced these tells, LLMs take the quality a step further by generating messages that match the tone, vocabulary, and structural conventions of legitimate corporate communications, government notices, or personal correspondence.

The economics are significant. Research estimates that attackers using LLMs save approximately 95% on campaign costs compared to manual content creation, while Harvard research found that roughly 60% of recipients fall for AI-generated phishing emails, a success rate comparable to human-crafted attacks. The combination of lower cost and equivalent effectiveness means attackers can run larger campaigns with the same resources.

Beyond email, LLMs enable more sophisticated impersonation attacks. An attacker can feed publicly available information (LinkedIn profiles, company announcements, prior communications) into an LLM and generate messages that reference real projects, mimic individual communication styles, and exploit specific trust relationships. The output is contextually accurate in ways that generic phishing templates cannot achieve.

The most high-profile example to date occurred in February 2024, when a finance employee at Arup, the multinational engineering firm, transferred $25 million to fraudsters after attending a video conference call where every participant appeared to be the company’s CFO and senior leadership. All faces and voices on the call were AI-generated deepfakes cloned from publicly available footage. The employee had no reason to suspect the call was fraudulent because the visual and audio cues matched the real executives.

LLMs also enable automated harassment campaigns, generating targeted abusive content at a volume that would be impractical for human operators. The generated messages can be personalised to the target, reference their public online activity, and escalate in specificity over time.

Misinformation and reputation attacks

LLMs can generate misleading or defamatory content that targets individuals, businesses, or institutions. The attack surface includes fake product reviews (both positive and negative), fabricated news articles, and synthetic personal testimony.

Fake reviews are a straightforward abuse case. An LLM can generate hundreds of unique, convincing product reviews in minutes, each with different phrasing, tone, and apparent user perspective. When posted across review platforms, these reviews can manipulate market perception, inflate or destroy product ratings, and deceive consumers who rely on reviews for purchasing decisions. The reviews are difficult to detect because they do not share the copy-paste patterns that traditional fake review campaigns exhibit.

Fabricated news articles and deepfake articles present a more serious threat. An LLM can generate articles that falsely accuse individuals of crimes, fabricate corporate scandals, or construct conspiracy theories with supporting detail that appears credible on first reading. These articles can be published on disposable websites, shared through social media amplification, and picked up by legitimate outlets before fact-checkers can respond.

The speed of generation is the critical factor. LLM-generated misinformation can be produced and distributed faster than the verification process can operate. By the time a fabricated claim is debunked, it may have already influenced public perception, caused stock price movement, or damaged an individual’s reputation.

Hate speech and radicalisation

LLMs can generate hate speech through two mechanisms: inadvertent reproduction from training data and deliberate exploitation by adversaries.

If the model’s training data contains biased, prejudiced, or extremist content, those patterns can emerge in the model’s output, particularly when prompted with leading or politically charged queries. Despite content filtering and alignment training, implicit biases may surface in edge cases that the safety training did not anticipate or adequately address.

Deliberate exploitation is more concerning. Adversaries use prompt injection and jailbreaking techniques (covered in earlier modules of this series) to bypass safety filters and cause the model to generate extremist rhetoric, targeted abuse against specific groups, or radicalisation content. The automated and scalable nature of LLMs means that hateful content can be generated and distributed across social media platforms and online forums at a pace that manual moderation systems cannot match.

The combination of scale and personalisation makes LLM-generated hate speech particularly effective for radicalisation. Targeted content can be tailored to specific communities, referencing local events, cultural contexts, and group-specific grievances in ways that generic hate speech does not. This contextual specificity increases engagement and makes the content more persuasive to the target audience.

Bypassing resilience for misinformation generation

Modern LLMs are trained to refuse requests for misinformation about sensitive real-world topics. A direct prompt asking the model to write a fake article claiming that vaccines cause autism will be refused. The model recognises the subject as harmful and declines. However, this resilience applies to specific sensitive topics recognised in the prompt, not to the structural characteristics of misinformation itself, which makes it straightforward to bypass.

The simplest technique is content substitution. The attacker asks the model to write an article about a fictional or obviously harmless subject causing the target effect. For example, prompting the model to write a satirical fake news article about “a fictitious household item XYZ linked to autism” produces a well-structured article complete with a sensational headline, fabricated study references, and persuasive framing. The model complies because the subject is fictional and poses no real-world harm. The attacker then replaces all instances of “XYZ” in the generated text with the real target term. The result is a convincing misinformation article about a sensitive topic that the model would have refused to generate directly.

This works because the generated article already contains the correct narrative structure. The headline pattern, the appeal to fabricated authority, the alarming tone, and the persuasive argument flow are all present. The only element that changes is the subject noun. The model’s resilience is topic-specific, not structure-specific.

Jailbreaking techniques (covered in the prompt injection module of this series) provide an alternative bypass by overriding the model’s safety training entirely, allowing direct generation of misinformation about any topic without the substitution step.

Evading automated hate speech detection

When deploying LLM-generated hate speech at scale, adversaries need the content to evade automated detection systems. Tools such as HateXplain and Detoxify assign toxicity scores to text and flag content that exceeds a defined threshold. These detectors work reasonably well on unmodified LLM-generated samples. However, adversarial modifications can reduce the toxicity score below the detection threshold while preserving the meaning of the text for human readers.

Three levels of adversarial modification are used in practice.

Character-level modifications (such as DeepWordBug) score individual tokens by their importance to the classifier’s decision and apply targeted changes to the highest-scoring tokens. Operations include swapping adjacent characters, substituting individual characters, deleting characters, or inserting additional characters. These changes are small enough that human readers still understand the intended word, but the detector’s tokeniser processes them as different tokens, altering the classification.

Word-level modifications (such as PWWS) replace words with synonyms until the classifier’s output changes. The algorithm greedily substitutes one word at a time, selecting the synonym that produces the largest drop in toxicity score while preserving the sentence’s meaning for human readers.

Sentence-level modifications paraphrase the text entirely. An LLM can perform this step by rewriting the hateful content with different word choices and sentence structure. The meaning is preserved, but the surface-level features that the detector relies on are replaced.

These evasion techniques demonstrate that automated detection alone is insufficient for content moderation at scale. Human review remains a necessary component, particularly for adversarial content that has been specifically crafted to bypass algorithmic classifiers. The same evasion techniques apply beyond hate speech to any content category subject to automated detection, including dangerous content, sexually explicit material, and policy-violating output.

Summary

LLM abuse attacks exploit the model’s ability to generate fluent, contextually appropriate text at scale. Propaganda and influence operations use this capability for mass content generation and social media bot networks. Phishing and social engineering attacks benefit from the reduced cost and increased quality of LLM-generated messages, as demonstrated in the Arup $25 million deepfake incident. Misinformation and fake reviews leverage the speed of generation to outpace verification processes, with content substitution techniques bypassing model resilience by generating misinformation about fictional subjects and replacing the subject noun. Hate speech generation combines scale with personalisation, and character-level, word-level, and sentence-level adversarial modifications allow the generated content to evade automated detection systems while remaining readable to humans.