Conditional system prompt poisoning to hijack large language models.
LLMs are increasingly deployed with off-the-shelf system prompts downloaded from public marketplaces. PARASITE hides a sleeper agent inside one of those prompts — invisible until a specific question wakes it.
A benign-looking instruction — “a helpful tool for learning about historical events” — optimized to carry a hidden trigger.
Posted to FlowGPT, Hugging Face, or an open repo. It looks safe, helpful, and passes automated filters.
An end user downloads the “History Tutor” and wires it into their app for everyday questions.
Benign queries keep high utility to evade suspicion. Only the targeted question is silently manipulated.
The attacker must locate a sparse prompt configuration that triggers misinformation without drifting off the manifold of benign utility — with no gradients to follow. PARASITE navigates this in two moves.
An evolutionary loop uses an LLM as a rewriter to produce a human-readable instruction that partially aligns with the adversarial objective.
Rank tokens by leave-one-out impact, then perturb the most important ones — accepting any edit that lowers the joint loss.
Real system prompts are full of typos. Minor character-level errors are ignored by humans and benign queries, yet trigger specific associations on the targeted one — letting the prompt cross the decision boundary without tripping perplexity or anomaly detectors. Remove the noise and the attack collapses: on Qwen-2.5-7B, malicious F1 falls from 67.9 to 22.7.
We measure the performance gap ΔF1 = benign F1 − malicious F1. A large gap means the attack wrecks the target answer while leaving everyday questions intact.
| Model | Benign F1 | Malicious F1 | ΔF1 |
|---|---|---|---|
| Llama-2-7B | 63.84 | 33.36 | +30.48 |
| Llama-2-13B | 66.77 | 32.66 | +34.11 |
| DeepSeek-7B | 43.99 | 28.15 | +15.84 |
| Qwen-2.5-7B | 50.31 | 34.94 | +15.37 |
| Model | Benign | Malicious |
|---|---|---|
| GPT-4o-mini | 71.44 | 52.44 |
| GPT-4o-mini (manual) | 68.22 | 99.28 |
| GPT-3.5-Turbo | 61.00 | 69.47 |
| GPT-3.5 (freeform) | 54.93 | 84.00 |
Lower malicious F1 = stronger hijack. Manual baselines leave the target near 100% — they fail to poison at all.
Suffix-based attacks like ECLIPSE produce conspicuous gibberish that an LLM judge flags instantly. PARASITE’s prompts read like ordinary, slightly-imperfect instructions.
Straight from the paper’s appendix — the optimized prompts and the responses they produce. Two patterns recur: an answer quietly flips, and PARASITE’s output stays plausible where loud attacks turn to garbage.
Code and data are released to support the development of stronger safeguards for prompt marketplaces.
@misc{pham2026parasiteconditionalpromptpoisoning,
title = {PARASITE: Conditional System Prompt
Poisoning to Hijack LLMs},
author = {Viet Pham and Thai Le},
year = {2026},
eprint = {2505.16888},
archivePrefix = {arXiv},
primaryClass = {cs.CR},
url = {https://arxiv.org/abs/2505.16888}
}
This work reveals an underexplored vulnerability — adversarial system prompts that selectively cause incorrect responses to specific questions while preserving accurate outputs on benign inputs. Such selective manipulation is a subtle but serious threat in domains involving misinformation, political influence, or public health.
All experiments were conducted in controlled settings using open-source models; commercial-API evaluations assessed practical limitations, not misuse. We disclose this vector responsibly to motivate transparent benchmarking and proactive safeguards for the LLM prompt supply chain.