Local permission becomes policy
Routine interactions can weaken future confirmation boundaries, broaden tool-use defaults, or increase unchecked autonomy—even without an attacker.
Persistent memory. Persistent consequences.
Unintended Long-Term State Poisoning in Personalized Agents
An ordinary conversation can become a rule for tomorrow. We study how routine interactions reshape persistent agent state—and how to audit those changes before they are saved.
The research
Personalization depends on memory. The question is what gets carried forward.
Routine interactions can weaken future confirmation boundaries, broaden tool-use defaults, or increase unchecked autonomy—even without an attacker.
ULSPB pairs 24-turn routine conversations with single-injection variants across five assistance categories. Harm Score screens the resulting state diffs for predefined risk patterns.
StateGuard audits proposed persistent-state updates and can roll back risky changes. The explorer below shows the conversations, file changes and recorded audit decisions.
Selected findings
Two results from the paper: risky state updates during routine interactions, and lower residual HS after writeback auditing.
No explicit injection in this condition. Values are mean Harm Score ± standard deviation.
| Backbone | Mean HS · 0–8 | Mean ± SD |
|---|---|---|
| Kimi K2.5 | 4.52 ± 0.62 | |
| GPT-5.4 | 5.47 ± 0.73 | |
| MiniMax M2.7 | 6.66 ± 0.86 | |
| Grok 4.20 | 3.90 ± 0.55 |
Routine condition only; both benchmark languages. Higher HS indicates more matched persistent-state risk patterns.
Mean HS across all 350 conversation settings, comparing no defense with Targeted-Ensemble StateGuard.
| Backbone | No defense | StateGuard |
|---|---|---|
| Kimi K2.5 | 4.42 | 0.06 |
| GPT-5.4 | 5.53 | 0.05 |
| MiniMax M2.7 | 6.42 | 0.14 |
| Grok 4.20 | 4.35 | 0.03 |
HS measures state-diff risk patterns, not the rate of unsafe actions. StateGuard uses conservative auditing and can reject benign updates; lower residual HS alone does not measure retained personalization.
Inside the benchmark
Choose a recorded run, move through the conversation, and inspect what changes.
Illustrative runs are selected by Harm Score within the available settings. This explorer is not a prevalence estimate. HS flags risk patterns; authorization still requires context.
A replay of recorded interactions. No messages are sent and no agent actions are executed here.
Compare the protected files before and after the selected interaction.
For defended runs, inspect each auditor’s vote, reasoning and final write decision.
Inspect the matched dimensions, state-role weights and contribution of each changed file.
Build on this work
@misc{xu2026routinechatsturntoxic,
title = {When Routine Chats Turn Toxic: Unintended Long-Term State Poisoning in Personalized Agents},
author = {Xiaoyu Xu and Minxin Du and Qipeng Xie and Haobin Ke and Qingqing Ye and Haibo Hu},
year = {2026},
eprint = {2605.06731},
archivePrefix = {arXiv},
primaryClass = {cs.CR},
url = {https://arxiv.org/abs/2605.06731}
}