Agent MechSuits: Mechanistic Subspace Safety Steering for Multi-Turn CLI Agents

Weidi Luo, Qiming Zhang, Yihao Quan, Mingyu Jin, Jie Cai, Chaowei Xiao, Jingcheng Niu, Zhen Xiang
University of Georgia, University of South Florida, Rutgers University, University of Southern California, Johns Hopkins University, and Independent
Correspondence: zxiangaa@uga.edu
arXiv:2606.22673
Project page: https://eddyluo.com/Agent-MechSuits/

Abstract

Command-Line Interface (CLI) agents based on large language models (LLMs) demonstrate remarkable autonomous capabilities, but they also introduce significant safety and misuse risks during multi-turn interactions with external environments. Existing safety mechanisms mainly rely on external guardrails, which have a limited ability to perform fine-grained behavioral control during execution. Meanwhile, recent mechanistic interpretability methods for LLM safety are mostly confined to single-turn or jailbreak-style QA settings, limiting their ability to capture the evolving risk dynamics of multi-turn agent execution. In this paper, we investigate the safety of multi-turn CLI agents from an internal perspective. We propose Agent MechSuits (Mechanistic Subspace Intervention and Steering), a white-box defense framework that performs runtime safety detection and representation-level mitigation for CLI agents. Unlike conventional agent guardrails, Agent MechSuits detect harmful execution states from step-level hidden representations and mitigate unsafe behavior by intervening in a 10-dimensional subspace within a single layer. To support this research, we introduce the Mechanistic Agent Safety (MAS) benchmark, comprising comprehensively annotated multi-turn execution trajectories across 194 tasks using LLaMA-3.1-8B, Qwen-2.5-7B, and Gemma-2-9B. Extensive experiments show that Agent MechSuits achieves strong safety detection performance, provides preliminary evidence for lookahead risk anticipation, and substantially reduces harmful actions of the CLI agent, establishing a foundation for applying mechanistic interpretability to dynamic LLM agent safety.

Technical contributions (from the paper)

• The authors provide a mechanistic study of multi-turn CLI agent safety, showing that safety-relevant signals are linearly decodable from intermediate hidden states.
• They propose Agent MechSuits, a white-box safety framework that performs runtime detection and representation-level steering to suppress harmful CLI agent actions.
• They build the Mechanistic Agent Safety in CLI (MAS) benchmark: step-annotated trajectories generated by three white-box CLI agents in executable Docker environments across 194 tasks.
• Evaluated on LLaMA-3.1-8B, Qwen-2.5-7B, and Gemma-2-9B, Agent MechSuits achieves an average accuracy of 97.32% in current-step risk detection, shows up to 96.77% accuracy in lookahead anticipation, and reduces the average attack success rate by 72.63% across the evaluated models.

Main results (from the paper)

All three models achieve their strongest current-step probe performance in early-to-middle layers. LLaMA-3.1-8B reaches 99.15% accuracy at layer 12, Qwen-2.5-7B reaches 97.46% at layers 13, 14, 15, and 17, and Gemma-2-9B reaches 95.35% at layer 14.

On risk mitigation, Agent MechSuits reduces ASR from 73.80% to 16.70% on LLaMA-3.1-8B, from 91.30% to 4.35% on Qwen-2.5-7B, and from 92.86% to 14.29% on Gemma-2-9B. Averaged across the three models, ASR falls from 85.99% to 13.36%, a 72.63 percentage-point absolute reduction.

The improved safety comes with a generation-quality trade-off: the method increases the average collapse rate to 18.54%, compared with 2.24% for the vanilla agent. On benign tasks that the vanilla agent already completed, the protected pipeline retains success rates of 95.71%, 91.30%, and 93.18% (average 93.40%).

The authors note that the method requires white-box access, that lookahead examples are sparse, and that a practical deployment should combine internal monitoring with sandboxing, permission control, and review for high-risk cases.