---
格式版本: 2
标题: "A Jagged Frontier: Evaluating Robustness of Code Agents to Semantics-Preserving Transformations"
原文链接: "https://arxiv.org/abs/2608.18389"
发布日期: "2026-08-18"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Tue, 18 Aug 2026 23:46:11 UTC (762 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-08-20T15:27:10+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-08-20T15:18:01+08:00"
入库时间: "2026-08-20T07:27:11.032Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=deployment&searchtype=all"
匹配关键词:
  - "deployment"
  - "AI"
相关厂家:
  - "Microsoft"
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 20
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "论文主题为AI代码代理的鲁棒性评估，与超节点/AI Rack/机柜级AI基础设施完全无关，仅摘要含deployment一词，无任何相关技术或商业信息。"
AI质检模型: "ali-deepseek-v4-flash"
AI质检时间: "2026-08-20T15:30:38+08:00"
AI主题相关性: 0
AI来源权威性: 10
AI新颖性: 5
AI技术细节: 0
AI商业部署信号: 0
AI完整性: 5
AI摘要: "该研究评估了代码智能体在代码库被改写成语义等价形式后的修复可靠性，发现多数配置下性能仅小幅下降，最严重配置平均下降6.7个百分点，16个配置中有6个出现统计显著退化。鲁棒性排名因脚手架而异，且更简单的mini-SWE agent表现更稳定。"
AI摘要模型: "ali-deepseek-v4-flash"
AI摘要时间: "2026-09-07T03:21:30.807Z"
采集批次: "2026年8月20日14点19分32秒"
采集批次ID: "20260820-141932-079"
去重键: "https://arxiv.org/abs/2608.18389"
---

## Computer Science > Artificial Intelligence

## Title:A Jagged Frontier: Evaluating Robustness of Code Agents to Semantics-Preserving Transformations

Authors:[Hasan Najib Mahmud](https://arxiv.org/search/cs?searchtype=author&query=Mahmud,+H+N) (1), [Shreya Gupta](https://arxiv.org/search/cs?searchtype=author&query=Gupta,+S) (2), [Isha Chaudhary](https://arxiv.org/search/cs?searchtype=author&query=Chaudhary,+I) (3), [Nathaniel Enis](https://arxiv.org/search/cs?searchtype=author&query=Enis,+N) (1), [Ravi Mangal](https://arxiv.org/search/cs?searchtype=author&query=Mangal,+R) (1), [Gagandeep Singh](https://arxiv.org/search/cs?searchtype=author&query=Singh,+G) (3), [Corina Pasareanu](https://arxiv.org/search/cs?searchtype=author&query=Pasareanu,+C) (4) ((1) Colorado State University, (2) Microsoft, (3) University of Illinois Urbana-Champaign, (4) Carnegie Mellon University)

[View PDF](https://arxiv.org/pdf/2608.18389) [HTML (experimental)](https://arxiv.org/html/2608.18389v1)

> Abstract:AI code agents are increasingly deployed to resolve real software issues, yet their reliability under superficial code variations remains poorly understood. We evaluate whether coding agents that repair repository-level issues remain reliable when the surrounding codebase is rewritten into a semantically equivalent form. We introduce a random variant sampler that applies common semantics-preserving transformations (SPTs) - spanning control-flow rewrites, dead-code injection, and identifier renaming - to produce perturbed variants. We evaluate two agentic scaffolds (mini-SWE agent and OpenCode) each backed by one of four frontier models (Claude Opus 4.5, Kimi K2.5, MiniMax M2.5, and Qwen 3.6-27B) across instances drawn from SWE-bench Verified and SWE-bench Pro. For each instance, the agent is run multiple times on the unperturbed and perturbed variants, yielding paired resolve-rate estimates that isolate the perturbation effect from intrinsic stochasticity. We find small degradation in most configurations: up to 6.7 percentage points mean resolve-rate drop in the most affected configurations with statistically significant degradations in 6 of 16 configurations of model, scaffold, and dataset. Crucially, no single model ranking by robustness holds across scaffolds - Qwen is among the most robust under mini-SWE agent on SWE-bench Verified yet the most brittle under OpenCode - revealing a jagged robustness frontier. The simpler scaffold (mini-SWE agent) is more robust to perturbation. Our results demonstrate that even top frontier models are susceptible to semantics-preserving perturbations although the effect is not uniform, raising concerns about the deployment reliability of AI code agents in diverse real-world codebases.

| Comments: |  |
| --- | --- |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | [arXiv:2608.18389](https://arxiv.org/abs/2608.18389) \[cs.AI\] |
|  | (or [arXiv:2608.18389v1](https://arxiv.org/abs/2608.18389v1) \[cs.AI\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2608.18389](https://doi.org/10.48550/arXiv.2608.18389) |

## Submission history

From: Hasan Najib Mahmud \[[view email](https://arxiv.org/show-email/13a63d44/2608.18389)\]  
**\[v1\]** Tue, 18 Aug 2026 23:46:11 UTC (762 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2608.18389) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))