Agent之Harness《Self-Harness: Harnesses That Improve Themselves》翻译与解读导读本文提出了一种区别于“优化模型参数”与“人工设计 Agent 框架Harness”的新范式——Self-Harness。论文认为限制 LLM Agent 能力提升的关键瓶颈已逐渐从模型本身转向运行 Harness而 Harness 又具有明显的模型特异性难以依靠人工持续维护。因此作者构建了一个无需人工工程师或更强外部模型参与的Agent 自主优化 Harness闭环通过**弱点挖掘Weakness Mining—修改生成Harness Proposal—回归验证Proposal Validation**三个阶段使 Agent 能够依据自身执行轨迹持续发现缺陷、修改运行机制并验证效果从而实现 Harness 的持续演化。实验表明该方法能够在多个不同系列的大模型上稳定提升任务性能并证明 Agent 可以从“被 Harness 塑造”进一步演进为“主动塑造 Harness”的新型智能体。背景痛点● 模型能力已不再是唯一瓶颈 当前 LLM Agent 的最终性能由基础模型Base Model与Harness 共同决定而现有研究长期聚焦模型能力提升却忽略了 Harness 对任务执行质量的决定性影响。● Harness 高度依赖人工经验 当前 Prompt、工具调用策略、运行逻辑、错误恢复机制等 Harness 组件主要依赖专家人工设计本质属于经验驱动工程缺乏自动优化能力维护成本随着模型数量增加而迅速上升。● 不存在通用 Harness 不同 LLM 在推理习惯、工具调用方式、错误模式及 Prompt 敏感性方面存在明显差异因此同一 Harness 很难适配所有模型导致模型迁移成本高、复用性差。● 现有自动优化方案依赖外部更强模型 一些 Harness 自动优化方法采用更强 Agent 或教师模型指导较弱 Agent 修改 Harness这种范式既增加系统复杂度也限制了系统真正的自主演化能力。具体的解决方案● 提出 Self-Harness 自优化范式 论文提出 Self-Harness使目标 Agent 能够自主分析自身行为并修改运行 Harness而非依赖人工工程师或外部更强模型完成优化实现 Harness 的内部闭环演化。● 以失败模式驱动 Harness 演化 系统首先从 Execution Trace 中挖掘具有代表性的失败模式Failure Pattern随后针对具体问题生成对应 Harness 修改而不是进行无目标的 Prompt 调整使优化具有明确的问题导向。● 采用最小修改Minimal Edit策略每次仅修改 Harness 中必要部分如 Prompt、工具调用规则、运行逻辑或错误恢复策略避免系统复杂度持续膨胀提高可解释性与可维护性。● 建立回归验证机制 所有候选修改均需经过回归测试仅保留能够解决原有问题且不会导致其他任务性能退化的修改从机制上保证 Harness 能够稳定演化。 核心思路步骤● 步骤一执行任务并记录完整轨迹。 Agent 在真实任务中运行记录 Prompt、工具调用、环境反馈、中间执行过程及最终结果形成 Execution Trace为后续分析提供依据。● 步骤二弱点挖掘Weakness Mining。 系统分析大量执行轨迹识别具有共性的失败模式而不仅仅统计单次错误从行为层面定位模型的固有弱点。● 步骤三HarnessProposal。 针对发现的失败模式自动生成多种候选 Harness 修改方案包括 Prompt 调整、工具调用优化、验证机制补充及运行流程改进等并坚持最小修改原则。● 步骤四ProposalValidation。 将所有候选修改放入回归测试集进行验证仅保留能够带来性能提升且不存在负迁移影响的修改方案。● 步骤五持续迭代演化。 已验证成功的 Harness 成为新的运行基础再次执行任务、发现新问题并继续优化形成持续、自主、自反馈的 Harness 演化闭环。 优势● 实现真正意义上的自主优化 相较于传统人工 Prompt Engineering 或依赖教师模型的 Meta-HarnessSelf-Harness 将 Harness 优化能力内生于 Agent实现自我改进。● 跨模型具有较好的泛化能力 在 MiniMax M2.5、Qwen3.5-35B-A3B 与 GLM-5 三类不同模型上均获得稳定性能提升说明该框架并非依赖某一特定模型特性。● 优化更加精准且可解释 修改直接来源于执行轨迹中的失败模式而非增加泛化性 Prompt因此生成的 Harness 更具针对性、可解释性和可执行性。● 降低长期维护成本 随着基础模型快速迭代Harness 能够自动适配模型行为变化减少人工重新调参与维护工作提高系统长期可持续性。● 保证持续演化稳定性 回归测试机制有效避免性能回退使 Harness 在长期演进过程中保持稳定提升而非因累积修改导致系统退化。论文结论与观点侧重经验与建议● 经验——Harness 已成为 Agent 能力的重要决定因素 实验表明仅优化 Harness 即可显著提升 Agent 性能说明运行框架已成为与基础模型同等重要的性能来源。● 经验——执行轨迹比最终结果更具优化价值 Execution Trace 能够暴露模型真实的行为缺陷失败模式分析比单纯比较任务结果更有助于发现系统性问题。● 经验——最小修改优于大规模重构 局部、渐进式 Harness 编辑不仅能够持续积累收益还能降低引入新错误的风险使系统保持较高可维护性。● 经验——实验验证跨模型有效 在 Terminal-Bench-2.0 上Held-out Pass Rate 分别由 40.5%→61.9%MiniMax M2.5、23.8%→38.1%Qwen3.5-35B-A3B、42.9%→57.1%GLM-5验证了 Self-Harness 的有效性与一定的泛化能力。● 建议——未来 Agent 应具备持续自我演化能力 作者认为未来智能体的发展重点不应局限于提升模型能力而应建立能够自主学习、自主修正、自主维护运行机制的长期演化系统使 Agent 不仅执行任务也持续优化自身运行环境。● 建议——Harness 优化应成为独立研究方向 随着模型更新速度不断加快Harness 自动优化将成为构建高性能 Agent 的关键能力其研究价值将与模型训练本身同样重要。目录《Self-Harness: Harnesses That Improve Themselves》翻译与解读Abstract1、IntroductionFigure 1:Three paradigms of harness improvement. In human harness engineering, human engineers manually revise the agent harness. In Meta-Harness, a stronger external agent guides the improvement of a weaker target agent. In Self-Harness, the agent improves its own operating harness.图1三种改进范式。在人类工程中人类工程师手动修改智能体的运行框架在元框架Meta-Harness中一个更强的外部智能体引导较弱的目标智能体进行改进在自框架Self-Harness中智能体自主优化自身的运行框架。5 Conclusion《Self-Harness: Harnesses That Improve Themselves》翻译与解读地址论文地址[2606.09498] Self-Harness: Harnesses That Improve Themselves时间2026年06月08日作者上海人工智能实验室AbstractThe performance of LLM-based agents is jointly shaped by their base models and the harnesses that mediate their interaction with the environment. Because different models exhibit distinct behaviors, effective harness design is inherently model-specific. Yet agent harnesses are still largely engineered by human experts, a paradigm that scales poorly as modern LLMs become increasingly diverse and rapidly evolving. In this paper, we introduce Self-Harness, a new paradigm in which an LLM-based agent improves its own operating harness, without relying on human engineers or stronger external agents. We operationalize Self-Harness as an iterative loop with three stages: Weakness Mining, which identifies model-specific failure patterns from execution traces; Harness Proposal, which generates diverse yet minimal harness modifications tied to these failures; and Proposal Validation, which accepts candidate edits only after regression testing. We instantiate Self-Harness on Terminal-Bench-2.0 using a minimal initial harness and three base models from diverse families: MiniMax M2.5, Qwen3.5-35B-A3B, and GLM-5. Across all three models, Self-Harness consistently improves performance, with held-out pass rates increasing from 40.5% to 61.9%, 23.8% to 38.1%, and 42.9% to 57.1%, respectively. Qualitative analyses further show that Self-Harness does not simply add generic instructions, but effectively turns model-specific weaknesses into concrete, executable harness changes. These results suggest a path toward LLM-based agents that are not merely shaped by their harnesses, but can also participate in reshaping them.基于大语言模型LLM的智能体性能由其基础模型以及调节其与环境交互的控制框架共同决定。由于不同模型表现出不同的行为特征有效的控制框架设计本质上具有模型特异性。然而目前智能体的控制框架仍主要依赖人工专家进行设计这种范式在现代LLM日益多样化且快速演进的背景下难以扩展。本文提出“自适应控制框架”Self-Harness这是一种新型范式使基于LLM的智能体无需依赖人类工程师或更强大的外部智能体即可自主优化自身的运行框架。我们将自适应控制框架实现为一个包含三个阶段的迭代循环**弱点挖掘**Weakness Mining从执行轨迹中识别模型特定的失败模式**操控方案提出**Harness Proposal针对这些失败生成多样且最小化的操控框架修改**方案验证**Proposal Validation仅当通过回归测试后才接受候选修改。我们在Terminal-Bench-2.0上以一个极简初始框架为基础结合来自不同家族的三个基础模型——MiniMax M2.5、Qwen3.5-35B-A3B和GLM-5实现了该方法的具体应用。在三种模型中Self-Harness始终能有效提升性能保留集外数据的通过率分别从 40.5% 提升至 61.9%从 23.8% 提升至 38.1%以及从 42.9% 提升至 57.1%。定性分析进一步表明Self-Harness 并非简单地添加通用指令而是能够将模型特有的弱点转化为具体且可执行的框架调整。这些结果为构建基于大语言模型的智能体指明了一条路径它们不仅受制于自身框架还能主动参与重塑该框架。For a conscious being, to exist is to change, to change is to mature, to mature is to go on creating oneself endlessly.—Henri Bergson, Creative Evolution对于一个有意识的存有而言存在即变化变化即成长成长即不断创造自我。——亨利·柏格森《创造进化论》1、IntroductionTo date, LLM-based agents are not shaped by their base model alone, but also by their harness: the surrounding system that situates the model and mediates its interaction with the environment. Although there is no universally accepted definition, a harness may include system prompts, tools, runtime mechanisms, verification rules, orchestration logic, and failure-recovery procedures. The same base model can thus exhibit substantially different performance under different harnesses [28, 5, 8].From early frameworks such as ReAct [29] to product- and platform-level systems such as Claude Code, Codex, and OpenHands, harnesses have largely been engineered by human experts [9, 16, 24, 36, 35]. While effective, this human-centered paradigm does not scale well with the diversity and rapid evolution of modern LLMs. Different models can exhibit distinct behavioral patterns, tool-use habits, error modes, and sensitivities to prompting [22, 21, 18]; consequently, a harness that works well for one model may be suboptimal for another [22, 5, 8]. As new models continue to be released at a rapid pace, manually redesigning and tuning a model-specific harness for each model becomes increasingly costly and untenable.迄今为止基于大语言模型LLM的智能体不仅由其基础模型决定还受到“框架”harness的影响即围绕模型运行并介导其与环境交互的系统。尽管目前尚无统一的定义但框架通常包括系统提示、工具、运行机制、验证规则、编排逻辑以及故障恢复流程等。因此同一基础模型在不同框架下可能表现出截然不同的性能[28, 5, 8]。从早期的ReAct框架[29]到Claude Code、Codex和OpenHands等产品级或平台级系统框架大多由人类专家设计[9, 16, 24, 36, 35]。尽管这种方法有效但以人类为中心的范式难以适应现代大语言模型日益增长的多样性及其快速演进的特点。不同模型可能表现出迥异的行为模式、工具使用习惯、错误类型以及对提示的敏感度[22, 21, 18]因此适用于某一模型的框架可能对另一模型效果不佳[22, 5, 8]。随着新模型以极快的速度不断发布为每个模型手动重新设计和调整专用的提示框架变得越来越昂贵且难以持续。In this paper, we explore a novel paradigm, Self-Harness: enabling an LLM-based agent to improve the very harness through which it operates (Figure 1). Unlike recent approaches that use stronger external agents to improve the harnesses of weaker ones [5, 8], Self-Harness seeks to internalize this improvement loop within the target agent itself. This paradigm reduces dependence on external guidance that may be costly, unavailable for frontier models, or mismatched to the target model’s failure modes. More broadly, in Bergson’s terms, this points toward a technical analogue of self-creation: a system not merely changed from without, but continually “going on creating itself.”We operationalize Self-Harness as an improvement loop that repeatedly turns behavioral evidence into harness updates (Figure 2). The loop consists of three stages. Weakness Mining: Starting from an initial harness, the agent with a fixed model is run on a set of tasks, producing execution traces with verifiable outcomes. The agent then clusters failed traces, allowing it to reason about model-specific failure patterns rather than isolated mistakes. Harness Proposal: Based on these failure patterns, the agent generates a small set of diverse yet minimal harness modifications, each tied to a specific failure mechanism. This constraint ensures that proposed edits remain targeted rather than overly general. Proposal Validation: Candidate modifications are evaluated through regression tests, and an edit is promoted only if it improves performance without causing measurable degradation on held-out tasks. If multiple candidate modifications pass the regression tests, they are merged into the next version of the harness, which then serves as the starting point for the next iteration.本文提出了一种新颖范式自适应提示Self-Harness即让基于大语言模型的智能体能够改进其自身所依赖的提示框架图1。与近期采用更强外部智能体来优化较弱智能体提示框架的方法[5, 8]不同自适应提示旨在将这种改进循环内化于目标智能体自身之中。该范式减少了对外部指导的依赖而这些指导可能成本高昂、无法用于前沿模型或与目标模型的失败模式不匹配。更广泛地说用柏格森的术语来说这指向了一种技术上的自我创造类比一个系统不仅从外部被改变而是持续地“自我生成”。我们将自适应提示具体化为一个改进循环该循环反复将行为证据转化为提示框架的更新图2。该循环包含三个阶段薄弱点挖掘——从初始提示框架开始使用固定模型的智能体在一组任务上运行生成带有可验证结果的执行轨迹。随后该代理会将失败的轨迹进行聚类从而能够针对模型特定的失败模式进行推理而非仅关注孤立的错误。Harness 提议基于这些失败模式代理生成一组少量、多样且最小化的 Harness 修改方案每个修改都与特定的失败机制相关联。这一约束确保了所提出的修改具有针对性而非过于宽泛。提议验证候选修改通过回归测试进行评估只有在提升性能的同时未对保留任务造成可测量的性能下降时才会被采纳。如果多个候选修改均通过回归测试则将其合并到下一版 Harness 中作为下一轮迭代的起点。In our experiments, we instantiate Self-Harness with a minimal initial harness (Figure 3) and three base models from diverse families: MiniMax M2.5, Qwen3.5-35B-A3B, and GLM-5 [2, 20, 14]. On Terminal-Bench-2.0, Self-Harness consistently improves performance across all three models (Figure 4). For held-in tasks, which provide execution traces to the evaluation system, the pass rate is increased from 43.0% to 50.0% for MiniMax M2.5, from 15.1% to 36.0% for Qwen3.5-35B-A3B, and from 47.7% to 57.0% for GLM-5. For held-out tasks, whose execution traces are never used as inputs to the evaluation system, the improvements remain substantial. The pass rate is increased from 40.5% to 61.9% for MiniMax M2.5, from 23.8% to 38.1% for Qwen3.5-35B-A3B, and from 42.9% to 57.1% for GLM-5. These results indicate that Self-Harness can evolve an initial harness into model-specific ones better suited to different base models. Moreover, it can discover broadly useful harness modifications that generalize to unseen tasks rather than merely overfitting to observed evaluation failures.Qualitative analyses further show that Self-Harness does more than simply make the prompt longer or add generic instructions. Instead, it introduces targeted changes that reflect the recurring problems each model encounters during execution, turning model-specific weaknesses into concrete harness-level interventions. For MiniMax M2.5, the changes encourage the agent to create required output files earlier, handle structured tool outputs more carefully, and stop unproductive tool-use loops before they become too long. For Qwen3.5-35B-A3B, the changes focus on checking dependencies in advance, avoiding repeated failed commands, breaking cycles of endless exploration, and reminding the agent to produce the required artifacts after tool errors. For GLM-5, the changes mainly help the agent preserve environment settings across shell commands and move more quickly from exploration to implementation and testing. Notably, Self-Harness can also introduce broader structural mechanisms, such as subagent-based decomposition and middleware creation, that go beyond local failure repair and improve the overall organization of problem solving.在我们的实验中我们使用一个最小化的初始引导框架图3和来自不同模型家族的三个基础模型MiniMax M2.5、Qwen3.5-35B-A3B 和 GLM-5 [2, 20, 14] 来实例化 Self-Harness。在 Terminal-Bench-2.0 上Self-Harness 在所有三个模型上均持续提升了性能图4。对于保留任务即向评估系统提供执行轨迹的任务MiniMax M2.5 的通过率从 43.0% 提升至 50.0%Qwen3.5-35B-A3B 从 15.1% 提升至 36.0%GLM-5 从 47.7% 提升至 57.0%。对于未见任务其执行轨迹从未作为输入提供给评估系统性能提升依然显著MiniMax M2.5 的通过率从 40.5% 提升至 61.9%Qwen3.5-35B-A3B 从 23.8% 提升至 38.1%GLM-5 从 42.9% 提升至 57.1%。这些结果表明Self-Harness 能够将初始引导框架演化为更适配不同基础模型的专用引导框架。此外它还能发现具有广泛适用性的引导修改方式使其能够泛化到未见过的任务而不仅仅是对已观察到的评估失败进行过拟合。定性分析进一步表明Self-Harness 并非仅仅延长提示或添加通用指令而是引入了针对每个模型在执行过程中反复出现的问题所做出的有针对性改进将模型特有的弱点转化为具体的操作层干预措施。对于 MiniMax M2.5这些改进促使智能体更早生成所需输出文件更谨慎地处理结构化工具输出并在无用的工具调用循环过长之前及时终止。对于 Qwen3.5-35B-A3B改进重点在于提前检查依赖关系、避免重复失败命令、打破无限探索循环并在工具出错后提醒智能体生成所需的中间产物。对于 GLM-5改进主要帮助智能体在不同 shell 命令之间保持环境设置并更快地从探索阶段转向实现与测试阶段。值得注意的是Self-Harness 还能引入更广泛的结构性机制例如基于子代理的分解和中间件的创建这些机制超越了局部故障修复提升了问题解决的整体组织效率。To summarize, our key contributions are as follows:• We propose Self-Harness, a novel paradigm for harness improvement that enables an LLM-based agent to design and refine the harness through which it operates, tailoring it to its own base model without human engineering effort or guidance from a stronger external agent.• We operationalize Self-Harness as an iterative loop that turns each model’s behavioral evidence into model-specific harness updates: it evaluates execution traces to identify recurring failure patterns, generates diverse yet minimal candidate edits, and promotes only those that pass regression tests.• Experiments on Terminal-Bench-2.0 show that Self-Harness improves performance across 3 models from diverse families, with absolute gains of up to 21.4 percentage points and relative improvements of up to 138%; qualitative analyses further confirm that different models benefit from distinct harness changes, suggesting that Self-Harness can turn model-specific weaknesses into concrete harness changes.综上所述我们的主要贡献如下• 我们提出了一种名为“自适应框架”Self-Harness的全新范式使基于大语言模型的智能体能够自主设计并优化其运行所依赖的框架无需人工工程干预或更强外部智能体的指导即可针对自身基础模型进行定制化调整。• 我们将自适应框架实现为一个迭代循环过程将每个模型的行为证据转化为特定于该模型的框架更新通过分析执行轨迹识别重复出现的失败模式生成多样但精简的候选修改方案并仅保留通过回归测试的修改内容。• 在Terminal-Bench-2.0上的实验表明自适应框架可提升来自不同家族的3个模型的性能绝对提升最高达21.4个百分点相对提升最高达138%定性分析进一步证实不同模型从不同的框架改进中获益表明自适应框架能够将模型特有的弱点转化为具体的框架优化措施。Figure 1:Three paradigms of harness improvement. In human harness engineering, human engineers manually revise the agent harness. In Meta-Harness, a stronger external agent guides the improvement of a weaker target agent. In Self-Harness, the agent improves its own operating harness.图1三种改进范式。在人类工程中人类工程师手动修改智能体的运行框架在元框架Meta-Harness中一个更强的外部智能体引导较弱的目标智能体进行改进在自框架Self-Harness中智能体自主优化自身的运行框架。Figure 2: Overview of one Self-Harness optimization loop. The current harness ht with fixed model is evaluated on tasks to collect execution traces, which are clustered into verifier-grounded failure patterns. The same model is then invoked under the current harness as a proposer, using the mined failure patterns to generate bounded candidate harness edits. Candidate edits are evaluated by regression tests on held-in and held-out splits. Accepted candidates are merged to update the harness to ht1, while rejected candidates are logged without changing the active harness. Throughout the loop, the model weights and evaluator remain fixed; only the surrounding harness is modified.图2自适应测试用例优化循环的概览。在固定模型的前提下当前测试用例ht在任务上执行以收集运行轨迹并将这些轨迹聚类为由验证器支持的失败模式。随后使用相同的模型作为提议者在当前测试用例下调用利用挖掘出的失败模式生成有限范围的候选测试用例修改。这些候选修改通过在保留集和测试集上的回归测试进行评估。被接受的候选修改将被合并以更新测试用例至ht1而被拒绝的则被记录但不改变当前活跃的测试用例。在整个循环过程中模型权重和评估器保持不变仅对周围的测试用例进行调整。5ConclusionThis paper studied whether a fixed language model can improve the harness that governs its own agent behavior. We introduced Self-Harness, a propose–evaluate–accept framework in which the model is evaluated under the current harness, receives structured evidence from its own execution traces, and proposes bounded edits to declared harness surfaces. Candidate harnesses are then re-evaluated under the same benchmark protocol, and only edits that satisfy a non-regressive acceptance rule are promoted into the harness lineage.The main lesson is that harness improvement should be treated as an empirical state transition. A useful harness edit must specify the behavior it aims to change, the surface it modifies, the evidence that motivates it, and the evaluation result that justifies promotion. By keeping the model, evaluator, and benchmark protocol fixed, Self-Harness isolates whether improved behavior comes from changes to the harness scaffold.Our experiments on Terminal-Bench-2.0 instantiate this protocol with a minimal DeepAgent-based baseline harness and three model backends. Self-Harness improves Pass (%) across all tested backends while preserving held-in and held-out performance under the acceptance rule. The retained edits are small, auditable changes to configurable harness surfaces, suggesting that even sparse initial harnesses can support useful self-improvement when proposals are constrained by execution evidence and validated by regression testing.本文研究了固定语言模型是否能够改进其自身代理行为所依赖的控制框架。我们提出了“自控框架”Self-Harness这是一个“提出—评估—采纳”的框架模型在当前框架下进行评估从自身的执行轨迹中获取结构化证据并对已声明的框架界面提出有限修改。随后候选框架会在相同的基准协议下重新评估只有满足非退化接受规则的修改才会被纳入框架演化序列。主要结论是框架的改进应被视为一种经验性的状态转移过程。一个有效的框架修改必须明确指出其旨在改变的行为、所修改的接口、支持该修改的证据以及证明其可被采纳的评估结果。通过保持模型、评估器和基准协议不变自控框架能够隔离出行为改善究竟是源于框架结构本身的调整。我们在Terminal-Bench-2.0上的实验以一个基于DeepAgent的最小基线框架和三个模型后端实现了这一协议。Self-Harness 在所有测试的后端上均提升了通过率Pass (%)同时在接纳规则下保持了保留和排除性能。所保留的修改是对可配置的Harness表面进行的小幅、可审计的更改表明即使初始Harness较为稀疏只要对提案施加执行约束仍能支持有效的自我改进。Self-Harness also has important limits. It studies bounded harness edits under fixed benchmarks, not open-ended self-improvement. Accepted edits may still reflect benchmark-specific failure patterns, and the protocol depends on the quality of verifier outcomes and trace records. Higher-stakes harness changes would require stronger acceptance gates than pass-rate non-regression alone.More broadly, Self-Harness points toward a style of agent engineering in which harnesses evolve through recorded, testable, and reversible changes. Future work can further explore application of self-harness-style self-improvement in broader environments, but the core requirement remains the same: self-improvement should be grounded in behavioral evidence rather than only in the proposer’s rationale for a plausible edit.Self-Harness 也存在重要局限性。它研究的是在固定基准下的有限Harness修改而非开放式的自我提升。被接受的修改可能仍反映特定基准的失败模式且该协议依赖于验证器结果和追踪记录的质量。对于高风险的Harness变更需要比仅基于通过率无回归更严格的接受机制。更广泛而言Self-Harness 指向了一种代理工程范式Harness 通过可记录、可测试且可逆的变更逐步演化。未来工作可进一步探索将此类自改进机制应用于更广泛的环境但核心要求始终不变自我改进应以行为证据为基础而不仅仅依赖于提案者对某项合理修改的推理依据。