A New Architecture for Search Agents
Large language models (LLMs) show promise as search agents in complex information-seeking tasks, yet they often struggle with long-horizon planning, state tracking, and coherent multi-step reasoning. The paper introduces Harness-1, a framework that equips search agents with a structured, externalized memory called a harness. This harness acts as an explicit, evolving state representation that the agent reads from and writes to during a search episode.
Unlike purely implicit chain-of-thought approaches, the harness externalizes the agent’s current goals, findings, and sub-question status, making the search process more transparent and controllable. The core idea draws inspiration from classic Reinforcement Learning: An Introduction by Sutton and Barto, where clear state representations are fundamental to effective decision-making. By giving the agent a dedicated workspace, Harness-1 aims to improve planning depth and reduce the cognitive load on the underlying LLM, enabling more robust performance on deep research tasks.
Training Agents with Reinforcement Learning
A central challenge in building search agents is the lack of supervised training data with optimal search trajectories. Harness-1 tackles this by using reinforcement learning (RL) to train the agent end-to-end. The agent is rewarded based on the quality of its final answer, allowing it to discover effective search strategies without human demonstrations.
The training loop treats the search process as a sequential decision-making problem. At each step, the agent issues a query, receives results, and updates its external harness. A policy gradient method optimizes the agent’s behavior to maximize the expected reward. This approach is conceptually related to reinforcement learning from human feedback (RLHF), but the reward signal here comes from an automated evaluation of the final output rather than a learned human preference model. The result is an agent that learns to balance exploration and exploitation, deciding when to dive deeper into a topic and when to synthesize an answer from gathered information.
The Harness: Externalizing Agent State
The key innovation is the harness itself — a structured, textual state that the agent maintains throughout a search session. Rather than relying solely on the LLM’s internal context window, the harness explicitly tracks:
- The original user question and any decomposed sub-questions.
- Information gathered so far, with citations.
- The current status of each sub-question (pending, in-progress, answered).
- A running draft of the final answer.
At each turn, the agent reads the current harness, decides on an action (e.g., search for a specific query, refine a sub-question, or finalize the answer), and then writes updates back to the harness. This read-write cycle creates a tight feedback loop. The externalized state makes the agent’s reasoning auditable and allows it to recover from dead ends by explicitly marking failed search directions. The harness design is general and can be adapted to various search environments and LLM backbones.
Evaluation on Deep Research Benchmarks
The paper evaluates Harness-1 on challenging, open-ended question-answering benchmarks that require multi-step web search and synthesis. The primary testbed is Harness-100, a curated set of 100 diverse, complex questions spanning science, history, and current events. Performance is measured by both automated metrics and human evaluation of answer completeness and accuracy.
Harness-1 significantly outperforms baseline LLM agents that lack an externalized state or are trained with imitation learning. The RL-trained agent learns to conduct more thorough investigations, issuing more diverse queries and spending more time on difficult sub-questions. Ablation studies confirm that both the harness structure and the RL training are crucial: removing the harness degrades performance, and switching to behavioral cloning reduces the agent’s ability to explore effectively. The agent generalizes beyond its training distribution, showing robust behavior on unseen question types.
Why Externalized State Matters
The success of Harness-1 underscores a broader principle in AI: externalizing cognitive state can dramatically improve an agent’s ability to handle complex, long-horizon tasks. By maintaining a persistent, structured memory, the agent avoids the context-window limitations and attention dilution that plague purely implicit reasoning approaches.
This design also makes the agent more interpretable. A human can inspect the harness at any point to understand what the agent knows, what it is currently investigating, and why it made certain decisions. In high-stakes applications like scientific research or legal analysis, this transparency is essential. The harness acts as a search agent’s notebook, capturing the evolving investigation in a way that is both machine-readable and human-auditable. This externalization strategy could influence the design of future autonomous agents beyond search, including coding assistants and task-planning systems.
Limitations and Future Directions
While Harness-1 represents a significant step forward, the paper acknowledges several limitations. The current harness structure is hand-designed, which may not be optimal for all domains. Future work could explore learning the harness schema itself. The RL training process is computationally intensive, requiring many episodes of simulated search. The reward function, based on final answer quality, is sparse and may not provide fine-grained feedback on intermediate steps.
The authors suggest several promising directions: incorporating more sophisticated credit assignment to reward good intermediate decisions, extending the harness to support multi-modal information like images and tables, and applying the framework to other search agents in AI contexts such as code generation or database querying. Scaling up the training to even larger models and more diverse question sets could further improve robustness and generalization.
