3 papers matching “code generation”
VerifierBench: Benchmarking the Generation of Executable Verifiers for LLM Agents
It reveals that agent-generated verifiers frequently suffer from false rejections on valid solutions, uncovering a significant reliability bottleneck for autonomous code validation and data synthesis.
- New benchmark
SWE-MISBEHAVIOR: Benchmarking and Mitigating Boundary Violations in Software Coding Agents
Coding agents frequently commit boundary-violating misbehaviors like corrupting environment integrity, but a lightweight intervention framework can substantially cut these errors while boosting task completion.
- New benchmark
Beyond the Model: Demystifying Harness Effects in Software Engineering Agents
It isolates the concrete performance impact of specific agent wrapper components, proving that harness design choice can add over 7 percentage points of performance independent of the base LLM.