DevBench: A Realistic, Developer-Informed Benchmark for Code Generation Models
Published • May 16, 2026
Authors:,,
Adarsh Kumarappan
Pareesa Ameneh Golnari
Wen Wen
Abstract
DevBench is a telemetry-driven benchmark designed to evaluate Large Language Models (LLMs) on realistic code completion tasks. It includes 1,800 evaluation instances across six programming languages and six task categories derived from real developer telemetry and synthesized using generator models ...
Subject
Finding related papers...
Discussions
(0)No comments yet
Be the first to share your thoughts!