Diff-Based Code Corruption using LLMs for Large-Scale Bugfix Benchmarking

There are various benchmarks to evaluate bugfixing capabilities of Large Language Models. However, most widespread benchmarks do not fully reflect real-world bugfixing practices. They are small, weakening statistical reliability, and the buggy programs are often similar to one another, potentially d... View More