ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation
Active
Evaluates LLMs on class-level code generation with 100 tasks constructed over 500 person-hours. The study shows that LLMs perform worse on class-level tasks compared to method-level tasks.
- Publisher
- Fudan University
- Domain
- Coding
- License
- mit
- Published
- Nov 2024
- Notable for
- Benchmark for evaluating Coding.
Cite
Notes
Only stored in your browser.
FAQ
- What is ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation?
- Evaluates LLMs on class-level code generation with 100 tasks constructed over 500 person-hours. The study shows that LLMs perform worse on class-level tasks compared to method-level tasks.
- What license is ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation under?
- ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation is available under mit.