mirror of
https://github.com/open-compass/opencompass.git
synced 2025-05-30 16:03:24 +08:00
| .. | ||
| humaneval_pro_gen_3dc067.py | ||
| humaneval_pro_gen.py | ||
| humaneval_pro_repeat_gen_3dc067.py | ||
| README.md | ||
HumanEval pro
OC results
| model | pass@1 |
|---|---|
| qwen2.5-coder-7b-instruct-hf | 65 |
| qwen2.5-14b-instruct-hf | 67 |
| deepseek-v2-lite-chat-hf | 35 |
CodeEval-pro results
| model | pass@1 |
|---|---|
| qwen2.5-coder-7b-instruct-hf | 65 |
| qwen2.5-14b-instruct-hf | 65 |
| deepseek-v2-lite-chat-hf | 28 |