mirror of
https://github.com/open-compass/opencompass.git
synced 2025-05-30 16:03:24 +08:00
| .. | ||
| mbpp_pro_gen_3dc067.py | ||
| mbpp_pro_gen.py | ||
| mbpp_pro_repeat_gen_3dc067.py | ||
| README.md | ||
MBPP pro
OC results
| model | pass@1 |
|---|---|
| qwen2.5-coder-7b-instruct-hf | 66 |
| qwen2.5-14b-instruct-hf | 64 |
| deepseek-v2-lite-chat-hf | 36 |
CodeEval-pro results
| model | pass@1 |
|---|---|
| qwen2.5-coder-7b-instruct-hf | 65 |
| qwen2.5-14b-instruct-hf | 65 |
| deepseek-v2-lite-chat-hf | 39 |