jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
d939e32438
OpenCompass/opencompass/configs/datasets/mbpp_pro
History
Dongsheng Zhu d939e32438 add bench
2025-05-09 02:36:39 +00:00
..
mbpp_pro_gen_3dc067.py add bench 2025-05-09 02:36:39 +00:00
mbpp_pro_gen.py add bench 2025-05-09 02:36:39 +00:00
README.md add bench 2025-05-09 02:36:39 +00:00

README.md

MBPP pro

OC results

model pass@1
qwen2.5-coder-7b-instruct-hf 66
qwen2.5-14b-instruct-hf 64
deepseek-v2-lite-chat-hf 36

CodeEval-pro results

model pass@1
qwen2.5-coder-7b-instruct-hf 65
qwen2.5-14b-instruct-hf 65
deepseek-v2-lite-chat-hf 39