jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
103304b15b
OpenCompass/opencompass/configs/datasets/mbpp_pro
History
Dongsheng Zhu 2c79dc5227
[Dataset] Add human_eval/mbpp pro (#2092) 
* add bench

* update

* bug fix

* time update

* add index

* fix repeat bug
2025-05-12 18:38:13 +08:00
..
mbpp_pro_gen_3dc067.py [Dataset] Add human_eval/mbpp pro (#2092) 2025-05-12 18:38:13 +08:00
mbpp_pro_gen.py [Dataset] Add human_eval/mbpp pro (#2092) 2025-05-12 18:38:13 +08:00
mbpp_pro_repeat_gen_3dc067.py [Dataset] Add human_eval/mbpp pro (#2092) 2025-05-12 18:38:13 +08:00
README.md [Dataset] Add human_eval/mbpp pro (#2092) 2025-05-12 18:38:13 +08:00

README.md

MBPP pro

OC results

model pass@1
qwen2.5-coder-7b-instruct-hf 66
qwen2.5-14b-instruct-hf 64
deepseek-v2-lite-chat-hf 36

CodeEval-pro results

model pass@1
qwen2.5-coder-7b-instruct-hf 65
qwen2.5-14b-instruct-hf 65
deepseek-v2-lite-chat-hf 39