# MBPP pro

## OC results

|           model            |   pass@1 |
|:--------------------------:|---------:|
|qwen2.5-coder-7b-instruct-hf|     66   |
|  qwen2.5-14b-instruct-hf   |     64   |
|  deepseek-v2-lite-chat-hf  |     36   |

## CodeEval-pro results

|           model            |   pass@1 |
|:--------------------------:|---------:|
|qwen2.5-coder-7b-instruct-hf|     65   |
|  qwen2.5-14b-instruct-hf   |     65   |
|  deepseek-v2-lite-chat-hf  |     39   |