jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
main
OpenCompass/opencompass/configs/datasets/SimpleQA
History
liushz c437135fad
[Feature] Add Openai Simpleqa dataset (#1720) 
* Add Openai SimpleQA dataset

* Add Openai SimpleQA dataset

* Add Openai SimpleQA dataset

* Update eval_simpleqa.py

---------

Co-authored-by: Linchen Xiao <xxllcc1993@gmail.com>
2024-11-28 19:16:07 +08:00
..
README.md [Feature] Add Openai Simpleqa dataset (#1720) 2024-11-28 19:16:07 +08:00
simpleqa_gen_0283c3.py [Feature] Add Openai Simpleqa dataset (#1720) 2024-11-28 19:16:07 +08:00
simpleqa_gen.py [Feature] Add Openai Simpleqa dataset (#1720) 2024-11-28 19:16:07 +08:00

README.md

OpenCompass SimpleQA dataset config for evaluation

1. Introduction

SimpleQA is a benchmark that evaluates the ability of language models to answer short, fact-seeking questions by OpenAI. The original site is https://github.com/openai/simple-evals.

2. How to use

Please refer to the demo evaluation script /opencompass/configs/mine/simpleqa_eval.py.