jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
91fba2c2e9
OpenCompass/opencompass/datasets
History
Hubert 91fba2c2e9
[Feat] support humaneval and mbpp pass@k (#598) 
* [Feat] support pass@ k

* [Feat] support pass@k

* [Feat] support pass@k

* [Feat] support pass@k

* [Feat] support pass@k

* [Feat] support pass@k docs

* update naming

---------

Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-16 21:22:06 +08:00
..
agieval [Sync] update (#517) 2023-10-27 20:31:22 +08:00
lawbench [Sync] sync with internal codes 20231019 (#488) 2023-10-18 23:37:35 -05:00
leval [Sync] Update LongEval (#443) 2023-09-27 16:32:40 +08:00
longbench [Sync] Update LongEval (#443) 2023-09-27 16:32:40 +08:00
__init__.py [Feature] Add FinanceIQ dataset (#596) 2023-11-16 17:47:57 +08:00
advglue.py [Feat] support adv_glue dataset for adversarial robustness (#205) 2023-08-16 18:42:06 +08:00
afqmcd.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
anli.py [Feature] Add Xiezhi SQuAD2.0 ANLI (#101) 2023-08-10 14:04:18 +08:00
anthropics_evals.py [Feat] support antropics evals dataset (#422) 2023-09-20 18:36:44 +08:00
arc.py Add Release Contraibution 2023-07-05 02:22:40 +00:00
ax.py Add release contribution 2023-07-05 03:15:31 +00:00
base.py Add release contribution 2023-07-05 03:15:31 +00:00
bbh.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
boolq.py [Feature] add llama-oriented dataset configs (#82) 2023-08-11 12:48:05 +08:00
bustum.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
c3.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
cb.py [Feat] support opencompass 2023-07-04 22:11:33 +08:00
ceval.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
chid.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
cibench.py [Fix] fix unnecessary import and update requirements (#555) 2023-11-08 17:58:49 +08:00
civilcomments.py [Feat] support opencompass 2023-07-04 22:11:33 +08:00
clozeTest_maxmin.py [Feature] Add py150 and maxmin (#562) 2023-11-09 22:05:25 +08:00
cluewsc.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
cmb.py fix cmb dataset (#587) 2023-11-14 16:13:39 +08:00
cmmlu.py [Feature] Add CMMLU dataset (#91) 2023-07-25 10:14:27 +08:00
cmnli.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
cmrc.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
commonsenseqa.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
copa.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
crowspairs.py update (#251) 2023-08-23 16:25:23 +08:00
csl.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
cvalues.py [Feat] Support CValues Responsibility dataset (#78) 2023-07-18 18:45:15 +08:00
drcd.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
drop.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
ds1000_interpreter.py [Feat] Support cibench (#538) 2023-11-07 19:11:44 +08:00
ds1000.py [Feat] support ds1000 dataset (#395) 2023-09-15 12:50:27 +08:00
eprstmt.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
FinanceIQ.py [Feature] Add FinanceIQ dataset (#596) 2023-11-16 17:47:57 +08:00
flores.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
game24.py [Fix] use sympy only when necessary (#255) 2023-08-24 10:15:20 +08:00
GaokaoBench.py [Feature] Add logger info and remove dataset bugs (#61) 2023-07-17 14:26:30 +08:00
govrepcrs.py Add release contribution 2023-07-05 03:15:31 +00:00
gsm8k.py [Sync] update model configs (#574) 2023-11-13 15:15:34 +08:00
hellaswag.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
huggingface.py [Feat] support opencompass 2023-07-04 22:11:33 +08:00
humaneval.py [Feat] support humaneval and mbpp pass@k (#598) 2023-11-16 21:22:06 +08:00
humanevalx.py [Feat] support codellama and preds collection tools (#335) 2023-08-31 11:14:42 +08:00
iwslt2017.py Add release contribution 2023-07-05 03:15:31 +00:00
jigsawmultilingual.py initial commit 2023-07-04 21:34:55 +08:00
kaoshi.py [Feature] Add kaoshi dataset (#392) 2023-09-22 18:46:33 +08:00
lambada.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
lcsts.py Support a batch of datasets. 2023-07-05 01:30:27 +00:00
lmeval.py [Sync] update github token (#475) 2023-10-13 06:50:54 -05:00
math.py [Feat] Support cibench (#538) 2023-11-07 19:11:44 +08:00
mathbench.py [Feature] Add mathbench dataset and circular evaluator (#408) 2023-10-18 04:08:31 -05:00
mbpp.py [Feat] support humaneval and mbpp pass@k (#598) 2023-11-16 21:22:06 +08:00
mmlu.py [Feature] Add logger info and remove dataset bugs (#61) 2023-07-17 14:26:30 +08:00
multirc.py initial commit 2023-07-04 21:34:55 +08:00
narrativeqa.py Add release contribution 2023-07-05 03:15:31 +00:00
natural_question.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
obqa.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
piqa.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
py150.py [Feature] Add py150 and maxmin (#562) 2023-11-09 22:05:25 +08:00
qasper.py Add Release Contraibution 2023-07-05 02:22:40 +00:00
qaspercut.py Add Release Contraibution 2023-07-05 02:22:40 +00:00
race.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
realtoxicprompts.py Update configs (#9) 2023-07-06 12:27:41 +08:00
record.py [Feature] Add qwen & qwen-chat support (#286) 2023-08-31 11:29:05 +08:00
safety.py Add Release Contraibution 2023-07-05 02:22:40 +00:00
scibench.py add evaluation of scibench (#393) 2023-09-22 17:42:08 +08:00
siqa.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
squad20.py [Feature] Add Xiezhi SQuAD2.0 ANLI (#101) 2023-08-10 14:04:18 +08:00
storycloze.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
strategyqa.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
subjective_cmp.py Fix bugs in subjective evaluation (#589) 2023-11-14 16:11:55 +08:00
summedits.py [Enhancement] Test linting in CI and fix existing linting errors (#69) 2023-07-17 15:59:10 +08:00
summscreen.py Support a batch of datasets. 2023-07-05 01:30:27 +00:00
tabmwp.py [fFeat] Add an opensource dataset Tabmwp (#505) 2023-11-03 11:15:46 +08:00
TheoremQA.py Update configs (#9) 2023-07-06 12:27:41 +08:00
tnews.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
triviaqa.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
triviaqarc.py Add Release Contraibution 2023-07-05 02:22:40 +00:00
truthfulqa.py [Feat] refine docs and codes for more user guides (#409) 2023-09-18 16:12:13 +08:00
tydiqa.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
wic.py Support a batch of datasets. 2023-07-05 01:30:27 +00:00
winograd.py initial commit 2023-07-04 21:34:55 +08:00
winogrande.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
wnli.py [Feat] implementation for support promptbench (#239) 2023-09-15 15:06:53 +08:00
wsc.py Update configs (#9) 2023-07-06 12:27:41 +08:00
xcopa.py Add Release Contraibution 2023-07-05 02:22:40 +00:00
xiezhi.py [Feature] Add Xiezhi SQuAD2.0 ANLI (#101) 2023-08-10 14:04:18 +08:00
xlsum.py update datasets 2023-07-05 01:45:26 +00:00
xsum.py update datasets 2023-07-05 01:45:26 +00:00