jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
f2af49337d
OpenCompass/opencompass/datasets
History
Mo Li f2af49337d
[Feature] Add ATC Choice Version (#1019) 
* Squashed commit of the following:

commit c48ad194c3976dc63d1b60d8c8ab2d5ff9e1cbfe
Author: DseidLi <2568818204@qq.com>
Date:   Tue Apr 2 16:57:43 2024 +0800

    add atc_choice

commit 3ac6efea29619573e6fac8fa3cce464853dcead0
Merge: 2d4e559 8e3a9c3
Author: DseidLi <2568818204@qq.com>
Date:   Tue Apr 2 16:41:38 2024 +0800

    Merge branch 'atc_choice' into atc_add_choice

commit 8e3a9c396a3e5546d3faf584183f6fd60b974d5e
Merge: 150a036 0a6a03f
Author: DseidLi <2568818204@qq.com>
Date:   Tue Mar 26 04:47:07 2024 +0800

    Merge branch 'main' into atc_choice

    Conflicts:
    	configs/summarizers/needlebench.py
    	opencompass/datasets/needlebench/multi.py
    	opencompass/datasets/needlebench/origin.py
    	opencompass/datasets/needlebench/parallel.py

commit 150a036d6d990f26a57c974d1af83d88c31a0f9d
Merge: 8d6ac9a 940dd18
Author: DseidLi <2568818204@qq.com>
Date:   Wed Mar 20 03:49:08 2024 +0800

    Merge branch 'needlebench_fix' into atc_choice

commit 8d6ac9a1a43b1c9d0f0ea27e7d58968a203ea898
Author: DseidLi <2568818204@qq.com>
Date:   Wed Mar 20 03:41:49 2024 +0800

    optimize needlebench code

commit 940dd18a4270f24bc69edd2a780182c68918e1a9
Author: DseidLi <2568818204@qq.com>
Date:   Wed Mar 20 03:39:46 2024 +0800

    fix vllm

commit d8be6877bc41051f3edcc0421c462c834c0f1c9a
Merge: ecad78a 2527fda
Author: DseidLi <2568818204@qq.com>
Date:   Tue Mar 19 21:07:08 2024 +0800

    Merge remote-tracking branch 'origin/add_1M_dataset' into atc_choice

commit 2527fda8a5
Author: DseidLi <2568818204@qq.com>
Date:   Tue Mar 19 16:03:40 2024 +0800

    add model configs

commit 75425acdf8
Author: DseidLi <2568818204@qq.com>
Date:   Tue Mar 19 16:02:15 2024 +0800

    add prompt postion args

commit 367ba1ba61
Author: DseidLi <2568818204@qq.com>
Date:   Wed Feb 28 21:40:00 2024 +0800

    add Needlebench-1000K configs

commit ecad78af14c4bb00fe325779114b384c57ab30bf
Author: DseidLi <2568818204@qq.com>
Date:   Thu Mar 14 22:08:32 2024 +0800

    fix atc

commit 08772c0787b18872abadc9ffec3223941a5ee0c2
Merge: 9f3f8cf caf1cf8
Author: DseidLi <2568818204@qq.com>
Date:   Thu Mar 14 22:07:28 2024 +0800

    Merge branch 'main' into atc_choice

    Conflicts:
    	configs/datasets/needlebench/readme.md
    	configs/datasets/needlebench/readme_zh-CN.md
    	configs/summarizers/needlebench.py
    	opencompass/datasets/needlebench/atc.py
    	opencompass/summarizers/needlebench.py

commit 9f3f8cfb4452722734d334114ac1d14110e57406
Author: DseidLi <2568818204@qq.com>
Date:   Thu Mar 14 21:35:53 2024 +0800

    add atc-choice test

commit 52be7c1202376b4e09821188b826f1a805328129
Author: DseidLi <2568818204@qq.com>
Date:   Wed Mar 6 02:54:15 2024 +0800

    update needlebench randomseed and add vllm qwen14b

commit fc1effce596ae2e5ece4933e8cd34aef8e64a6f9
Merge: 4e747ed caf1cf8
Author: DseidLi <2568818204@qq.com>
Date:   Wed Mar 6 02:51:14 2024 +0800

    Merge branch 'main' into add_model_configs

commit 31834f9b23af3354ac3581ec86d693d0f05cdd1c
Merge: 7dabc82 120bf8b
Author: DseidLi <2568818204@qq.com>
Date:   Sun Mar 3 23:29:42 2024 +0800

    Merge branch 'main' of https://github.com/open-compass/opencompass into atc_choice

commit 4e747ed1988ddbcfcc7fff334601259ade72d363
Author: DseidLi <2568818204@qq.com>
Date:   Sun Mar 3 22:15:25 2024 +0800

    add internlm2-lmdeploy model and gemma configs

commit 7dabc828123d711c8cf834d6aab4137bb55e85ed
Author: DseidLi <2568818204@qq.com>
Date:   Sat Mar 2 17:26:15 2024 +0800

    add atc choice version -ZH

commit 996f8ae43d
Author: DseidLi <2568818204@qq.com>
Date:   Wed Feb 28 16:58:56 2024 +0800

    update readme for needlebench

commit f7266e873c
Author: DseidLi <2568818204@qq.com>
Date:   Wed Feb 28 16:44:53 2024 +0800

    move readme.md

commit 1c7375681d
Author: DseidLi <2568818204@qq.com>
Date:   Wed Feb 28 16:38:31 2024 +0800

    fix linting error

commit b6524f3ebf
Author: DseidLi <2568818204@qq.com>
Date:   Wed Feb 28 16:33:51 2024 +0800

    lint summarizer

commit c0d1190e39
Author: DseidLi <2568818204@qq.com>
Date:   Wed Feb 28 16:29:03 2024 +0800

    add needlebench intro, fix summarizer

commit 0965baf785
Author: DseidLi <2568818204@qq.com>
Date:   Mon Feb 26 13:31:26 2024 +0800

    fix bug in needlebench summarizer

commit 5d32b31eb8
Author: DseidLi <2568818204@qq.com>
Date:   Sat Feb 24 03:19:08 2024 +0800

    update act prompt

commit af82a7f085
Merge: 32bf9fe 53fe788
Author: DseidLi <2568818204@qq.com>
Date:   Fri Feb 23 17:50:32 2024 +0800

    Merge remote-tracking branch 'upstream/main' into needlebench

commit 32bf9fe802
Author: DseidLi <2568818204@qq.com>
Date:   Fri Feb 23 17:31:32 2024 +0800

    simplify needlebench 32k, 128k, 200k for eval

commit a7cb025e05
Author: DseidLi <2568818204@qq.com>
Date:   Fri Feb 23 14:48:58 2024 +0800

    add needlebench

* fix summarizer

* remove repeated code

* remove chinese comments
2024-04-07 15:46:20 +08:00
..
agieval [Sync] update (#517) 2023-10-27 20:31:22 +08:00
IFEval [Sync] Sync Internal (#941) 2024-03-04 14:42:36 +08:00
infinitebench [Feature] Add InfiniteBench (#739) 2023-12-26 15:36:27 +08:00
lawbench fix lawbench 2-1 f0.5 score calculation bug (#795) 2024-02-06 22:20:11 +08:00
leval [Sync] Update LongEval (#443) 2023-09-27 16:32:40 +08:00
longbench [Sync] Fix cmnli, fix vicuna meta template, fix longbench postprocess and other minor fixes (#625) 2023-11-23 14:05:59 +08:00
lveval [Feature] add lveval benchmark (#914) 2024-03-04 11:22:03 +08:00
medbench [Fix] Update MedBench (#845) 2024-01-26 17:56:13 +08:00
needlebench [Feature] Add ATC Choice Version (#1019) 2024-04-07 15:46:20 +08:00
NPHardEval Support NPHardEval (#835) 2024-02-05 15:52:28 +08:00
reasonbench [Sync] Sync with internal codes 2023.01.08 (#777) 2024-01-08 14:07:24 +00:00
subjective [Feature] add mtbench (#829) 2024-01-24 12:11:47 +08:00
teval [Sync] Merge branch 'dev' into zfz/update-keyset-demo (#876) 2024-02-05 23:29:10 +08:00
__init__.py [Feature] Add the implement of QuALITY datasets (#976) 2024-03-15 21:22:38 +08:00
advglue.py [Feat] support adv_glue dataset for adversarial robustness (#205) 2023-08-16 18:42:06 +08:00
afqmcd.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
anli.py [Feature] Add Xiezhi SQuAD2.0 ANLI (#101) 2023-08-10 14:04:18 +08:00
anthropics_evals.py [Feat] support antropics evals dataset (#422) 2023-09-20 18:36:44 +08:00
apps.py [Fix] Update APPS/TACO (#988) 2024-03-19 20:21:39 +08:00
arc.py [Feature] Contamination analysis for MMLU, Hellaswag, and ARC_c (#699) 2024-01-08 15:51:48 +08:00
ax.py Add release contribution 2023-07-05 03:15:31 +00:00
base.py Add release contribution 2023-07-05 03:15:31 +00:00
bbh.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
boolq.py [Feature] add llama-oriented dataset configs (#82) 2023-08-11 12:48:05 +08:00
bustum.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
c3.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
cb.py [Feat] support opencompass 2023-07-04 22:11:33 +08:00
ceval.py [Feature] Add Data Contamination Analysis (#639) 2023-12-08 10:00:11 +08:00
chid.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
cibench.py [Feat] minor update agent related (#839) 2024-01-26 14:15:51 +08:00
circular.py [Sync] Add InternLM2 Keyset Evaluation Demo (#807) 2024-01-17 13:48:12 +08:00
civilcomments.py [Feat] support opencompass 2023-07-04 22:11:33 +08:00
clozeTest_maxmin.py [Feature] Add py150 and maxmin (#562) 2023-11-09 22:05:25 +08:00
cluewsc.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
cmb.py [Sync] Fix cmnli, fix vicuna meta template, fix longbench postprocess and other minor fixes (#625) 2023-11-23 14:05:59 +08:00
cmmlu.py [Feature] Add CMMLU dataset (#91) 2023-07-25 10:14:27 +08:00
cmnli.py [Sync] minor test (#683) 2023-12-11 17:42:53 +08:00
cmrc.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
commonsenseqa_cn.py [Feature] Add Chinese version: commonsenseqa, crowspairs and nq (#144) 2023-11-30 15:33:02 +08:00
commonsenseqa.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
copa.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
crowspairs_cn.py [Feature] Add Chinese version: commonsenseqa, crowspairs and nq (#144) 2023-11-30 15:33:02 +08:00
crowspairs.py update (#251) 2023-08-23 16:25:23 +08:00
csl.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
custom.py [Sync] Sync with internal codes 2023.01.08 (#777) 2024-01-08 14:07:24 +00:00
cvalues.py [Feat] Support CValues Responsibility dataset (#78) 2023-07-18 18:45:15 +08:00
drcd.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
drop.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
ds1000_interpreter.py [Feat] Support cibench (#538) 2023-11-07 19:11:44 +08:00
ds1000.py [Sync] Add InternLM2 Keyset Evaluation Demo (#807) 2024-01-17 13:48:12 +08:00
eprstmt.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
FinanceIQ.py [Feature] Add FinanceIQ dataset (#596) 2023-11-16 17:47:57 +08:00
flores.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
game24.py [Fix] use sympy only when necessary (#255) 2023-08-24 10:15:20 +08:00
GaokaoBench.py [Feature] Add logger info and remove dataset bugs (#61) 2023-07-17 14:26:30 +08:00
govrepcrs.py Add release contribution 2023-07-05 03:15:31 +00:00
gpqa.py [Sync] update 20240308 (#953) 2024-03-11 22:34:19 +08:00
gsm8k.py fix bug of gsm8k_postprocess (#863) 2024-02-06 23:52:47 +08:00
gsm_hard.py [Feature] Add GSM_Hard dataset (#619) 2023-11-27 17:40:34 +08:00
hellaswag.py [Sync] Sync Internal (#941) 2024-03-04 14:42:36 +08:00
huggingface.py [Feat] support opencompass 2023-07-04 22:11:33 +08:00
humaneval_multi.py [feat] support multipl-e (#846) 2024-02-06 23:30:28 +08:00
humaneval.py [Sync] Sync Internal (#941) 2024-03-04 14:42:36 +08:00
humanevalx.py [Sync] Add InternLM2 Keyset Evaluation Demo (#807) 2024-01-17 13:48:12 +08:00
hungarian_math.py [Sync] Sync with internal codes 2023.01.08 (#777) 2024-01-08 14:07:24 +00:00
iwslt2017.py Add release contribution 2023-07-05 03:15:31 +00:00
jigsawmultilingual.py initial commit 2023-07-04 21:34:55 +08:00
jsonl.py [Sync] Sync with internal codes 2023.01.08 (#777) 2024-01-08 14:07:24 +00:00
kaoshi.py [Feature] Add kaoshi dataset (#392) 2023-09-22 18:46:33 +08:00
lambada.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
lcsts.py Support a batch of datasets. 2023-07-05 01:30:27 +00:00
lmeval.py [Sync] update github token (#475) 2023-10-13 06:50:54 -05:00
mastermath2024v1.py [Feature] Add new dataset mastermath2024v1 (#744) 2024-01-01 15:53:24 +08:00
math401.py [Sync] Sync with internal codes 2023.01.08 (#777) 2024-01-08 14:07:24 +00:00
math_intern.py [Sync] Updata dataset cfg for internMath (#837) 2024-01-24 16:30:32 +08:00
math.py [Sync] Sync with internal codes 2023.01.08 (#777) 2024-01-08 14:07:24 +00:00
mathbench.py Add aritch to mathbench (#607) 2023-11-20 19:40:41 +08:00
mbpp.py [Sync] update 20240308 (#953) 2024-03-11 22:34:19 +08:00
mmlu.py [Feature] Contamination analysis for MMLU, Hellaswag, and ARC_c (#699) 2024-01-08 15:51:48 +08:00
multirc.py initial commit 2023-07-04 21:34:55 +08:00
narrativeqa.py Add release contribution 2023-07-05 03:15:31 +00:00
natural_question_cn.py [Feature] Add Chinese version: commonsenseqa, crowspairs and nq (#144) 2023-11-30 15:33:02 +08:00
natural_question.py [Sync] Sync Internal (#941) 2024-03-04 14:42:36 +08:00
obqa.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
OpenFinData.py [Feature] Support OpenFinData (#896) 2024-02-29 12:55:07 +08:00
piqa.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
py150.py [Feature] Add py150 and maxmin (#562) 2023-11-09 22:05:25 +08:00
qasper.py Add Release Contraibution 2023-07-05 02:22:40 +00:00
qaspercut.py Add Release Contraibution 2023-07-05 02:22:40 +00:00
QuALITY.py [Feature] Add the implement of QuALITY datasets (#976) 2024-03-15 21:22:38 +08:00
race.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
realtoxicprompts.py Update configs (#9) 2023-07-06 12:27:41 +08:00
record.py [Feature] Add qwen & qwen-chat support (#286) 2023-08-31 11:29:05 +08:00
rolebench.py added rolebench dataset. (#633) 2023-12-01 22:54:42 +08:00
safety.py Add Release Contraibution 2023-07-05 02:22:40 +00:00
scibench.py add evaluation of scibench (#393) 2023-09-22 17:42:08 +08:00
siqa.py [Sync] Add InternLM2 Keyset Evaluation Demo (#807) 2024-01-17 13:48:12 +08:00
squad20.py [Feature] Add Xiezhi SQuAD2.0 ANLI (#101) 2023-08-10 14:04:18 +08:00
storycloze.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
strategyqa.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
summedits.py [Enhancement] Test linting in CI and fix existing linting errors (#69) 2023-07-17 15:59:10 +08:00
summscreen.py Support a batch of datasets. 2023-07-05 01:30:27 +00:00
svamp.py [Feature] Add SVAMP dataset (#604) 2023-11-22 14:54:39 +08:00
tabmwp.py [fFeat] Add an opensource dataset Tabmwp (#505) 2023-11-03 11:15:46 +08:00
taco.py [Fix] Update APPS/TACO (#988) 2024-03-19 20:21:39 +08:00
TheoremQA.py [Sync] Sync Internal (#941) 2024-03-04 14:42:36 +08:00
tnews.py [Sync] update (#517) 2023-10-27 20:31:22 +08:00
triviaqa.py [Sync] Merge branch 'dev' into zfz/update-keyset-demo (#876) 2024-02-05 23:29:10 +08:00
triviaqarc.py Add Release Contraibution 2023-07-05 02:22:40 +00:00
truthfulqa.py [Feat] refine docs and codes for more user guides (#409) 2023-09-18 16:12:13 +08:00
tydiqa.py [Feature] Use dataset in local path (#570) 2023-11-13 13:00:37 +08:00
wic.py Support a batch of datasets. 2023-07-05 01:30:27 +00:00
wikibench.py [Sync] minor test (#683) 2023-12-11 17:42:53 +08:00
winograd.py initial commit 2023-07-04 21:34:55 +08:00
winogrande.py [Sync] Sync Internal (#941) 2024-03-04 14:42:36 +08:00
wnli.py [Feat] implementation for support promptbench (#239) 2023-09-15 15:06:53 +08:00
wsc.py Update configs (#9) 2023-07-06 12:27:41 +08:00
xcopa.py Add Release Contraibution 2023-07-05 02:22:40 +00:00
xiezhi.py [Feature] Add Xiezhi SQuAD2.0 ANLI (#101) 2023-08-10 14:04:18 +08:00
xlsum.py update datasets 2023-07-05 01:45:26 +00:00
xsum.py update datasets 2023-07-05 01:45:26 +00:00