jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
8142f399a8
OpenCompass/opencompass/datasets/__init__.py

112 lines
4.9 KiB
Python
Raw Normal View History

[Feat] support adv_glue dataset for adversarial robustness (#205) * [Feat] support adv_glue dataset for adversarial robustness * reorg files * minor fix * minor fix
2023-08-16 18:42:06 +08:00
from .advglue import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .afqmcd import * # noqa: F401, F403
from .agieval import * # noqa: F401, F403
[Feature] Add Xiezhi SQuAD2.0 ANLI (#101) * add Xiezhi SQuAD2.0 ANLI; update WSC * update * update * update doc string
2023-08-10 14:04:18 +08:00
from .anli import AnliDataset # noqa: F401, F403
[Feat] support antropics evals dataset (#422) * [Feat] support anthropics ai risk dataset * [Feat] support anthropics evals dataset * [Feat] support anthropics evals dataset
2023-09-20 18:36:44 +08:00
from .anthropics_evals import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .arc import * # noqa: F401, F403
from .ax import * # noqa: F401, F403
from .bbh import * # noqa: F401, F403
from .boolq import * # noqa: F401, F403
from .bustum import * # noqa: F401, F403
from .c3 import * # noqa: F401, F403
from .cb import * # noqa: F401, F403
from .ceval import * # noqa: F401, F403
from .chid import * # noqa: F401, F403
[Feat] Support cibench (#538) * [Feat] support cidataset * [Feat] support cidataset * [Feat] support cidataset * [Feat] support cidataset * minor fix * minor fix * minor fix * minor fix * minor fix * minor fix * rename cibench * rename cibench * rename cibench * rename cibench * minor fix * minor fix * minor fix
2023-11-07 19:11:44 +08:00
from .cibench import * # noqa: F401, F403
[Feature] Add circular eval (#610) * refactor default, add circular summarizer * add circular * update impl * update doc * minor update * no more to be added
2023-11-23 16:45:47 +08:00
from .circular import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .civilcomments import * # noqa: F401, F403
[Feature] Add py150 and maxmin (#562) * [feat] add clozeTesst_maxmin dataset * [feat] add py150 datasets * [feat] change __init__.py in opencompass/datasets * [fix] pre-commit check * [fix] rename py150 and masxmin datasets in configs * [feat] add gen.py of py150 and maxmin in configs/datasets
2023-11-09 22:05:25 +08:00
from .clozeTest_maxmin import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .cluewsc import * # noqa: F401, F403
[Dataset] Add CMB (#376) * Add CMB * modify CMB --------- Co-authored-by: wangxidong <xidongw@163.com>
2023-09-12 19:16:41 +08:00
from .cmb import * # noqa: F401, F403
[Feature] Add CMMLU dataset (#91) * add CMMLU * debug cmmlu * add slurm args `qos` * fix format: space before comment * remove unused variable * change the location of `answer is` --------- Co-authored-by: 李浩楠 <lihaonan@lihaonandeMacBook-Air.local> Co-authored-by: 李浩楠 <haonan.li> Co-authored-by: Leymore <zfz-960727@163.com>
2023-07-25 10:14:27 +08:00
from .cmmlu import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .cmnli import * # noqa: F401, F403
from .cmrc import * # noqa: F401, F403
from .commonsenseqa import * # noqa: F401, F403
[Feature] Add Chinese version: commonsenseqa, crowspairs and nq (#144) * add Chinese version: csqa crowspairs nq * Update cn_data * Update cn_data * update format --------- Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn> Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-30 15:33:02 +08:00
from .commonsenseqa_cn import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .copa import * # noqa: F401, F403
from .crowspairs import * # noqa: F401, F403
[Feature] Add Chinese version: commonsenseqa, crowspairs and nq (#144) * add Chinese version: csqa crowspairs nq * Update cn_data * Update cn_data * update format --------- Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn> Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-30 15:33:02 +08:00
from .crowspairs_cn import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .csl import * # noqa: F401, F403
[Sync] update configs (#734)
2023-12-25 21:59:16 +08:00
from .custom import * # noqa: F401, F403
[Feat] Support CValues Responsibility dataset (#78) * [Feat] support CValues * minor fix
2023-07-18 18:45:15 +08:00
from .cvalues import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .drcd import * # noqa: F401, F403
from .drop import * # noqa: F401, F403
[Feat] support ds1000 dataset (#395) * [Feat] support ds1000 datase
2023-09-15 12:50:27 +08:00
from .ds1000 import * # noqa: F401, F403
[Feat] Support cibench (#538) * [Feat] support cidataset * [Feat] support cidataset * [Feat] support cidataset * [Feat] support cidataset * minor fix * minor fix * minor fix * minor fix * minor fix * minor fix * rename cibench * rename cibench * rename cibench * rename cibench * minor fix * minor fix * minor fix
2023-11-07 19:11:44 +08:00
from .ds1000_interpreter import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .eprstmt import * # noqa: F401, F403
[Feature] Add FinanceIQ dataset (#596)
2023-11-16 17:47:57 +08:00
from .FinanceIQ import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .flores import * # noqa: F401, F403
[Feature] Add Tree-of-Thought method (#173) * Add ToT method * Update ToT * Update ToT * Update ToT * Update ToT * Update ToT * Update ToT * Update ToT * Update chain_of_thought.md * Update icl_tot_inferencer.py --------- Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
2023-08-23 12:23:05 +08:00
from .game24 import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .GaokaoBench import * # noqa: F401, F403
from .govrepcrs import * # noqa: F401, F403
[Feature] Add GPQA Dataset (#729) * check * message * add * change prompt * change a para nameq * modify name of the file * delete an useless file
2024-01-01 15:54:40 +08:00
from .gpqa import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .gsm8k import * # noqa: F401, F403
[Feature] Add GSM_Hard dataset (#619) * Add SVAMP dataset * Add SVAMP dataset * Add SVAMP dataset * Add gsm_hard dataset * Add gsm_hard dataset * format --------- Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-27 17:40:34 +08:00
from .gsm_hard import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .hellaswag import * # noqa: F401, F403
from .huggingface import * # noqa: F401, F403
from .humaneval import * # noqa: F401, F403
[Feat] Add codegeex2 and Humanevalx (#210) * add codegeex2 * add humanevalx dataset * add evaluator * update evaluator * update configs * update clean code * update configs * fix lint * remove sleep * fix lint * update docs * fix lint
2023-08-17 11:03:16 +08:00
from .humanevalx import * # noqa: F401, F403
[Sync] Sync with internal codes 2023.01.08 (#777)
2024-01-08 22:07:24 +08:00
from .hungarian_math import * # noqa: F401, F403
[Fix] Fix IFEval (#906) * fix ifeval * fix ifeval * fix ifeval * fix ifeval
2024-02-22 16:51:34 +08:00
from .IFEval.ifeval import IFEvalDataset, IFEvaluator # noqa: F401, F403
[Feature] Add InfiniteBench (#739) * add InfiniteBench * add InfiniteBench --------- Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>
2023-12-26 15:36:27 +08:00
from .infinitebench import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .iwslt2017 import * # noqa: F401, F403
from .jigsawmultilingual import * # noqa: F401, F403
[Sync] Sync with internal codes 2023.01.08 (#777)
2024-01-08 22:07:24 +08:00
from .jsonl import JsonlDataset # noqa: F401, F403
[Feature] Add kaoshi dataset (#392) * Add ToT method * Update ToT * Update ToT * Update ToT * Update ToT * Update ToT * Add Koashi * Update Kaoshi * Update Kaoshi * Update kaoshi * Update kaoshi * Update Kaoshi * Update Kaoshi * Update Kaoshi * Update Kaoshi * update Kaoshi * update * update * fix --------- Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>
2023-09-22 18:46:33 +08:00
from .kaoshi import KaoshiDataset, KaoshiEvaluator # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .lambada import * # noqa: F401, F403
[Feature] Add lawbench (#460) * add lawbench * update requirements * update
2023-10-13 19:51:36 +08:00
from .lawbench import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .lcsts import * # noqa: F401, F403
[Dataset] Refactor LEval
2023-08-25 11:46:23 +08:00
from .leval import * # noqa: F401, F403
[Dataset] LongBench (#236) Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>
2023-08-21 14:15:20 +08:00
from .longbench import * # noqa: F401, F403
[Feature] Add new dataset mastermath2024v1 (#744) * add new dataset mastermath2024v1 * change it to simplified chinese prompt * change file name
2024-01-01 15:53:24 +08:00
from .mastermath2024v1 import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .math import * # noqa: F401, F403
[Sync] Sync with internal codes 2023.01.08 (#777)
2024-01-08 22:07:24 +08:00
from .math401 import * # noqa: F401, F403
[Sync] Updata dataset cfg for internMath (#837) Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
2024-01-24 16:30:32 +08:00
from .math_intern import * # noqa: F401, F403
[Feature] Add mathbench dataset and circular evaluator (#408) * add_mathbench * update mathbench * support non circular eval dataset --------- Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn> Co-authored-by: yingfhu <yingfhu@gmail.com>
2023-10-18 17:08:31 +08:00
from .mathbench import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .mbpp import * # noqa: F401, F403
[Feature] Add medbench (#678) * update medbench * medbench update * format medbench * format --------- Co-authored-by: 施晓明 <PJLAB\shixiaoming@pjnl104220118l.pjlab.org> Co-authored-by: Leymore <zfz-960727@163.com>
2023-12-09 16:05:46 +08:00
from .medbench import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .mmlu import * # noqa: F401, F403
from .multirc import * # noqa: F401, F403
from .narrativeqa import * # noqa: F401, F403
from .natural_question import * # noqa: F401, F403
[Feature] Add Chinese version: commonsenseqa, crowspairs and nq (#144) * add Chinese version: csqa crowspairs nq * Update cn_data * Update cn_data * update format --------- Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn> Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-30 15:33:02 +08:00
from .natural_question_cn import * # noqa: F401, F403
Support NPHardEval (#835) * support NPHardEval * add .md file and fix minor bugs * refactor and minor fix --------- Co-authored-by: Leymore <zfz-960727@163.com>
2024-02-05 15:52:28 +08:00
from .NPHardEval import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .obqa import * # noqa: F401, F403
[Feature] Support OpenFinData (#896) * [Feature] Support OpenFinData * add README for OpenFinData * update README
2024-02-29 12:55:07 +08:00
from .OpenFinData import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .piqa import * # noqa: F401, F403
[Feature] Add py150 and maxmin (#562) * [feat] add clozeTesst_maxmin dataset * [feat] add py150 datasets * [feat] change __init__.py in opencompass/datasets * [fix] pre-commit check * [fix] rename py150 and masxmin datasets in configs * [feat] add gen.py of py150 and maxmin in configs/datasets
2023-11-09 22:05:25 +08:00
from .py150 import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .qasper import * # noqa: F401, F403
from .qaspercut import * # noqa: F401, F403
from .race import * # noqa: F401, F403
from .realtoxicprompts import * # noqa: F401, F403
[Feature] Add ReasonBench(Internal) dataset (#577) * [Feature] Add reasonbench dataset * add configs for supporting generative inference & merge datasets in the same category * modify config filename to prompt version * fix codes to meet pre-commit requirements * lint the code to meet pre-commit requirements * Align Load_data Sourcecode Briefly * fix bugs * reduce code redundancy
2023-12-20 17:57:42 +08:00
from .reasonbench import ReasonBenchDataset # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .record import * # noqa: F401, F403
from .safety import * # noqa: F401, F403
add evaluation of scibench (#393) * add evaluation of scibench * add evaluation of scibench * update scibench * remove scibench evaluator --------- Co-authored-by: Leymore <zfz-960727@163.com>
2023-09-22 17:42:08 +08:00
from .scibench import ScibenchDataset, scibench_postprocess # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .siqa import * # noqa: F401, F403
[Feature] Add Xiezhi SQuAD2.0 ANLI (#101) * add Xiezhi SQuAD2.0 ANLI; update WSC * update * update * update doc string
2023-08-10 14:04:18 +08:00
from .squad20 import SQuAD20Dataset, SQuAD20Evaluator # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .storycloze import * # noqa: F401, F403
from .strategyqa import * # noqa: F401, F403
reorganize subject files (#801)
2024-01-16 18:03:11 +08:00
from .subjective import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .summedits import * # noqa: F401, F403
from .summscreen import * # noqa: F401, F403
[Feature] Add SVAMP dataset (#604) * Add SVAMP dataset * Add SVAMP dataset * Add SVAMP dataset
2023-11-22 14:54:39 +08:00
from .svamp import * # noqa: F401, F403
[fFeat] Add an opensource dataset Tabmwp (#505) * TabMWP * TabMWP * fixed * fixed * fixed * done * done * done --------- Co-authored-by: caomaosong <caomaosong@pjlab.org.cn>
2023-11-03 11:15:46 +08:00
from .tabmwp import * # noqa: F401, F403
[Sync] Merge branch 'dev' into zfz/update-keyset-demo (#876)
2024-02-05 23:29:10 +08:00
from .teval import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .TheoremQA import * # noqa: F401, F403
from .tnews import * # noqa: F401, F403
from .triviaqa import * # noqa: F401, F403
from .triviaqarc import * # noqa: F401, F403
from .truthfulqa import * # noqa: F401, F403
[Feature] Add tydiqa-goldp (#75) Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
2023-07-18 14:54:35 +08:00
from .tydiqa import * # noqa: F401, F403
[Dataset] Refactor LEval
2023-08-25 11:46:23 +08:00
from .wic import * # noqa: F401, F403
[Feature] Add wikibench dataset (#655) * Add WikiBench * Add WikiBench * format --------- Co-authored-by: Leymore <zfz-960727@163.com>
2023-12-01 14:56:54 +08:00
from .wikibench import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .winograd import * # noqa: F401, F403
from .winogrande import * # noqa: F401, F403
[Feat] implementation for support promptbench (#239) * [Feat] support adv_glue dataset for adversarial robustness * reorg files * minor fix * minor fix * support prompt bench demo * minor fix * minor fix * minor fix * minor fix * minor fix * minor fix * minor fix * minor fix
2023-09-15 15:06:53 +08:00
from .wnli import wnliDataset # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .wsc import * # noqa: F401, F403
from .xcopa import * # noqa: F401, F403
[Feature] Add Xiezhi SQuAD2.0 ANLI (#101) * add Xiezhi SQuAD2.0 ANLI; update WSC * update * update * update doc string
2023-08-10 14:04:18 +08:00
from .xiezhi import XiezhiDataset, XiezhiRetriever # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .xlsum import * # noqa: F401, F403
from .xsum import * # noqa: F401, F403
Reference in New Issue Copy Permalink