jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
bc2969dba8
OpenCompass/opencompass/datasets/__init__.py

152 lines
6.7 KiB
Python
Raw Normal View History

[Feat] support adv_glue dataset for adversarial robustness (#205) * [Feat] support adv_glue dataset for adversarial robustness * reorg files * minor fix * minor fix
2023-08-16 18:42:06 +08:00
from .advglue import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .afqmcd import * # noqa: F401, F403
from .agieval import * # noqa: F401, F403
[Datasets] Add datasets CMO&AIME (#1610) * add datasets cmo&aime * delete unused modules * modify prompt * update __init__ * update data load and add README * update data load * update performance * update md5 * remove indents * add indent * fix log for debug mode
2024-10-28 18:08:02 +08:00
from .aime2024 import * # noqa: F401, F403
[Feature] Add Xiezhi SQuAD2.0 ANLI (#101) * add Xiezhi SQuAD2.0 ANLI; update WSC * update * update * update doc string
2023-08-10 14:04:18 +08:00
from .anli import AnliDataset # noqa: F401, F403
[Feat] support antropics evals dataset (#422) * [Feat] support anthropics ai risk dataset * [Feat] support anthropics evals dataset * [Feat] support anthropics evals dataset
2023-09-20 18:36:44 +08:00
from .anthropics_evals import * # noqa: F401, F403
[Sync] update taco (#1030)
2024-04-09 17:50:23 +08:00
from .apps import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .arc import * # noqa: F401, F403
[Update] Update max_out_len for datasets (#1726) * [Update] Update max_out_len for datasets * Update eval_regression_chat_objective_fullbench.py * Update eval_regression_chat.py * Update eval_regression_chat.py * Update oc_score_baseline_fullbench.yaml --------- Co-authored-by: zhulinJulia24 <145004780+zhulinJulia24@users.noreply.github.com>
2024-12-02 11:42:07 +08:00
from .arc_prize_public_evaluation import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .ax import * # noqa: F401, F403
[Feature] BABILong Dataset added (#1684) * update * update * update * update
2024-11-14 15:32:43 +08:00
from .babilong import * # noqa: F401, F403
[Feature] Add support for BBEH dataset (#1925) * bbeh * bbeh * fix_smallbugs_bbeh * removeprint * results --------- Co-authored-by: yufeng zhao <zhaoyufeng@pjlab.org.cn>
2025-03-12 10:53:31 +08:00
from .bbeh import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .bbh import * # noqa: F401, F403
[Update] Update Skywork/Qwen-QwQ (#1728) * Update JuderBench * Support O1-style Prompts * Update Code * Update OpenAI * Update BigCodeBench * Update BigCodeBench * Update BigCodeBench * Update BigCodeBench * Update BigCodeBench * Update
2024-12-05 19:30:43 +08:00
from .bigcodebench import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .boolq import * # noqa: F401, F403
from .bustum import * # noqa: F401, F403
from .c3 import * # noqa: F401, F403
Calm dataset (#1385) * Add CALM Dataset
2024-08-01 10:03:21 +08:00
from .calm import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .cb import * # noqa: F401, F403
from .ceval import * # noqa: F401, F403
support CHARM (https://github.com/opendatalab/CHARM) reasoning tasks (#1190) * support CHARM (https://github.com/opendatalab/CHARM) reasoning tasks * fix lint error * add dataset card for CHARM * minor refactor * add txt --------- Co-authored-by: wujiang <wujiang@pjlab.org.cn> Co-authored-by: Leymore <zfz-960727@163.com>
2024-05-27 13:48:22 +08:00
from .charm import * # noqa: F401, F403
[Feature] Add ChemBench (#1032) * add ChemBench * update results * molbench -> ChemBench --------- Co-authored-by: Leymore <zfz-960727@163.com>
2024-04-12 08:46:26 +08:00
from .chembench import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .chid import * # noqa: F401, F403
Add Chinese SimpleQA config (#1697) * add chinese simpleqa config * add chinese simpleqa config * add chinese simpleqa config * add chinese simpleqa config * Update CsimpleQA * Update CsimpleQA * Update CsimpleQA * Update CsimpleQA * Update CsimpleQA * Update CsimpleQA * pdate Csimpleqa --------- Co-authored-by: 明念 <heyancheng.hyc@taobao.com> Co-authored-by: liushz <qq1791167085@163.com>
2024-12-11 18:03:39 +08:00
from .chinese_simpleqa import * # noqa: F401, F403
[Feat] Support cibench (#538) * [Feat] support cidataset * [Feat] support cidataset * [Feat] support cidataset * [Feat] support cidataset * minor fix * minor fix * minor fix * minor fix * minor fix * minor fix * rename cibench * rename cibench * rename cibench * rename cibench * minor fix * minor fix * minor fix
2023-11-07 19:11:44 +08:00
from .cibench import * # noqa: F401, F403
[Feature] Add circular eval (#610) * refactor default, add circular summarizer * add circular * update impl * update doc * minor update * no more to be added
2023-11-23 16:45:47 +08:00
from .circular import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .civilcomments import * # noqa: F401, F403
[Feature] Add py150 and maxmin (#562) * [feat] add clozeTesst_maxmin dataset * [feat] add py150 datasets * [feat] change __init__.py in opencompass/datasets * [fix] pre-commit check * [fix] rename py150 and masxmin datasets in configs * [feat] add gen.py of py150 and maxmin in configs/datasets
2023-11-09 22:05:25 +08:00
from .clozeTest_maxmin import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .cluewsc import * # noqa: F401, F403
[Dataset] Add CMB (#376) * Add CMB * modify CMB --------- Co-authored-by: wangxidong <xidongw@163.com>
2023-09-12 19:16:41 +08:00
from .cmb import * # noqa: F401, F403
[Feature] Add CMMLU dataset (#91) * add CMMLU * debug cmmlu * add slurm args `qos` * fix format: space before comment * remove unused variable * change the location of `answer is` --------- Co-authored-by: 李浩楠 <lihaonan@lihaonandeMacBook-Air.local> Co-authored-by: 李浩楠 <haonan.li> Co-authored-by: Leymore <zfz-960727@163.com>
2023-07-25 10:14:27 +08:00
from .cmmlu import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .cmnli import * # noqa: F401, F403
[Datasets] Add datasets CMO&AIME (#1610) * add datasets cmo&aime * delete unused modules * modify prompt * update __init__ * update data load and add README * update data load * update performance * update md5 * remove indents * add indent * fix log for debug mode
2024-10-28 18:08:02 +08:00
from .cmo_fib import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .cmrc import * # noqa: F401, F403
from .commonsenseqa import * # noqa: F401, F403
[Feature] Add Chinese version: commonsenseqa, crowspairs and nq (#144) * add Chinese version: csqa crowspairs nq * Update cn_data * Update cn_data * update format --------- Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn> Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-30 15:33:02 +08:00
from .commonsenseqa_cn import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .copa import * # noqa: F401, F403
from .crowspairs import * # noqa: F401, F403
[Feature] Add Chinese version: commonsenseqa, crowspairs and nq (#144) * add Chinese version: csqa crowspairs nq * Update cn_data * Update cn_data * update format --------- Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn> Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-30 15:33:02 +08:00
from .crowspairs_cn import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .csl import * # noqa: F401, F403
[Sync] update configs (#734)
2023-12-25 21:59:16 +08:00
from .custom import * # noqa: F401, F403
[Feat] Support CValues Responsibility dataset (#78) * [Feat] support CValues * minor fix
2023-07-18 18:45:15 +08:00
from .cvalues import * # noqa: F401, F403
[Feature] Add dingo test (#1529) * add qa dingo * update * change name qa to dingo * eval model: llm_base * update path * change name and move path * add eval_dingo * update import * add for pip * add dingo package * change import place * update import place * fix lint fail * isort * double quoted --------- Co-authored-by: sj <shijin@pjlab.org.cn>
2024-09-29 19:24:58 +08:00
from .dingo import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .drcd import * # noqa: F401, F403
from .drop import * # noqa: F401, F403
[Feature] update drop dataset from openai simple eval (#1092) * [Feature] update drop dataset from openai simple eval * update drop template presentation * update --------- Co-authored-by: Leymore <zfz-960727@163.com>
2024-05-06 13:37:08 +08:00
from .drop_simple_eval import * # noqa: F401, F403
[Feat] support ds1000 dataset (#395) * [Feat] support ds1000 datase
2023-09-15 12:50:27 +08:00
from .ds1000 import * # noqa: F401, F403
[Feat] Support cibench (#538) * [Feat] support cidataset * [Feat] support cidataset * [Feat] support cidataset * [Feat] support cidataset * minor fix * minor fix * minor fix * minor fix * minor fix * minor fix * rename cibench * rename cibench * rename cibench * rename cibench * minor fix * minor fix * minor fix
2023-11-07 19:11:44 +08:00
from .ds1000_interpreter import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .eprstmt import * # noqa: F401, F403
[Feature] Add FinanceIQ dataset (#596)
2023-11-16 17:47:57 +08:00
from .FinanceIQ import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .flores import * # noqa: F401, F403
[Feature] Add Tree-of-Thought method (#173) * Add ToT method * Update ToT * Update ToT * Update ToT * Update ToT * Update ToT * Update ToT * Update ToT * Update chain_of_thought.md * Update icl_tot_inferencer.py --------- Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
2023-08-23 12:23:05 +08:00
from .game24 import * # noqa: F401, F403
[Feature] Add GaoKaoMath Dataset for Evaluation & MATH Model Eval Config (#1589) * Add GaoKaoMath Dataset * Add MATH LLM Eval * Update GAOKAO Math Eval Dataset * Update GAOKAO Math Eval Dataset
2024-10-12 19:13:06 +08:00
from .gaokao_math import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .GaokaoBench import * # noqa: F401, F403
[Update] Update Skywork/Qwen-QwQ (#1728) * Update JuderBench * Support O1-style Prompts * Update Code * Update OpenAI * Update BigCodeBench * Update BigCodeBench * Update BigCodeBench * Update BigCodeBench * Update BigCodeBench * Update
2024-12-05 19:30:43 +08:00
from .generic import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .govrepcrs import * # noqa: F401, F403
[Feature] Add GPQA Dataset (#729) * check * message * add * change prompt * change a para nameq * modify name of the file * delete an useless file
2024-01-01 15:54:40 +08:00
from .gpqa import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .gsm8k import * # noqa: F401, F403
[Feature] Add GSM_Hard dataset (#619) * Add SVAMP dataset * Add SVAMP dataset * Add SVAMP dataset * Add gsm_hard dataset * Add gsm_hard dataset * format --------- Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-27 17:40:34 +08:00
from .gsm_hard import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .hellaswag import * # noqa: F401, F403
[Feature] Add HLE (Humanity's Last Exam) dataset (#1902) * Support OlympiadBench Benchmark * Support OlympiadBench Benchmark * Support OlympiadBench Benchmark * update dataset path * Update olmpiadBench * Update olmpiadBench * Update olmpiadBench * Add HLE dataset * Add HLE dataset * Add HLE dataset --------- Co-authored-by: sudanl <sudanl@foxmail.com>
2025-03-04 16:42:37 +08:00
from .hle import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .huggingface import * # noqa: F401, F403
from .humaneval import * # noqa: F401, F403
[Feature] Support ModelScope datasets (#1289) * add ceval, gsm8k modelscope surpport * update race, mmlu, arc, cmmlu, commonsenseqa, humaneval and unittest * update bbh, flores, obqa, siqa, storycloze, summedits, winogrande, xsum datasets * format file * format file * update dataset format * support ms_dataset * udpate dataset for modelscope support * merge myl_dev and update test_ms_dataset * udpate dataset for modelscope support * update readme * update eval_api_zhipu_v2 * remove unused code * add get_data_path function * update readme * remove tydiqa japanese subset * add ceval, gsm8k modelscope surpport * update race, mmlu, arc, cmmlu, commonsenseqa, humaneval and unittest * update bbh, flores, obqa, siqa, storycloze, summedits, winogrande, xsum datasets * format file * format file * update dataset format * support ms_dataset * udpate dataset for modelscope support * merge myl_dev and update test_ms_dataset * update readme * udpate dataset for modelscope support * update eval_api_zhipu_v2 * remove unused code * add get_data_path function * remove tydiqa japanese subset * update util * remove .DS_Store * fix md format * move util into package * update docs/get_started.md * restore eval_api_zhipu_v2.py, add environment setting * Update dataset * Update * Update * Update * Update --------- Co-authored-by: Yun lin <yunlin@U-Q9X2K4QV-1904.local> Co-authored-by: Yunnglin <mao.looper@qq.com> Co-authored-by: Yun lin <yunlin@laptop.local> Co-authored-by: Yunnglin <maoyl@smail.nju.edu.cn> Co-authored-by: zhangsongyang <zhangsongyang@pjlab.org.cn>
2024-07-29 13:48:32 +08:00
from .humaneval_multi import * # noqa: F401, F403
[Feat] Add codegeex2 and Humanevalx (#210) * add codegeex2 * add humanevalx dataset * add evaluator * update evaluator * update configs * update clean code * update configs * fix lint * remove sleep * fix lint * update docs * fix lint
2023-08-17 11:03:16 +08:00
from .humanevalx import * # noqa: F401, F403
[Sync] Sync with internal codes 2023.01.08 (#777)
2024-01-08 22:07:24 +08:00
from .hungarian_math import * # noqa: F401, F403
[Fix] Fix IFEval (#906) * fix ifeval * fix ifeval * fix ifeval * fix ifeval
2024-02-22 16:51:34 +08:00
from .IFEval.ifeval import IFEvalDataset, IFEvaluator # noqa: F401, F403
[Feature] Support inference ppl datasets (#1315) * commit inference ppl datasets * revised format * revise * revise * revise * revise * revise * revise
2024-07-22 17:59:30 +08:00
from .inference_ppl import InferencePPLDataset # noqa: F401, F403
[Feature] Add InfiniteBench (#739) * add InfiniteBench * add InfiniteBench --------- Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>
2023-12-26 15:36:27 +08:00
from .infinitebench import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .iwslt2017 import * # noqa: F401, F403
from .jigsawmultilingual import * # noqa: F401, F403
[Sync] Sync with internal codes 2023.01.08 (#777)
2024-01-08 22:07:24 +08:00
from .jsonl import JsonlDataset # noqa: F401, F403
[Feature] Add kaoshi dataset (#392) * Add ToT method * Update ToT * Update ToT * Update ToT * Update ToT * Update ToT * Add Koashi * Update Kaoshi * Update Kaoshi * Update kaoshi * Update kaoshi * Update Kaoshi * Update Kaoshi * Update Kaoshi * Update Kaoshi * update Kaoshi * update * update * fix --------- Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>
2023-09-22 18:46:33 +08:00
from .kaoshi import KaoshiDataset, KaoshiEvaluator # noqa: F401, F403
[Feature] Add Korbench dataset (#1713) * first version for korbench * first stage for korbench * korbench_1 * korbench_1 * korbench_1 * korbench_1 * korbench_1_revised * korbench_combined_1 * korbench_combined_1 * kor_combined * kor_combined * update --------- Co-authored-by: MaiziXiao <xxllcc1993@gmail.com>
2024-11-25 20:11:27 +08:00
from .korbench import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .lambada import * # noqa: F401, F403
[Feature] Add lawbench (#460) * add lawbench * update requirements * update
2023-10-13 19:51:36 +08:00
from .lawbench import * # noqa: F401, F403
[Feature] Update pip install (#1324) * [Feature] Update pip install * Update Configuration * Update * Update * Update * Update Internal Config * Update collect env
2024-07-29 18:32:50 +08:00
from .LCBench import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .lcsts import * # noqa: F401, F403
[Dataset] Refactor LEval
2023-08-25 11:46:23 +08:00
from .leval import * # noqa: F401, F403
[Feature] Support LiveCodeBench (#1617) * Update * Update LCB * Update * Update * Update * Update * Update
2024-10-21 20:50:39 +08:00
from .livecodebench import * # noqa: F401, F403
[Update] Update Skywork/Qwen-QwQ (#1728) * Update JuderBench * Support O1-style Prompts * Update Code * Update OpenAI * Update BigCodeBench * Update BigCodeBench * Update BigCodeBench * Update BigCodeBench * Update BigCodeBench * Update
2024-12-05 19:30:43 +08:00
from .livemathbench import * # noqa: F401, F403
from .livereasonbench import * # noqa: F401, F403
[Feature] Add LiveStemBench Dataset (#1794) * [Fix] Fix vllm max_seq_len parameter transfer * [Fix] Fix vllm max_seq_len parameter transfer * Add livestembench dataset * Add livestembench dataset * Add livestembench dataset * Update livestembench_gen_3e3c50.py * Update eval_livestembench.py * Update eval_livestembench.py
2024-12-31 15:17:39 +08:00
from .livestembench import * # noqa: F401, F403
[Feature] Adding support for LLM Compression Evaluation (#1108) * fixed formatting based on pre-commit tests * fixed typo in comments; reduced the number of models in the eval config * fixed a bug in LLMCompressionDataset, where setting samples=None would result in passing test[:None] to load_dataset * removed unnecessary variable in _format_table_pivot; changed lark_reporter message to English
2024-04-30 10:51:01 +08:00
from .llm_compression import LLMCompressionDataset # noqa: F401, F403
[Dataset] LongBench (#236) Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>
2023-08-21 14:15:20 +08:00
from .longbench import * # noqa: F401, F403
[Feature] Add Longbenchv2 support (#1801) * Create eval_longbenchv2.py * Create longbenchv2_gen.py * Update __init__.py * Create longbenchv2.py * Update datasets_info.py * update * update * update * update * update * update --------- Co-authored-by: abrohamLee <146956824+abrohamLee@users.noreply.github.com>
2025-01-03 12:04:29 +08:00
from .longbenchv2 import * # noqa: F401, F403
[Feature] add lveval benchmark (#914) * add lveval benchmark * add LVEval readme file * update LVEval readme file * Update configs/eval_bluelm_32k_lveval.py * Update configs/eval_llama2_7b_lveval.py --------- Co-authored-by: yuantao <yuantao@infini-ai.com> Co-authored-by: Mo Li <82895469+DseidLi@users.noreply.github.com>
2024-03-04 11:22:03 +08:00
from .lveval import * # noqa: F401, F403
[Feature] Add new dataset mastermath2024v1 (#744) * add new dataset mastermath2024v1 * change it to simplified chinese prompt * change file name
2024-01-01 15:53:24 +08:00
from .mastermath2024v1 import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .math import * # noqa: F401, F403
[Sync] Sync with internal codes 2023.01.08 (#777)
2024-01-08 22:07:24 +08:00
from .math401 import * # noqa: F401, F403
[Sync] Updata dataset cfg for internMath (#837) Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
2024-01-24 16:30:32 +08:00
from .math_intern import * # noqa: F401, F403
[Feature] Add mathbench dataset and circular evaluator (#408) * add_mathbench * update mathbench * support non circular eval dataset --------- Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn> Co-authored-by: yingfhu <yingfhu@gmail.com>
2023-10-18 17:08:31 +08:00
from .mathbench import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .mbpp import * # noqa: F401, F403
[Feature] Add medbench (#678) * update medbench * medbench update * format medbench * format --------- Co-authored-by: 施晓明 <PJLAB\shixiaoming@pjnl104220118l.pjlab.org> Co-authored-by: Leymore <zfz-960727@163.com>
2023-12-09 16:05:46 +08:00
from .medbench import * # noqa: F401, F403
add mgsm datasets (#1081) * add mgsm datasets * fix lint * fix lint * update mgsm * update mgsm * ease code spell * update * update * update --------- Co-authored-by: Leymore <zfz-960727@163.com>
2024-05-06 15:29:34 +08:00
from .mgsm import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .mmlu import * # noqa: F401, F403
[Feature] Support MMLU-CF Benchmark (#1775) * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * Update mmlu-cf * Update mmlu-cf * Update mmlu-cf * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * [Feature] Support MMLU-CF Benchmark * Remove outside configs --------- Co-authored-by: liushz <qq1791167085@163.com>
2025-01-09 14:11:20 +08:00
from .mmlu_cf import * # noqa: F401, F403
[Sync] Sync with internal codes 2024.06.28 (#1279)
2024-06-28 14:16:34 +08:00
from .mmlu_pro import * # noqa: F401, F403
fix links (#1120)
2024-05-08 15:13:18 +08:00
from .MMLUArabic import * # noqa: F401, F403
[Feature] Support MMMLU & MMMLU-lite Benchmark (#1565) * rm folder * modify format according to reviewer * modify format according to reviewer * modify format according to reviewer * add some files requirement * fix some bug * fix bug * change load type * Update MMMLU Dataset * Update MMMLU Dataset * Add MMMLU-Lite Dataset * update MMMMLU datast * update MMMMLU datast * update MMMMLU datast --------- Co-authored-by: BobTsang <BobTsang1995@gmail.com> Co-authored-by: liushz <qq1791167085@163.com>
2024-10-17 19:09:34 +08:00
from .mmmlu import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .multirc import * # noqa: F401, F403
[Feature] MuSR Datset Evaluation (#1689) * MuSR Datset Evaluation * MuSR Datset Evaluation Add an assertion and a Readme.md
2024-11-14 20:42:12 +08:00
from .musr import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .narrativeqa import * # noqa: F401, F403
from .natural_question import * # noqa: F401, F403
[Feature] Add Chinese version: commonsenseqa, crowspairs and nq (#144) * add Chinese version: csqa crowspairs nq * Update cn_data * Update cn_data * update format --------- Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn> Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-30 15:33:02 +08:00
from .natural_question_cn import * # noqa: F401, F403
Support NPHardEval (#835) * support NPHardEval * add .md file and fix minor bugs * refactor and minor fix --------- Co-authored-by: Leymore <zfz-960727@163.com>
2024-02-05 15:52:28 +08:00
from .NPHardEval import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .obqa import * # noqa: F401, F403
[Feature] Support OlympiadBench Benchmark (#1841) * Support OlympiadBench Benchmark * Support OlympiadBench Benchmark * Support OlympiadBench Benchmark * update dataset path * Update olmpiadBench * Update olmpiadBench * Update olmpiadBench --------- Co-authored-by: liushz <qq1791167085@163.com>
2025-01-24 10:00:01 +08:00
from .OlympiadBench import * # noqa: F401, F403
[Feature] Support OpenFinData (#896) * [Feature] Support OpenFinData * add README for OpenFinData * update README
2024-02-29 12:55:07 +08:00
from .OpenFinData import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .piqa import * # noqa: F401, F403
[Feature] Add py150 and maxmin (#562) * [feat] add clozeTesst_maxmin dataset * [feat] add py150 datasets * [feat] change __init__.py in opencompass/datasets * [fix] pre-commit check * [fix] rename py150 and masxmin datasets in configs * [feat] add gen.py of py150 and maxmin in configs/datasets
2023-11-09 22:05:25 +08:00
from .py150 import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .qasper import * # noqa: F401, F403
from .qaspercut import * # noqa: F401, F403
[Feature] Add the implement of QuALITY datasets (#976) #976
2024-03-15 21:22:38 +08:00
from .QuALITY import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .race import * # noqa: F401, F403
from .realtoxicprompts import * # noqa: F401, F403
[Feature] Add ReasonBench(Internal) dataset (#577) * [Feature] Add reasonbench dataset * add configs for supporting generative inference & merge datasets in the same category * modify config filename to prompt version * fix codes to meet pre-commit requirements * lint the code to meet pre-commit requirements * Align Load_data Sourcecode Briefly * fix bugs * reduce code redundancy
2023-12-20 17:57:42 +08:00
from .reasonbench import ReasonBenchDataset # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .record import * # noqa: F401, F403
[Feature] Add Ruler datasets (#1310) * [Feature] Add Ruler datasets * pre-commit fixed * Add model specific tokenizer to dataset * pre-commit modified * remove unused import * fix linting * add trust_remote to tokenizer load * lint fix * comments resolved * fix lint * Add readme * Fix lint * ruler refactorize * fix lint * lint fix * updated * lint fix * fix wonderwords import issue * prompt modified * update * readme updated * update * ruler dataset added * Update --------- Co-authored-by: tonysy <sy.zhangbuaa@gmail.com>
2024-08-20 11:40:11 +08:00
from .ruler import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .safety import * # noqa: F401, F403
add evaluation of scibench (#393) * add evaluation of scibench * add evaluation of scibench * update scibench * remove scibench evaluator --------- Co-authored-by: Leymore <zfz-960727@163.com>
2023-09-22 17:42:08 +08:00
from .scibench import ScibenchDataset, scibench_postprocess # noqa: F401, F403
[Feature] Add support for SciCode (#1417) * add SciCode * add SciCode * add SciCode * add SciCode * add SciCode * add SciCode * add SciCode * add SciCode w/ bg * add scicode * Update README.md * Update README.md * Delete configs/eval_SciCode.py * rename * 1 * rename * Update README.md * Update scicode.py * Update scicode.py * fix some bugs * Update * Update --------- Co-authored-by: root <HariSeldon0> Co-authored-by: tonysy <sy.zhangbuaa@gmail.com>
2024-08-22 13:42:25 +08:00
from .scicode import * # noqa: F401, F403
[Feature] Add Openai Simpleqa dataset (#1720) * Add Openai SimpleQA dataset * Add Openai SimpleQA dataset * Add Openai SimpleQA dataset * Update eval_simpleqa.py --------- Co-authored-by: Linchen Xiao <xxllcc1993@gmail.com>
2024-11-28 19:16:07 +08:00
from .simpleqa import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .siqa import * # noqa: F401, F403
[Feature] Add Xiezhi SQuAD2.0 ANLI (#101) * add Xiezhi SQuAD2.0 ANLI; update WSC * update * update * update doc string
2023-08-10 14:04:18 +08:00
from .squad20 import SQuAD20Dataset, SQuAD20Evaluator # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .storycloze import * # noqa: F401, F403
from .strategyqa import * # noqa: F401, F403
reorganize subject files (#801)
2024-01-16 18:03:11 +08:00
from .subjective import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .summedits import * # noqa: F401, F403
from .summscreen import * # noqa: F401, F403
[Feature] Support SuperGPQA (#1924) * support supergpqa * remove unnecessary code * remove unnecessary code * Add Readme * Add Readme * fix lint * fix lint * update * update --------- Co-authored-by: mkj3085003 <mkj3085003@gmail.com> Co-authored-by: MaiziXiao <xxllcc1993@gmail.com>
2025-03-11 19:32:08 +08:00
from .supergpqa import * # noqa: F401, F403
[Feature] Add SVAMP dataset (#604) * Add SVAMP dataset * Add SVAMP dataset * Add SVAMP dataset
2023-11-22 14:54:39 +08:00
from .svamp import * # noqa: F401, F403
[fFeat] Add an opensource dataset Tabmwp (#505) * TabMWP * TabMWP * fixed * fixed * fixed * done * done * done --------- Co-authored-by: caomaosong <caomaosong@pjlab.org.cn>
2023-11-03 11:15:46 +08:00
from .tabmwp import * # noqa: F401, F403
[Sync] update taco (#1030)
2024-04-09 17:50:23 +08:00
from .taco import * # noqa: F401, F403
[Sync] Merge branch 'dev' into zfz/update-keyset-demo (#876)
2024-02-05 23:29:10 +08:00
from .teval import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .TheoremQA import * # noqa: F401, F403
from .tnews import * # noqa: F401, F403
from .triviaqa import * # noqa: F401, F403
from .triviaqarc import * # noqa: F401, F403
from .truthfulqa import * # noqa: F401, F403
[Feature] Add tydiqa-goldp (#75) Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
2023-07-18 14:54:35 +08:00
from .tydiqa import * # noqa: F401, F403
[Dataset] Refactor LEval
2023-08-25 11:46:23 +08:00
from .wic import * # noqa: F401, F403
[Feature] Add wikibench dataset (#655) * Add WikiBench * Add WikiBench * format --------- Co-authored-by: Leymore <zfz-960727@163.com>
2023-12-01 14:56:54 +08:00
from .wikibench import * # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .winograd import * # noqa: F401, F403
from .winogrande import * # noqa: F401, F403
[Feat] implementation for support promptbench (#239) * [Feat] support adv_glue dataset for adversarial robustness * reorg files * minor fix * minor fix * support prompt bench demo * minor fix * minor fix * minor fix * minor fix * minor fix * minor fix * minor fix * minor fix
2023-09-15 15:06:53 +08:00
from .wnli import wnliDataset # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .wsc import * # noqa: F401, F403
from .xcopa import * # noqa: F401, F403
[Feature] Add Xiezhi SQuAD2.0 ANLI (#101) * add Xiezhi SQuAD2.0 ANLI; update WSC * update * update * update doc string
2023-08-10 14:04:18 +08:00
from .xiezhi import XiezhiDataset, XiezhiRetriever # noqa: F401, F403
initial commit
2023-07-04 21:34:55 +08:00
from .xlsum import * # noqa: F401, F403
from .xsum import * # noqa: F401, F403
Reference in New Issue Copy Permalink