jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
pypi_workflow_dispatch_038
OpenCompass/configs/datasets
History
Alexander Lam 1bd594fc62
[Feature] Added CompassArena-SubjectiveBench with Bradley-Terry Model (#1751) 
* fix lint issues

* updated gitignore

* changed infer_order from random to double for the pairwise_judge.py (not changing for pairwise_bt_judge.py

* added return statement to CompassArenaBradleyTerrySummarizer to return overall score for each judger model
2024-12-16 13:41:28 +08:00
..
adv_glue [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
agieval [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
anli [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
anthropics_evals [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
apps [Sync] Sync with internal codes 2024.06.28 (#1279) 2024-06-28 14:16:34 +08:00
ARC_c [Feature] Dataset prompts update for ARC, BoolQ, Race (#1527) 2024-09-13 10:30:43 +08:00
ARC_e [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
bbh [Feature] Support import configs/models/summarizers from whl (#1376) 2024-08-01 00:42:48 +08:00
calm Calm dataset (#1385) 2024-08-01 10:03:21 +08:00
ceval [Feature] Support import configs/models/summarizers from whl (#1376) 2024-08-01 00:42:48 +08:00
CHARM [Feature] Update CHARM Memeorziation (#1230) 2024-07-26 18:42:30 +08:00
ChemBench [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
CIBench [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
civilcomments [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
clozeTest_maxmin [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
CLUE_afqmc [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
CLUE_C3 [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
CLUE_cmnli [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
CLUE_CMRC [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
CLUE_DRCD [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
CLUE_ocnli [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
cmb [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
cmmlu [Feature] Support import configs/models/summarizers from whl (#1376) 2024-08-01 00:42:48 +08:00
collections [Sync] Sync with internal codes 2024.06.28 (#1279) 2024-06-28 14:16:34 +08:00
commonsenseqa [Bug] Commonsenseqa dataset fix (#1425) 2024-08-16 15:54:07 +08:00
commonsenseqa_cn [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
compassbench_20_v1_1 [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
compassbench_20_v1_1_public [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
compassbench_v1_3 [Fix] Compatible with old versions (#1616) 2024-10-21 10:16:29 +08:00
contamination [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
crowspairs [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
crowspairs_cn [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
cvalues [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
demo [Doc] quick start swap tabs (#1263) 2024-07-05 23:51:42 +08:00
dingo [Feature] Add dingo test (#1529) 2024-09-29 19:24:58 +08:00
drop [Sync] Sync with internal codes 2024.06.28 (#1279) 2024-06-28 14:16:34 +08:00
ds1000 [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
FewCLUE_bustm [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
FewCLUE_chid [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
FewCLUE_cluewsc [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
FewCLUE_csl [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
FewCLUE_eprstmt [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
FewCLUE_ocnli_fc [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
FewCLUE_tnews [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
FinanceIQ [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
flores [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
game24 [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
gaokao_math [Feature] Add GaoKaoMath Dataset for Evaluation & MATH Model Eval Config (#1589) 2024-10-12 19:13:06 +08:00
GaokaoBench [Fix] the automatically download for several datasets (#1652) 2024-11-01 15:57:18 +08:00
GLUE_CoLA [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
GLUE_MRPC [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
GLUE_QQP [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
govrepcrs [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
gpqa [Fix] gpqa_few_shot_ppl prompt bug (#1627) 2024-10-21 16:59:06 +08:00
gsm8k [Feature] Add custom model postprocess function (#1519) 2024-09-18 14:40:51 +08:00
gsm8k_contamination [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
gsm_hard [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
hellaswag [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
humaneval [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
humaneval_cn [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
humaneval_multi [feat] support multipl-e (#846) 2024-02-06 23:30:28 +08:00
humaneval_plus [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
humanevalx [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
hungarian_exam [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
IFEval [Doc] Update running command in README (#1206) 2024-05-30 00:06:39 +08:00
inference_ppl [Feature] Support inference ppl datasets (#1315) 2024-07-22 17:59:30 +08:00
infinitebench [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
iwslt2017 [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
jigsawmultilingual [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
kaoshi [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
lambada [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
lawbench [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
LCBench [Update] Update Skywork/Qwen-QwQ (#1728) 2024-12-05 19:30:43 +08:00
lcsts [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
leval [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
livecodebench [Update] Update configurations (#1704) 2024-11-21 16:51:18 +08:00
llm_compression [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
longbench [Feature] Longbench dataset update 2024-09-06 15:50:12 +08:00
lveval [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
mastermath2024v1 [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
math [Feature] Add GaoKaoMath Dataset for Evaluation & MATH Model Eval Config (#1589) 2024-10-12 19:13:06 +08:00
math401 [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
MathBench [Feature] Update MathBench & Math base model config (#1550) 2024-09-23 14:03:59 +08:00
mbpp [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
mbpp_cn [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
mbpp_plus [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
MedBench [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
mgsm [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
mmlu [Feature] Add custom model postprocess function (#1519) 2024-09-18 14:40:51 +08:00
mmlu_pro [Fix] Compatible with old versions (#1616) 2024-10-21 10:16:29 +08:00
MMLUArabic [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
narrativeqa [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
needlebench [Feature] Needlebench auto-download update (#1480) 2024-09-05 17:22:42 +08:00
NPHardEval [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
nq [Fix] the automatically download for several datasets (#1652) 2024-11-01 15:57:18 +08:00
nq_cn [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
obqa [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
OpenFinData [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
piqa [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
PJExam [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
promptbench [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
py150 [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
qabench [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
qasper [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
qaspercut [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
QuALITY [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
race [Feature] Dataset prompts update for ARC, BoolQ, Race (#1527) 2024-09-13 10:30:43 +08:00
realtoxicprompts [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
rolebench [Feature] Add abbr for rolebench dataset (#1431) 2024-08-20 11:22:48 +08:00
ruler [Update] Add RULER 64k config (#1709) 2024-11-25 19:35:27 +08:00
s3eval [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
safety Align prompt files with their hash (#1) 2023-07-05 18:28:58 +08:00
scibench [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
scicode [Feature] Add SciCode summarizer config (#1514) 2024-09-10 16:06:02 +08:00
siqa [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
squad20 [Feature] Add Xiezhi SQuAD2.0 ANLI (#101) 2023-08-10 14:04:18 +08:00
storycloze [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
strategyqa [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
subjective [Feature] Added CompassArena-SubjectiveBench with Bradley-Terry Model (#1751) 2024-12-16 13:41:28 +08:00
summedits [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
summscreen [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
SuperGLUE_AX_b [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
SuperGLUE_AX_g [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
SuperGLUE_BoolQ [Feature] Dataset prompts update for ARC, BoolQ, Race (#1527) 2024-09-13 10:30:43 +08:00
SuperGLUE_CB [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
SuperGLUE_COPA [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
SuperGLUE_MultiRC [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
SuperGLUE_ReCoRD [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
SuperGLUE_RTE [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
SuperGLUE_WiC [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
SuperGLUE_WSC [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
SVAMP [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
TabMWP [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
taco [Sync] Sync with internal codes 2024.06.28 (#1279) 2024-06-28 14:16:34 +08:00
teval [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
TheoremQA [Doc] Update running command in README (#1206) 2024-05-30 00:06:39 +08:00
triviaqa [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
triviaqarc [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
truthfulqa [Fix] Update SciCode and Gemma model (#1449) 2024-08-23 10:42:27 +08:00
tydiqa [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
wikibench [Fix] the automatically download for several datasets (#1652) 2024-11-01 15:57:18 +08:00
wikitext [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
winograd [Feature] Support import configs/models/summarizers from whl (#1376) 2024-08-01 00:42:48 +08:00
winogrande [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00
XCOPA [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
xiezhi [Format] Add config lints (#892) 2024-05-14 15:35:58 +08:00
XLSum Update configs (#9) 2023-07-06 12:27:41 +08:00
Xsum [Feature] Support ModelScope datasets (#1289) 2024-07-29 13:48:32 +08:00