jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
810 Commits 11 Branches 35 Tags 66 MiB
pypi_workflow_dispatch_038
Commit Graph

264 Commits

Author SHA1 Message Date
Fengzhe Zhou
d6aaac22e7
[Feature] Update cmb (#571) 2023-11-13 00:09:05 +08:00
Kevin Wang
7f77e8dae5
[Docs] fix dataset name error (#533) 2023-11-10 18:54:20 +08:00
Hubert
95e0da0173
[Docs] add humanevalx dataset link in config (#559) 
* [Docs] add humanevalx dataset link in config

* [Docs] add humanevalx dataset link in config

* minor fix
 2023-11-10 18:18:58 +08:00
jingmingzhuo
b3cbef3226
[Feature] Add py150 and maxmin (#562) 
* [feat] add clozeTesst_maxmin dataset

* [feat] add py150 datasets

* [feat] change __init__.py in opencompass/datasets

* [fix] pre-commit check

* [fix] rename py150 and masxmin datasets in configs

* [feat] add gen.py of py150 and maxmin in configs/datasets
 2023-11-09 22:05:25 +08:00
Hubert
bb2ecf416e
[Feat] Support cibench (#538) 
* [Feat] support cidataset

* [Feat] support cidataset

* [Feat] support cidataset

* [Feat] support cidataset

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* rename cibench

* rename cibench

* rename cibench

* rename cibench

* minor fix

* minor fix

* minor fix
 2023-11-07 19:11:44 +08:00
liushz
214a34f0b8
【Feature】Update Mathbench dataset prompt and fix small errors (#546) 
* Update mathbench

* Update mathbench

* Update mathbench
 2023-11-06 21:58:31 +08:00
bittersweet1999
f25a980043
[fFeat] Add an opensource dataset Tabmwp (#505) 
* TabMWP

* TabMWP

* fixed

* fixed

* fixed

* done

* done

* done

---------

Co-authored-by: caomaosong <caomaosong@pjlab.org.cn>
 2023-11-03 11:15:46 +08:00
Qing
229a65f305
[Fix] Fix typo in WSC prompt (#520) 
Co-authored-by: wq.chu <wq.chu@tianrang-inc.com>
 2023-10-30 12:16:26 +08:00
Fengzhe Zhou
dbb20b8270
[Sync] update (#517) 2023-10-27 20:31:22 +08:00
Wei Jueqi
b62842335d
[Doc] Update Subjective docs (#510) 
* rename

* add en subdoc

* fix name

* fix writing

* update

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-10-27 16:27:24 +08:00
Hubert
b3f5d9e421
[Feat] support math/gms8k agent config (#494) 
* support math agent

* support gsm8k agent

* support gsm8k agent

* minor fix

* minor fix

* minor fix

* Update configs/eval_codeagent.py
 2023-10-25 23:05:15 +08:00
liushz
2737249f31
[Feature] Add mathbench dataset and circular evaluator (#408) 
* add_mathbench

* update mathbench

* support non circular eval dataset

---------

Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
Co-authored-by: yingfhu <yingfhu@gmail.com>
 2023-10-18 04:08:31 -05:00
Leymore
861942ab1b
[Feature] Add lawbench (#460) 
* add lawbench

* update requirements

* update
 2023-10-13 06:51:36 -05:00
Leymore
fbf5089c40
[Sync] update github token (#475) 2023-10-13 06:50:54 -05:00
Leymore
d7ff933a73
[Fix] Use jieba rouge in lcsts (#459) 
* use jieba rouge in lcsts

* use rouge_chinese
 2023-10-09 10:10:33 +08:00
Tong Gao
119bfd1569
[Refactor] Move fix_id_list to Retriever (#442) 
* [Refactor] Move fix_id_list to Retriever

* update

* move to base

* fix
 2023-10-07 12:53:41 +08:00
philipwangOvO
3bb3d330eb
[Sync] Update LongEval (#443) 2023-09-27 16:32:40 +08:00
Kevin Wang
dc1b82c346
[SIG] add GLUE_MRPC dataset (#440) 2023-09-27 11:44:54 +08:00
Kevin Wang
14fdecfecc
[Dataset] add GLUE QQP dataset (#438) 2023-09-27 11:36:43 +08:00
Kevin Wang
d8354fe5d8
[SIG] add GLUE_CoLA dataset (#406) 
* [Dataset] add GLUE_CoLA dataset

* [update] use HFDataset to load glue/cola dataset

* update

---------

Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>
 2023-09-27 11:30:44 +08:00
Kevin Wang
012546666b
[SIG] add WikiText-2&103 (#397) 
* fix conflict

* add eval_cfg
 2023-09-26 14:31:15 +08:00
liushz
c5224c2a91
[Feature] Add kaoshi dataset (#392) 
* Add ToT method

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Add Koashi

* Update Kaoshi

* Update Kaoshi

* Update kaoshi

* Update kaoshi

* Update Kaoshi

* Update Kaoshi

* Update Kaoshi

* Update Kaoshi

* update Kaoshi

* update

* update

* fix

---------
Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>
 2023-09-22 18:46:33 +08:00
TTTTTiam
2a62bea1a4
add evaluation of scibench (#393) 
* add evaluation of scibench

* add evaluation of scibench

* update scibench

* remove scibench evaluator

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-09-22 17:42:08 +08:00
Hubert
8803f7f7a6
[Feat] support antropics evals dataset (#422) 
* [Feat] support anthropics ai risk dataset

* [Feat] support anthropics evals dataset

* [Feat] support anthropics evals dataset
 2023-09-20 18:36:44 +08:00
Hubert
2c15a0c01d
[Feat] refine docs and codes for more user guides (#409) 2023-09-18 16:12:13 +08:00
Hubert
a11cb45c83
[Feat] implementation for support promptbench (#239) 
* [Feat] support adv_glue dataset for adversarial robustness

* reorg files

* minor fix

* minor fix

* support prompt bench demo

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix
 2023-09-15 15:06:53 +08:00
Hubert
de8a154795
[Feat] support ds1000 dataset (#395) 
* [Feat] support ds1000 datase
 2023-09-15 12:50:27 +08:00
Xidong Wang
47a752cd56
[Dataset] Add CMB (#376) 
* Add CMB

* modify CMB

---------

Co-authored-by: wangxidong <xidongw@163.com>
 2023-09-12 19:16:41 +08:00
Leymore
b48d084020
[Fix] update bbh implement & fix bbh suffix (#371) 2023-09-08 15:14:30 +08:00
Hubert
ddb8197212
[Feat] support wizardcoder series (#344) 
* [Feat] support wizardcoder series

* minor fix
 2023-09-06 17:52:35 +08:00
Leymore
7ca6ba625e
[Feature] Add qwen & qwen-chat support (#286) 
* add and apply update suffix tool

* add tool doc

* add qwen configs

* add cmmlu

* rename bbh

* update datasets

* delete

* update hf_qwen_7b.py
 2023-08-31 11:29:05 +08:00
Leymore
c26ecdb1b0
[Feature] Add and apply update suffix tool (#280) 
* add and apply update suffix tool

* add dataset suffix updater as precommit hook

* update workflow

* update scripts

* update ci

* update

* ci with py3.8

* run in serial

* update bbh

* use py 3.10

* update pre commit zh cn
 2023-08-28 17:35:04 +08:00
Tong Gao
9058be07b8
[Feature] Simplify entry script (#204) 
* [Feature] Simply entry script

* update
 2023-08-25 17:36:30 +08:00
Tong Gao
fda42fd5fd
[Fix] wrong path in dataset collections (#272) 2023-08-25 15:50:30 +08:00
philipwangOvO
3f37c40aa3
[Dataset] Refactor LEval 2023-08-25 11:46:23 +08:00
liushz
02ce139bc6
[Feature] Add Tree-of-Thought method (#173) 
* Add ToT method

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update chain_of_thought.md

* Update icl_tot_inferencer.py

---------

Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
 2023-08-23 12:23:05 +08:00
philipwangOvO
655a807f4b
[Dataset] LongBench (#236) 
Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>
 2023-08-21 14:15:20 +08:00
Ezra-Yu
17ccaa5980
[Feat] Add codegeex2 and Humanevalx (#210) 
* add codegeex2

* add humanevalx dataset

* add evaluator

* update evaluator

* update configs

* update clean code

* update configs

* fix lint

* remove sleep

* fix lint

* update docs

* fix lint
 2023-08-17 11:03:16 +08:00
Hubert
0fe2366a72
[Feat] support adv_glue dataset for adversarial robustness (#205) 
* [Feat] support adv_glue dataset for adversarial robustness

* reorg files

* minor fix

* minor fix
 2023-08-16 18:42:06 +08:00
Hubert
7c393192af
[Fix] fix bug for postprocessor (#195) 
* [Fix] fix bug for postprocessor

* minor fix
 2023-08-11 18:41:12 +08:00
Tong Gao
bf79ff1c6d
[Feature] Add LEval datasets 
Co-authored-by: kennymckormick <dhd@pku.edu.cn>
 2023-08-11 17:38:31 +08:00
Hubert
8d9cee060f
[Feat] update postprocessor to get first option more accurately (#193) 
* [Feat] update postprocessor to get first option

* minor fix

* minor fix
 2023-08-11 17:33:00 +08:00
Leymore
14332e08fd
[Feature] add llama-oriented dataset configs (#82) 
* add llama-oriented dataset configs

* update

* revert cvalues & update llama_example
 2023-08-11 12:48:05 +08:00
Hubert
5a9539f375
[Feat] add safety to collections (#185) 
* [Feat] add safety to collections

* minor fix
 2023-08-11 11:19:26 +08:00
Tong Gao
2931f3dcb8
[Enhancement] Add humaneval postprocessor for GPT models & eval config for GPT4, enhance the original humaneval postprocessor (#129) 
* [Enhancement] Enhance humaneval postprocessor

* add human-eval testcase

* update

* update

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-08-10 16:31:12 +08:00
Leymore
e7fc54baf1
[Feature] Add Xiezhi SQuAD2.0 ANLI (#101) 
* add Xiezhi SQuAD2.0 ANLI; update WSC

* update

* update

* update doc string
 2023-08-10 14:04:18 +08:00
Leymore
876ade71a5
[Fix] Fix AGIEval multiple choice (#137) 
* update agieval data

* rename variables
 2023-08-10 11:38:24 +08:00
Tong Gao
c00179d46b
[Feature] Evaluating acc based on minimum edit distance, update SIQA (#130) 
* [Feature] Support evaluating acc based on minimum edit distance, update SIQA

* update
 2023-08-01 14:24:27 +08:00
Leymore
d862f570aa
[Feature] Add SC (#126) 
* add self-consistency

* add CoT method Self-Consistency

* fix typo error and update openicl_eval

* add tydiQA-GoldP task

* fix sc

* rename gsm8k_sc

* fix sc

* add self-consistency doc

* refine sc

---------

Authored-by: liushz <qq1791167085@163.com>
 2023-07-28 17:29:37 +08:00
Hubert
b7184e9db5
[Refactor] Update crows-pairs evaluation (#98) 
* [Refactor] Update crows-pairs evaluation

* [Refactor] Update crows-pairs evaluation

* minor
 2023-07-26 11:21:32 +08:00
Haonan Li
e9cdb24ddd
[Feature] Add CMMLU dataset (#91) 
* add CMMLU

* debug cmmlu

* add slurm args `qos`

* fix format: space before comment

* remove unused variable

* change the location of `answer is`

---------

Co-authored-by: 李浩楠 <lihaonan@lihaonandeMacBook-Air.local>
Co-authored-by: 李浩楠 <haonan.li>
Co-authored-by: Leymore <zfz-960727@163.com>
 2023-07-25 10:14:27 +08:00
Hubert
f83e125e5a
[Feat] Support CValues Responsibility dataset (#78) 
* [Feat] support CValues

* minor fix
 2023-07-18 18:45:15 +08:00
liushz
f36c0496f3
[Feature] Add tydiqa-goldp (#75) 
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
 2023-07-18 14:54:35 +08:00
Leymore
1326aff77e
[Feature] Add logger info and remove dataset bugs (#61) 
* Add logger info and remove dataset bugs

* fix typo
 2023-07-17 14:26:30 +08:00
Leymore
86d5ec3d0f
Update configs (#9) 
* Update implements

* Update
 2023-07-06 12:27:41 +08:00
Tong Gao
16e759b996
Align prompt files with their hash (#1) 
* fix bbh

* fix bbh

* rename
 2023-07-05 18:28:58 +08:00
mzr1996
04dd01a235 Update configs and code 2023-07-05 11:45:08 +08:00
Leymore
c94cc94348 Add release contribution 2023-07-05 03:15:31 +00:00
tonysy
e6b5bdcb87 OpenCompass Public MR 2023-07-05 03:15:21 +00:00
Ezra-Yu
cbe9fe2cdb Add Release Contraibution 2023-07-05 02:22:40 +00:00
cky
36f111100f update datasets 2023-07-05 01:45:26 +00:00
mzr1996
3cfe73de3f Support a batch of datasets. 2023-07-05 01:30:27 +00:00
yingfhu
fb11108723 [Feat] support opencompass 2023-07-04 22:11:33 +08:00
gaotongxiao
7d346000bb initial commit 2023-07-04 21:34:55 +08:00