18 Commits

Author	SHA1	Message	Date
Xingjun.Wang	edab1c07ba	[Feature] Support ModelScope datasets (#1289) ... * add ceval, gsm8k modelscope surpport * update race, mmlu, arc, cmmlu, commonsenseqa, humaneval and unittest * update bbh, flores, obqa, siqa, storycloze, summedits, winogrande, xsum datasets * format file * format file * update dataset format * support ms_dataset * udpate dataset for modelscope support * merge myl_dev and update test_ms_dataset * udpate dataset for modelscope support * update readme * update eval_api_zhipu_v2 * remove unused code * add get_data_path function * update readme * remove tydiqa japanese subset * add ceval, gsm8k modelscope surpport * update race, mmlu, arc, cmmlu, commonsenseqa, humaneval and unittest * update bbh, flores, obqa, siqa, storycloze, summedits, winogrande, xsum datasets * format file * format file * update dataset format * support ms_dataset * udpate dataset for modelscope support * merge myl_dev and update test_ms_dataset * update readme * udpate dataset for modelscope support * update eval_api_zhipu_v2 * remove unused code * add get_data_path function * remove tydiqa japanese subset * update util * remove .DS_Store * fix md format * move util into package * update docs/get_started.md * restore eval_api_zhipu_v2.py, add environment setting * Update dataset * Update * Update * Update * Update --------- Co-authored-by: Yun lin <yunlin@U-Q9X2K4QV-1904.local> Co-authored-by: Yunnglin <mao.looper@qq.com> Co-authored-by: Yun lin <yunlin@laptop.local> Co-authored-by: Yunnglin <maoyl@smail.nju.edu.cn> Co-authored-by: zhangsongyang <zhangsongyang@pjlab.org.cn>	2024-07-29 13:48:32 +08:00
Fengzhe Zhou	d59189b87f	[Doc] Update running command in README (#1206)	2024-05-30 00:06:39 +08:00
Fengzhe Zhou	2954913d9b	[Sync] bump version (#1204)	2024-05-28 23:09:59 +08:00
Fengzhe Zhou	2b3d4150f3	[Sync] update evaluator (#1175)	2024-05-21 14:22:46 +08:00
Fengzhe Zhou	aa2dd2b58c	[Format] Add config lints (#892)	2024-05-14 15:35:58 +08:00
Fengzhe Zhou	d43392a3bb	[Feature] Add mmlu prompt from simple_evals, openai (#1074) ... * add mmlu prompt from simple_evals, openai * return empty str on failure	2024-05-06 13:26:26 +08:00
Fengzhe Zhou	b03d5dc531	[Sync] Sync Internal (#941)	2024-03-04 14:42:36 +08:00
Fengzhe Zhou	d34ba11106	[Sync] Merge branch 'dev' into zfz/update-keyset-demo (#876)	2024-02-05 23:29:10 +08:00
liyucheng09	0b2863039e	[Feature] Contamination analysis for MMLU, Hellaswag, and ARC_c (#699) ... * Contamination analysis for ARC_c, mmlu, and Hellaswag * update `eval_contamination.py` * update `contamination.py` summarizer * fix `eval_contamination.py` * add mmlu groups for contamination analysis	2024-01-08 15:51:48 +08:00
Yang Yong	54345c56b7	Update LightllmApi and Fix mmlu bug (#738) ... * Update LightllmApi and Fix mmlu bug * checkout mmlu_gen_a484b3.py --------- Co-authored-by: Leymore <zfz-960727@163.com>	2023-12-27 13:49:08 +08:00
Fengzhe Zhou	9083dea683	[Sync] some renaming (#641)	2023-11-27 16:06:49 +08:00
Tong Gao	119bfd1569	[Refactor] Move fix_id_list to Retriever (#442) ... * [Refactor] Move fix_id_list to Retriever * update * move to base * fix	2023-10-07 12:53:41 +08:00
Leymore	86d5ec3d0f	Update configs (#9) ... * Update implements * Update	2023-07-06 12:27:41 +08:00
Tong Gao	16e759b996	Align prompt files with their hash (#1) ... * fix bbh * fix bbh * rename	2023-07-05 18:28:58 +08:00
mzr1996	04dd01a235	Update configs and code	2023-07-05 11:45:08 +08:00
Leymore	c94cc94348	Add release contribution	2023-07-05 03:15:31 +00:00
tonysy	e6b5bdcb87	OpenCompass Public MR	2023-07-05 03:15:21 +00:00
gaotongxiao	7d346000bb	initial commit	2023-07-04 21:34:55 +08:00