Mor-Li
d1da4a577c
Add NeedleBench_V2
2025-05-09 19:37:39 +08:00
bittersweet1999
ddc9cc0afb
[Add] add a config to Judge dataset all ( #2077 )
...
* fix pip version
* fix pip version
* add judgedatasetall
* add judgedatasetall
* add judgedatasetall
2025-05-07 10:57:23 +08:00
bittersweet1999
37cbaf8d92
[Add] Add Judgerbenchv2 ( #2067 )
...
* fix pip version
* fix pip version
* add judgerbenchv2
* Update __init__.py
2025-04-30 17:12:34 +08:00
Taolin Zhang
b6148aa198
add Judgebench ( #2066 )
...
* add rewardbench
* add rewardbench
* add rmb datasets
* add rmb datasets
* add judgebench
* add judgebench
2025-04-30 15:01:10 +08:00
Taolin Zhang
c69110361b
[Add] add rewardbench ( #2029 )
...
* add rewardbench
* add rewardbench
2025-04-21 17:18:51 +08:00
Myhs_phz
75e7834b59
[Feature] Add Datasets: ClimateQA,Physics ( #2017 )
...
* feat ClimateQA
* feat PHYSICS
* fix
* fix
* fix
* fix
2025-04-14 20:18:47 +08:00
Songyang Zhang
c98599271b
[Update] Update OlympiadBench and Update LLM Judge ( #1954 )
2025-03-18 20:15:20 +08:00
Linchen Xiao
0b7f76e193
[Bug] Fix Summarizer logic ( #1953 )
2025-03-17 18:25:08 +08:00
Yufeng Zhao
15c825a51a
[Update] Bbeh harmony summarizer added ( #1951 )
...
* bbeh
* bbeh
* fix_smallbugs_bbeh
* removeprint
* harmonic
* update_summerizer
* harmonic-tested
* harmonic-tested
* clean
* clean
* cleaned_rebased
---------
Co-authored-by: yufeng zhao <zhaoyufeng@pjlab.org.cn>
2025-03-17 17:19:56 +08:00
Yufeng Zhao
bc2969dba8
[Feature] Add support for BBEH dataset ( #1925 )
...
* bbeh
* bbeh
* fix_smallbugs_bbeh
* removeprint
* results
---------
Co-authored-by: yufeng zhao <zhaoyufeng@pjlab.org.cn>
2025-03-12 10:53:31 +08:00
Songyang Zhang
c84bc18ac1
[Update] Support OlympiadBench-Math/OmniMath/LiveMathBench-Hard ( #1899 )
...
* [Update] Support OlympiadBench-Math/OmniMath/LiveMathBench-Hard with LLM Verify
* Update
* Update
* Update DeepSeek-R1 example
* Update DeepSeek-R1 example
* Update DeepSeek-R1 example
2025-03-03 18:56:11 +08:00
Shudong Liu
412199f802
[Feature] Support OlympiadBench Benchmark ( #1841 )
...
* Support OlympiadBench Benchmark
* Support OlympiadBench Benchmark
* Support OlympiadBench Benchmark
* update dataset path
* Update olmpiadBench
* Update olmpiadBench
* Update olmpiadBench
---------
Co-authored-by: liushz <qq1791167085@163.com>
2025-01-24 10:00:01 +08:00
Linchen Xiao
a6193b4c02
[Refactor] Code refactoarization ( #1831 )
...
* Update
* fix lint
* update
* fix lint
2025-01-20 19:17:38 +08:00
Zhao Qihao
e039f3efa0
[Feature] Support MMLU-CF Benchmark ( #1775 )
...
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* Update mmlu-cf
* Update mmlu-cf
* Update mmlu-cf
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* [Feature] Support MMLU-CF Benchmark
* Remove outside configs
---------
Co-authored-by: liushz <qq1791167085@163.com>
2025-01-09 14:11:20 +08:00
liushz
c437135fad
[Feature] Add Openai Simpleqa dataset ( #1720 )
...
* Add Openai SimpleQA dataset
* Add Openai SimpleQA dataset
* Add Openai SimpleQA dataset
* Update eval_simpleqa.py
---------
Co-authored-by: Linchen Xiao <xxllcc1993@gmail.com>
2024-11-28 19:16:07 +08:00
wanyu2018umac
90efcf2216
[Feature] Add P-MMEval ( #1714 )
...
* Update with PMMEval
* Update
* Update __init__.py
* Fix Bugs
* Delete .pre-commit-config.yaml
* Pull merge
---------
Co-authored-by: liushz <qq1791167085@163.com>
2024-11-27 21:26:18 +08:00
Yufeng Zhao
300adc31e8
[Feature] Add Korbench dataset ( #1713 )
...
* first version for korbench
* first stage for korbench
* korbench_1
* korbench_1
* korbench_1
* korbench_1
* korbench_1_revised
* korbench_combined_1
* korbench_combined_1
* kor_combined
* kor_combined
* update
---------
Co-authored-by: MaiziXiao <xxllcc1993@gmail.com>
2024-11-25 20:11:27 +08:00
Chang Lan
5c1916ea4c
[Update] Add RULER 64k config ( #1709 )
2024-11-25 19:35:27 +08:00
abrohamLee
e9e4b69ddb
[Feature] MuSR Datset Evaluation ( #1689 )
...
* MuSR Datset Evaluation
* MuSR Datset Evaluation
Add an assertion and a Readme.md
2024-11-14 20:42:12 +08:00
Linchen Xiao
e92a5d4230
[Feature] BABILong Dataset added ( #1684 )
...
* update
* update
* update
* update
2024-11-14 15:32:43 +08:00
Linchen Xiao
835bf75a36
[Feature] Add long context evaluation for base models ( #1666 )
...
* [Update] Add base long context evaluation
* update
2024-11-08 10:53:29 +08:00
Bob Tsang
dd0b655bd0
[Feature] Support MMMLU & MMMLU-lite Benchmark ( #1565 )
...
* rm folder
* modify format according to reviewer
* modify format according to reviewer
* modify format according to reviewer
* add some files requirement
* fix some bug
* fix bug
* change load type
* Update MMMLU Dataset
* Update MMMLU Dataset
* Add MMMLU-Lite Dataset
* update MMMMLU datast
* update MMMMLU datast
* update MMMMLU datast
---------
Co-authored-by: BobTsang <BobTsang1995@gmail.com>
Co-authored-by: liushz <qq1791167085@163.com>
2024-10-17 19:09:34 +08:00
klein
24915aeb3f
[BUG] Update CIbench config( #1544 )
...
* BUG: Update cibench.py
* BUG: Update cibench.py
2024-09-23 18:32:27 +08:00
Songyang Zhang
6997990c93
[Feature] Update Models ( #1518 )
...
* Update Models
* Update
* Update humanevalx
* Update
* Update
2024-09-12 23:35:30 +08:00
Alexander Lam
a31a77c5c1
[Feature] Add SciCode summarizer config ( #1514 )
...
* [Feature] added SciCode summarizer config and dataset config for with background evaluation
* fix lint issues
* removed unnecessary type in summarizer group
2024-09-10 16:06:02 +08:00
Linchen Xiao
6c9cd9a260
[Feature] Needlebench auto-download update ( #1480 )
...
* update
* update
* update
2024-09-05 17:22:42 +08:00
liushz
9fdbc744dc
[Fix] Update option postprocess & mathbench language summarizer ( #1413 )
...
* Update option postprocess & mathbench language summarizer
* Update option postprocess & mathbench language summarizer
---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
2024-08-22 14:49:07 +08:00
Linchen Xiao
a4b54048ae
[Feature] Add Ruler datasets ( #1310 )
...
* [Feature] Add Ruler datasets
* pre-commit fixed
* Add model specific tokenizer to dataset
* pre-commit modified
* remove unused import
* fix linting
* add trust_remote to tokenizer load
* lint fix
* comments resolved
* fix lint
* Add readme
* Fix lint
* ruler refactorize
* fix lint
* lint fix
* updated
* lint fix
* fix wonderwords import issue
* prompt modified
* update
* readme updated
* update
* ruler dataset added
* Update
---------
Co-authored-by: tonysy <sy.zhangbuaa@gmail.com>
2024-08-20 11:40:11 +08:00
Linchen Xiao
8e55c9c6ee
[Update] Compassbench v1.3 ( #1396 )
...
* stash files
* compassbench subjective evaluation added
* evaluation update
* fix lint
* update docs
* Update lint
* changes saved
* changes saved
* CompassBench subjective summarizer added (#1349 )
* subjective summarizer added
* fix lint
[Fix] Fix MathBench (#1351 )
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
[Update] Update model support list (#1353 )
* fix pip version
* fix pip version
* update model support
subjective summarizer updated
knowledge, math objective done (data need update)
remove secrets
objective changes saved
knowledge data added
* secrets removed
* changed added
* summarizer modified
* summarizer modified
* compassbench coding added
* fix lint
* objective summarizer updated
* compass_bench_v1.3 updated
* update files in config folder
* remove unused model
* lcbench modified
* removed model evaluation configs
* remove duplicated sdk implementation
---------
Co-authored-by: zhangsongyang <zhangsongyang@pjlab.org.cn>
2024-08-12 19:09:19 +08:00
Songyang Zhang
c09fc79ba8
[Feature] Support OpenAI ChatCompletion ( #1389 )
...
* [Feature] Support import configs/models/summarizers from whl
* Update
* Update openai sdk
* Update
* Update gemma
2024-08-01 19:10:13 +08:00
Songyang Zhang
46cc7894e1
[Feature] Support import configs/models/summarizers from whl ( #1376 )
...
* [Feature] Support import configs/models/summarizers from whl
* Update LCBench configs
* Update
* Update
* Update
* Update
* update
* Update
* Update
* Update
* Update
* Update
2024-08-01 00:42:48 +08:00
