Myhs-phz
c5abd9201a
fix
2025-03-24 11:24:28 +00:00
Myhs-phz
8e1a41c532
fix
2025-03-24 11:20:46 +00:00
Myhs-phz
21cbb2691d
fix
2025-03-24 11:17:05 +00:00
Myhs-phz
7cfe1cfe04
fix
2025-03-24 11:06:03 +00:00
Myhs-phz
4261b316a6
fix
2025-03-24 10:40:58 +00:00
Myhs-phz
6a51f41c31
fix
2025-03-24 10:34:41 +00:00
Myhs-phz
796e2d0177
fix
2025-03-24 10:29:32 +00:00
Myhs-phz
199052e92e
fix
2025-03-24 10:15:34 +00:00
Myhs-phz
4f07a1c457
fix
2025-03-24 08:56:04 +00:00
Myhs-phz
2639d113d8
fix
2025-03-24 08:27:45 +00:00
Myhs_phz
37307fa996
[Update] Add QWQ32b model config ( #1959 )
...
lint / lint (push) Waiting to run
* feat qwq-32b
* fix
* feat phi_4
---------
Co-authored-by: Linchen Xiao <xxllcc1993@gmail.com>
2025-03-24 14:51:39 +08:00
Linchen Xiao
db96161a4e
[Update] Add SuperGPQA subset metrics ( #1966 )
2025-03-24 14:25:12 +08:00
Linchen Xiao
aa05993922
[Update] Add dataset configurations of no max_out_len ( #1967 )
...
* [Update] Add dataset configurations of no max_out_len
* update test torch version
* update test torch version
* update test torch version
* update test torch version
2025-03-24 14:24:12 +08:00
Linchen Xiao
64128916d0
[Update] Increase memory size for CPU job of VOLC Runner ( #1962 )
...
* [Update] Increase memory size for CPU job of VOLC Runner
* [Update] Increase memory size for CPU job of VOLC Runner
2025-03-24 11:21:14 +08:00
Dongsheng Zhu
8a5029b121
[Feature] Add MultiPL-E & Code Evaluator ( #1963 )
...
* multiple_code develop
* multiple_code update
* comments upadate
* index upadate
2025-03-21 20:09:25 +08:00
Myhs-phz
b1c2f60d32
fix hook
2025-03-19 09:14:52 +00:00
Myhs-phz
ffe00a830d
feat
2025-03-19 03:37:23 +00:00
Myhs-phz
716c02785c
fix and doc
2025-03-19 02:03:45 +00:00
Myhs-phz
cc9761e882
fix
2025-03-19 01:16:59 +00:00
Linchen Xiao
b9de8b0e2b
[Update] Unset disallowed_special token for Openai model ( #1960 )
2025-03-18 20:24:07 +08:00
Songyang Zhang
c98599271b
[Update] Update OlympiadBench and Update LLM Judge ( #1954 )
2025-03-18 20:15:20 +08:00
Jason Cheung
5d2d253d83
[BUG] Fix model_kwargs pass logic for vllm ( #1958 )
2025-03-18 20:08:15 +08:00
Myhs-phz
b9b69febc3
back
2025-03-18 09:33:06 +00:00
Linchen Xiao
0b7f76e193
[Bug] Fix Summarizer logic ( #1953 )
2025-03-17 18:25:08 +08:00
Myhs-phz
53c6725d19
fix
2025-03-17 09:40:57 +00:00
Yufeng Zhao
15c825a51a
[Update] Bbeh harmony summarizer added ( #1951 )
...
* bbeh
* bbeh
* fix_smallbugs_bbeh
* removeprint
* harmonic
* update_summerizer
* harmonic-tested
* harmonic-tested
* clean
* clean
* cleaned_rebased
---------
Co-authored-by: yufeng zhao <zhaoyufeng@pjlab.org.cn>
2025-03-17 17:19:56 +08:00
Linchen Xiao
854c6bf025
[Update] Update requirement and base evaluator
2025-03-13 20:52:50 +08:00
Linchen Xiao
1c60e3a0f6
[Update] Add configurations for llmjudge dataset ( #1940 )
...
* Add configurations for llmjudge dataset
* update
2025-03-13 17:30:04 +08:00
Myhs-phz
51f5792f7c
fix
2025-03-13 08:54:30 +00:00
Myhs-phz
60b230e285
fix datasets in fullbench_int3
2025-03-12 13:58:58 +00:00
liushz
709bc4af0e
[Update] Add AIME2025 oss info ( #1936 )
...
* Support OlympiadBench Benchmark
* Support OlympiadBench Benchmark
* Support OlympiadBench Benchmark
* update dataset path
* Update olmpiadBench
* Update olmpiadBench
* Update olmpiadBench
* Add HLE dataset
* Add HLE dataset
* Add HLE dataset
* Add AIME2025 oss info
---------
Co-authored-by: sudanl <sudanl@foxmail.com>
2025-03-12 18:41:16 +08:00
Yufeng Zhao
bc2969dba8
[Feature] Add support for BBEH dataset ( #1925 )
...
* bbeh
* bbeh
* fix_smallbugs_bbeh
* removeprint
* results
---------
Co-authored-by: yufeng zhao <zhaoyufeng@pjlab.org.cn>
2025-03-12 10:53:31 +08:00
Myhs-phz
4b9838fb94
feat datasetrefine drop
2025-03-11 11:59:05 +00:00
Kangreen
59e49aedf1
[Feature] Support SuperGPQA ( #1924 )
...
* support supergpqa
* remove unnecessary code
* remove unnecessary code
* Add Readme
* Add Readme
* fix lint
* fix lint
* update
* update
---------
Co-authored-by: mkj3085003 <mkj3085003@gmail.com>
Co-authored-by: MaiziXiao <xxllcc1993@gmail.com>
2025-03-11 19:32:08 +08:00
Linchen Xiao
e403fd21be
[Fix] Fix math-verify evaluator ( #1917 )
...
* update
* update
* update
2025-03-11 17:35:04 +08:00
Linchen Xiao
cbf84fb33c
[Feature] Update LLM Evaluation for MMLU-Pro ( #1923 )
2025-03-07 21:01:20 +08:00
Myhs_phz
570c30cf1b
[Fix] Fix CLI option for results persistence ( #1920 )
...
* fix
* fix
* fix
2025-03-07 18:24:30 +08:00
Shudong Liu
277d7946f5
[Fix] Fix typo in deepseed_r1.md ( #1916 )
2025-03-05 19:37:22 +08:00
Myhs_phz
1585c0adbe
[Feature] Evaluation Results Persistence ( #1894 )
...
* feat results_station.py
* lint
* feat save_to_station
* feat result_station.py and lint
* feat
* fix
* fix and lint
* fix
* fix subjective processing
* fix
* fix
* style function name
* lint
2025-03-05 18:33:34 +08:00
Myhs_phz
54324657f0
[Docs] Results persistance ( #1908 )
...
* feat persistance.md
* doc
* doc
* lint
* doc
* fix
* doc
2025-03-05 18:23:54 +08:00
Dongsheng Zhu
fff2d51440
[Update] Code evaluation alignment ( #1909 )
...
* code alignment
* update oss md5
* bigcodebench update
* lint
* lint_
* lint yapf
2025-03-04 18:49:38 +08:00
Linchen Xiao
5547fd1592
[Bump] Bump version to 0.4.1
2025-03-04 18:26:14 +08:00
liushz
198c08632e
[Feature] Add HLE (Humanity's Last Exam) dataset ( #1902 )
...
* Support OlympiadBench Benchmark
* Support OlympiadBench Benchmark
* Support OlympiadBench Benchmark
* update dataset path
* Update olmpiadBench
* Update olmpiadBench
* Update olmpiadBench
* Add HLE dataset
* Add HLE dataset
* Add HLE dataset
---------
Co-authored-by: sudanl <sudanl@foxmail.com>
2025-03-04 16:42:37 +08:00
Songyang Zhang
c84bc18ac1
[Update] Support OlympiadBench-Math/OmniMath/LiveMathBench-Hard ( #1899 )
...
* [Update] Support OlympiadBench-Math/OmniMath/LiveMathBench-Hard with LLM Verify
* Update
* Update
* Update DeepSeek-R1 example
* Update DeepSeek-R1 example
* Update DeepSeek-R1 example
2025-03-03 18:56:11 +08:00
Junnan Liu
f0809fe6f6
[Update] Fix Hard Configs With General GPassK ( #1906 )
...
* support dataset repeat and g-pass compute for each evaluator
* fix pre-commit errors
* delete print
* delete gpassk_evaluator and fix potential errors
* change `repeat` to `n`
* fix `repeat` to `n` in openicl_eval
* update doc for multi-run and g-pass
* update latex equation in doc
* update eng doc for multi-run and g-pass
* update datasets.md
* update datasets.md
* fix multi-line equation
* fix multi-line equation
* fix multi-line equation
* fix multi-line equation
* fix multi-line equation
* fix multi-line equation
* fix multi-line equation in zh_cn user_guides
* mmodify pre-commit-zh-cn
* recover pre-commit and edit math expr in doc
* del [TIP]
* del cite tag in doc
* del extract_model param in livemathbench config
* fix livemathbench hard configs
2025-03-03 18:17:15 +08:00
Linchen Xiao
6a573f671b
[Fix] Fix compatible issue
2025-03-03 15:35:57 +08:00
Junnan Liu
73c80953c6
[Feature] Support Dataset Repeat and G-Pass Compute for Each Evaluator ( #1886 )
...
* support dataset repeat and g-pass compute for each evaluator
* fix pre-commit errors
* delete print
* delete gpassk_evaluator and fix potential errors
* change `repeat` to `n`
* fix `repeat` to `n` in openicl_eval
* update doc for multi-run and g-pass
* update latex equation in doc
* update eng doc for multi-run and g-pass
* update datasets.md
* update datasets.md
* fix multi-line equation
* fix multi-line equation
* fix multi-line equation
* fix multi-line equation
* fix multi-line equation
* fix multi-line equation
* fix multi-line equation in zh_cn user_guides
* mmodify pre-commit-zh-cn
* recover pre-commit and edit math expr in doc
* del [TIP]
* del cite tag in doc
* del extract_model param in livemathbench config
2025-02-26 19:43:12 +08:00
zhulinJulia24
6042b88e58
[CI] update dailytest sceduler and baseline's score( #1898 )
2025-02-26 19:04:01 +08:00
Linchen Xiao
bdb2d46f59
[Feature] Add general math, llm judge evaluator ( #1892 )
...
* update_doc
* update llm_judge
* update README
* update md file name
2025-02-26 15:08:50 +08:00
Songyang Zhang
fd6fbf01a2
[Update] Support AIME-24 Evaluation for DeepSeek-R1 series ( #1888 )
...
* Update
* Update
* Update
* Update
2025-02-25 20:34:41 +08:00
