x54-729
bbdca5eb4c
[BUG] Fix eos token handling and add comments for InternTrain ( #1569 )
...
Co-authored-by: x54-729 <xingshuhao.dispatch@pjlab.org.cn>
2024-09-30 15:46:06 +08:00
Linchen Xiao
763d7755b6
[BUG]GaokaoBench dataset fix ( #1583 )
2024-09-30 15:13:26 +08:00
shijinpjlab
7528b8ab8a
[Feature] Add dingo test ( #1529 )
...
* add qa dingo
* update
* change name qa to dingo
* eval model: llm_base
* update path
* change name and move path
* add eval_dingo
* update import
* add for pip
* add dingo package
* change import place
* update import place
* fix lint fail
* isort
* double quoted
---------
Co-authored-by: sj <shijin@pjlab.org.cn>
2024-09-29 19:24:58 +08:00
Yi Ding
85a28874aa
[BUG]: Fix Bailing API configs ( #1570 )
2024-09-27 11:56:57 +08:00
Songyang Zhang
e8437db98f
[Feature] Update BailingLM/OpenAI verbose ( #1568 )
...
* [Feature] 1. Update CoreBench Base\n 2. Fix lint issue in BalingAPI
* Update
* [Feature] Update API
* Update
2024-09-27 11:15:25 +08:00
Songyang Zhang
7d50294117
[Feature] Update Bailing ( #1567 )
...
* [Feature] 1. Update CoreBench Base\n 2. Fix lint issue in BalingAPI
* Update
* Update
* Update
2024-09-26 18:56:17 +08:00
Songyang Zhang
a7bacfdf7e
[Feature] Update CoreBench 2.0 ( #1566 )
...
* [Feature] 1. Update CoreBench Base\n 2. Fix lint issue in BalingAPI
* Update
* Update
2024-09-26 18:44:00 +08:00
Yi Ding
3f833186dc
[Feature] Support the reasoning from BaiLing LLM ( #1541 )
...
* [Feature] Support the reasoning from BaiLing LLM
This commit includes the access to BaiLing LLM and gets the reasoning.
* Add the api example
The example of evalute bailing api
* Revise the generation arguments
Based on current experiment, we update some generation arguments for better reasoning
* [fix] set the batch size
* Retry under flowcontrol of serverside
* add dependent package into requirement.txt
add dependent package retrying to clean up the pre-comment check.
* correct the file names and make the file copy
correct the file names.
copy the files under configs to opencompass
* fix the lint issue
---------
Co-authored-by: christopher.dy <christopher.dy@antgroup.com>
2024-09-26 16:49:52 +08:00
Linchen Xiao
80cda1980e
[BUG] fix followbench dataset config ( #1564 )
...
* [BUG] fix followbench dataset config
* [BUG] fix followbench dataset config
2024-09-25 20:58:34 +08:00
zhulinJulia24
aa43eaf267
[CI] add more models into testcase and test env of cu12 ( #1558 )
...
* update
* update
* Update pr-run-test.yml
* update
* update
* update
* update
* Update daily-run-test.yml
* update
* updaste
* update
* update
* update
* Update daily-run-test.yml
* update
* update
* Update daily-run-test.yml
* Update daily-run-test.yml
* update
* update
* update
* update
* update
* Update daily-run-test.yml
* update
---------
Co-authored-by: zhulin1 <zhulin1@pjlab.org.cn>
2024-09-25 17:07:27 +08:00
zhulinJulia24
87df8a73a3
[CI] add a common summarizer for qabench summarizer ( #1545 )
...
* update
* update
* update
---------
Co-authored-by: zhulin1 <zhulin1@pjlab.org.cn>
2024-09-25 13:40:47 +08:00
Linchen Xiao
c3fb9065db
[Feature] Add dlc sleep time ( #1562 )
2024-09-25 11:53:48 +08:00
Songyang Zhang
fe84bbd9a0
[Feature] Add Config for CoreBench ( #1547 )
...
* [Feature] Add Config for CoreBench
* Update
2024-09-25 11:36:43 +08:00
Chuanyang Jin
17eefc0e1e
[Fix] Correct typos ( #1561 )
2024-09-25 11:27:17 +08:00
liushz
83eeb52b09
[Feature] Update WikiBench base model config ( #1553 )
...
* Update MathBench & WikiBench for FullBench
* Update MathBench & WikiBench for FullBench
* Update GPQA & MMLU_Pro
* Update MathBench & WikiBench for FullBench
* Update MathBench & WikiBench for FullBench
* Update MathBench & WikiBench for FullBench
* Update MathBench & Math base config
* Update WikiBench base model config
---------
Co-authored-by: liushz <liuhongwei@pjlab.rog.cn>
2024-09-25 11:26:36 +08:00
Songyang Zhang
e7681943f3
[Feature] Update the max_out_len for many models ( #1559 )
2024-09-24 21:52:28 +08:00
bittersweet1999
a2e9bc0c41
[Fix] fix duplicate error in partitioner ( #1552 )
...
* fix pip version
* fix pip version
* fix duplicate error in paritioner
* fix duplicate error in paritioner
2024-09-23 19:45:21 +08:00
x54-729
335667183a
[Feature] Add Interntrain model support ( #1548 )
...
Co-authored-by: x54-729 <xingshuhao.dispatch@pjlab.org.cn>
2024-09-23 19:10:26 +08:00
klein
24915aeb3f
[BUG] Update CIbench config( #1544 )
...
* BUG: Update cibench.py
* BUG: Update cibench.py
2024-09-23 18:32:27 +08:00
liushz
a0cfd61129
[Feature] Update MathBench & Math base model config ( #1550 )
...
* Update MathBench & WikiBench for FullBench
* Update MathBench & WikiBench for FullBench
* Update GPQA & MMLU_Pro
* Update MathBench & WikiBench for FullBench
* Update MathBench & WikiBench for FullBench
* Update MathBench & WikiBench for FullBench
* Update MathBench & Math base config
---------
Co-authored-by: liushz <liuhongwei@pjlab.rog.cn>
2024-09-23 14:03:59 +08:00
Songyang Zhang
ee058e25b2
[Feature] Support verbose for OpenAI API ( #1546 )
2024-09-20 17:12:52 +08:00
hailsham
a81bbb85bf
[FIX] Added handling for the "begin section" in meta_template to APITemplateParser ( #1405 )
...
Co-authored-by: leifei <nuuooo@icloud.com>
2024-09-19 18:12:04 +08:00
Songyang Zhang
5a27c2bd6f
[Model] Support Qwen2.5 Instruct ( #1543 )
2024-09-19 16:16:07 +08:00
Songyang Zhang
be460fbb21
[Feature] Support OpenAI O1 models ( #1539 )
...
* [Feature] Support OpenAI O1 models
* Update README.md
---------
Co-authored-by: liushz <qq1791167085@163.com>
2024-09-18 22:41:17 +08:00
liushz
2e9db77d57
[Feature] Add custom model postprocess function ( #1519 )
...
Co-authored-by: liushz <liuhongwei@pjlab.rog.cn>
2024-09-18 14:40:51 +08:00
liushz
c9a7026f59
[Feature] Update MathBench & WikiBench for FullBench ( #1521 )
...
* Update MathBench & WikiBench for FullBench
* Update MathBench & WikiBench for FullBench
* Update GPQA & MMLU_Pro
* Update MathBench & WikiBench for FullBench
* Update MathBench & WikiBench for FullBench
* Update MathBench & WikiBench for FullBench
---------
Co-authored-by: liushz <liuhongwei@pjlab.rog.cn>
2024-09-18 14:35:30 +08:00
Songyang Zhang
cfbd308edf
[Doc] Update README ( #1528 )
...
* '
* Update
2024-09-14 16:02:17 +08:00
Linchen Xiao
90279b6461
[Feature] Dataset prompts update for ARC, BoolQ, Race ( #1527 )
2024-09-13 10:30:43 +08:00
Songyang Zhang
6997990c93
[Feature] Update Models ( #1518 )
...
* Update Models
* Update
* Update humanevalx
* Update
* Update
2024-09-12 23:35:30 +08:00
zhulinJulia24
3754dc1b67
update ( #1522 )
...
Co-authored-by: zhulin1 <zhulin1@pjlab.org.cn>
2024-09-12 15:00:52 +08:00
bittersweet1999
7c7fa36235
[Feature] add support for internal Followbench ( #1511 )
...
* fix pip version
* fix pip version
* add internal followbench
* add internal followbench
* fix lint
* fix lint
2024-09-11 13:32:34 +08:00
Linchen Xiao
317763381c
update ( #1517 )
2024-09-11 13:31:20 +08:00
bittersweet1999
c2bcd8725e
[Fix] Fix wildbench ( #1508 )
...
* fix pip version
* fix pip version
* fix_wildbench
2024-09-10 17:35:07 +08:00
Alexander Lam
a31a77c5c1
[Feature] Add SciCode summarizer config ( #1514 )
...
* [Feature] added SciCode summarizer config and dataset config for with background evaluation
* fix lint issues
* removed unnecessary type in summarizer group
2024-09-10 16:06:02 +08:00
Mo Li
5b93592242
[Fix] Fix link-check workflow by adjusting line breaks in URL ignore patterns ( #1507 )
...
* update link-check
* update link-check
* update link-check
2024-09-10 10:20:40 +08:00
Linchen Xiao
b5f8afb57b
[Bump] Bump version to 0.3.2.post1
2024-09-06 19:09:30 +08:00
Linchen Xiao
f04f3546bc
[Fix] Import fix ( #1500 )
2024-09-06 18:29:24 +08:00
Linchen Xiao
ff18545f0e
[Bump] Bump version to 0.3.2 ( #1497 )
2024-09-06 16:10:45 +08:00
Linchen Xiao
87ffa71d68
[Feature] Longbench dataset update
2024-09-06 15:50:12 +08:00
Albert Yan
928d0cfc3a
[Feature] Add support for Rendu API ( #1468 )
...
* Add support for Rendu API
* fix lint issue
* fix lint issue
* fix lint issue
* Update
---------
Co-authored-by: 13190 <zeyu.yan@transn.com>
Co-authored-by: tonysy <sy.zhangbuaa@gmail.com>
2024-09-06 01:00:43 +08:00
Hari Seldon
faf5260155
[Feature] Optimize Evaluation Speed of SciCode ( #1489 )
...
* update scicode
* update comments
* remove redundant variable
* Update
---------
Co-authored-by: tonysy <sy.zhangbuaa@gmail.com>
2024-09-06 00:59:41 +08:00
liushz
00fc8da5be
[Feature] Add model postprocess function ( #1484 )
...
* Add model postprocess function
* Add model postprocess function
* Add model postprocess function
* Add model postprocess function
* Add model postprocess function
* Add model postprocess function
* Add model postprocess function
* Add model postprocess function
---------
Co-authored-by: liushz <liuhongwei@pjlab.rog.cn>
2024-09-05 21:10:29 +08:00
Maxime SHE
45efdc994d
[Feature] Add an attribute api_key into TurboMindAPIModel default None ( #1475 )
...
Co-authored-by: Maxime <maximeshe@163.com>
Add an attribute api_key into TurboMindAPIModel default None then we can set the api_key while using lmdeploy to deploy the llm model
2024-09-05 17:51:16 +08:00
Linchen Xiao
6c9cd9a260
[Feature] Needlebench auto-download update ( #1480 )
...
* update
* update
* update
2024-09-05 17:22:42 +08:00
zhulinJulia24
716d46e1f5
[ci] fix badcase and add env info ( #1491 )
...
* update
* update
---------
Co-authored-by: zhulin1 <zhulin1@pjlab.org.cn>
2024-09-05 16:43:45 +08:00
zhulinJulia24
fb6a0df652
[ci] fix test env for vllm and add vllm baselines ( #1481 )
...
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
---------
Co-authored-by: zhulin1 <zhulin1@pjlab.org.cn>
2024-09-04 19:24:09 +08:00
Linchen Xiao
da74cbfa39
[Fix] Model configs update
2024-09-04 18:57:10 +08:00
Linchen Xiao
95aad6c282
[Fix] Requirements update
2024-09-03 18:50:40 +08:00
Linchen Xiao
9693be46b7
[Feature] Mmlu-pro auto-download ( #1464 )
...
* update
* update
* update
* update
* update
2024-08-30 10:03:40 +08:00
zhulinJulia24
f34209766d
[ci] fix test env ( #1470 )
...
* Update daily-run-test.yml
* Update daily-run-test.yml
* Update pr-run-test.yml
* Update daily-run-test.yml
* update
* update
---------
Co-authored-by: zhulin1 <zhulin1@pjlab.org.cn>
2024-08-29 14:48:17 +08:00
