Fengzhe Zhou
d6aaac22e7
[Feature] Update cmb ( #571 )
2023-11-13 00:09:05 +08:00
Songyang Zhang
9e42cb163b
[Feature] Update xunfei api ( #572 )
...
* update xunfei api
* fix lint
* avoid warning
2023-11-10 22:46:06 +08:00
jingmingzhuo
b3cbef3226
[Feature] Add py150 and maxmin ( #562 )
...
* [feat] add clozeTesst_maxmin dataset
* [feat] add py150 datasets
* [feat] change __init__.py in opencompass/datasets
* [fix] pre-commit check
* [fix] rename py150 and masxmin datasets in configs
* [feat] add gen.py of py150 and maxmin in configs/datasets
2023-11-09 22:05:25 +08:00
Hubert
889a6b26ae
[Fix] fix log re-direct ( #564 )
2023-11-09 19:34:19 +08:00
Hubert
cf5a6d1ab7
[Fix] fix unnecessary import and update requirements ( #555 )
2023-11-08 17:58:49 +08:00
Hubert
9f8a721313
[Fix] fix registry error with internal ( #551 )
...
* [Fix] fix conflict with internal
* [Fix] fix conflict with internal
2023-11-07 20:01:23 +08:00
Hubert
bb2ecf416e
[Feat] Support cibench ( #538 )
...
* [Feat] support cidataset
* [Feat] support cidataset
* [Feat] support cidataset
* [Feat] support cidataset
* minor fix
* minor fix
* minor fix
* minor fix
* minor fix
* minor fix
* rename cibench
* rename cibench
* rename cibench
* rename cibench
* minor fix
* minor fix
* minor fix
2023-11-07 19:11:44 +08:00
Songyang Zhang
239c2a346e
[Feature] Add support for MiniMax API ( #548 )
...
* update requirement
* update requirement
* update with minimax
* update api model
* Update readme
* fix error
---------
Co-authored-by: zhangsongyang <zhangsongyang@pjlab.org.cn>
2023-11-06 21:57:32 +08:00
Hubert
1ccdfaa623
[Feat] support xunfei api ( #547 )
2023-11-06 19:29:26 +08:00
Yuan Liu
6e31520128
[Feature]: To be compatible with the latest version of MiniGPT-4 ( #539 )
...
* [Feature]: To be compatible with the latest version of MiniGPT-4
* [Feature]: User try and except
Co-authored-by: Fengzhe Zhou <zfz-960727@163.com>
* [Fix]: Fix lint
---------
Co-authored-by: bensenliu <bensenliu@tencent.com>
Co-authored-by: Fengzhe Zhou <zfz-960727@163.com>
2023-11-04 09:50:36 +08:00
bittersweet1999
f25a980043
[fFeat] Add an opensource dataset Tabmwp ( #505 )
...
* TabMWP
* TabMWP
* fixed
* fixed
* fixed
* done
* done
* done
---------
Co-authored-by: caomaosong <caomaosong@pjlab.org.cn>
2023-11-03 11:15:46 +08:00
Hubert
b9270c3a60
[Fix] Fix local debug mode not restrict the resources ( #522 )
...
* [Fix] fix local debug mode not restrict the resources
* minor fix
2023-10-30 18:13:43 +08:00
Qing
e2355a2ede
[Feature] Add multi model viz ( #509 )
...
* add viz_multi_model.py tool
* Modify the viz_multi_model.py script according to the review
* highlight multiple optimal scores
---------
Co-authored-by: wq.chu <wq.chu@tianrang-inc.com>
Co-authored-by: Leymore <zfz-960727@163.com>
2023-10-30 12:11:33 +08:00
Fengzhe Zhou
6a398d171c
Bump version to 0.1.7 ( #518 )
2023-10-27 20:32:27 +08:00
Fengzhe Zhou
dbb20b8270
[Sync] update ( #517 )
2023-10-27 20:31:22 +08:00
Hubert
6f07af3039
[Feat] Support local runner for windows ( #515 )
2023-10-27 17:16:22 +08:00
Fengzhe Zhou
df07391ed8
[Fix] Enforce do_sample=False in HF model ( #506 )
...
* update hf model wrapper
* patch llama
---------
Co-authored-by: bot <bot@bot.com>
2023-10-27 16:54:19 +08:00
Wei Jueqi
b62842335d
[Doc] Update Subjective docs ( #510 )
...
* rename
* add en subdoc
* fix name
* fix writing
* update
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-10-27 16:27:24 +08:00
Fengzhe Zhou
e3d4901bed
[Feat] Add _set_model_kwargs_torch_dtype for HF model ( #507 )
...
* add _set_model_kwargs_torch_dtype for hf models
* add logger
2023-10-27 11:45:41 +08:00
Fengzhe Zhou
6405cd2db5
use example summarizer by default ( #508 )
2023-10-27 11:45:29 +08:00
Hubert
b3f5d9e421
[Feat] support math/gms8k agent config ( #494 )
...
* support math agent
* support gsm8k agent
* support gsm8k agent
* minor fix
* minor fix
* minor fix
* Update configs/eval_codeagent.py
2023-10-25 23:05:15 +08:00
Hubert
ac3a2c4501
[Feat] local api speed up with fixed concurrent users ( #497 )
...
* [Feat] local api speed up
* fix lint
* fix lint
* minor fix
* add example api
2023-10-25 21:12:20 +08:00
Leymore
4dd9a3fc10
[Sync] sync with internal codes 20231019 ( #488 )
2023-10-18 23:37:35 -05:00
liushz
2737249f31
[Feature] Add mathbench dataset and circular evaluator ( #408 )
...
* add_mathbench
* update mathbench
* support non circular eval dataset
---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
Co-authored-by: yingfhu <yingfhu@gmail.com>
2023-10-18 04:08:31 -05:00
Leymore
fccfcb6f5b
fix summary default ( #483 )
2023-10-17 11:32:38 +08:00
Leymore
6317da08b3
Bump version to 0.1.6 ( #478 )
2023-10-13 06:54:51 -05:00
Leymore
7d9e386821
[Fix] Split if and only if complete eos string shows up ( #477 )
2023-10-13 06:52:20 -05:00
Leymore
861942ab1b
[Feature] Add lawbench ( #460 )
...
* add lawbench
* update requirements
* update
2023-10-13 06:51:36 -05:00
Leymore
fbf5089c40
[Sync] update github token ( #475 )
2023-10-13 06:50:54 -05:00
Leymore
362c33dff4
fix jieba rouge ( #467 )
2023-10-12 10:25:19 +08:00
Leymore
d7ff933a73
[Fix] Use jieba rouge in lcsts ( #459 )
...
* use jieba rouge in lcsts
* use rouge_chinese
2023-10-09 10:10:33 +08:00
Tong Gao
119bfd1569
[Refactor] Move fix_id_list to Retriever ( #442 )
...
* [Refactor] Move fix_id_list to Retriever
* update
* move to base
* fix
2023-10-07 12:53:41 +08:00
Lyu Han
6738247142
Integrate turbomind inference via its RPC API instead of its python API ( #414 )
...
* support tis
* integrate turbomind inference via its RPC API instead of its python API
* update guide
* update ip address spec
* update according to reviewer's comments
2023-10-07 10:27:48 +08:00
Leymore
9db5652638
[Feature] re-implement ceval load dataset ( #446 )
2023-09-27 21:18:48 +08:00
Hubert
d9f3e88dfe
[Fix] fix clp potential error and support bs>1 ( #439 )
...
* [Fix] fix clp potential error and support bs>1
* [Fix] fix clp potential error and support bs>1
* minor fix
* minor fix
2023-09-27 16:32:57 +08:00
philipwangOvO
3bb3d330eb
[Sync] Update LongEval ( #443 )
2023-09-27 16:32:40 +08:00
Tong Gao
9b21613d17
Bump version to 0.1.5 ( #432 )
2023-09-22 19:17:23 +08:00
chenbohua3
b2926eac8f
[Feature] support customize config path ( #423 )
...
* support customize config path
* support customize config path
* support customize config path
2023-09-22 19:12:02 +08:00
liushz
c5224c2a91
[Feature] Add kaoshi dataset ( #392 )
...
* Add ToT method
* Update ToT
* Update ToT
* Update ToT
* Update ToT
* Update ToT
* Add Koashi
* Update Kaoshi
* Update Kaoshi
* Update kaoshi
* Update kaoshi
* Update Kaoshi
* Update Kaoshi
* Update Kaoshi
* Update Kaoshi
* update Kaoshi
* update
* update
* fix
---------
Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>
2023-09-22 18:46:33 +08:00
TTTTTiam
2a62bea1a4
add evaluation of scibench ( #393 )
...
* add evaluation of scibench
* add evaluation of scibench
* update scibench
* remove scibench evaluator
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-09-22 17:42:08 +08:00
Tong Gao
07574fddbb
[Fix] keep keys ( #431 )
2023-09-22 17:30:54 +08:00
Tong Gao
a1ea3c094a
[Sync] Initial support of subjective evaluation ( #421 )
...
Co-authored-by: Leymore <zfz-960727@163.com>
2023-09-22 15:42:31 +08:00
Ma Zerun
0f2c388280
Support GSM8k evaluation with tools by Lagent and LangChain ( #277 )
...
* Support GSM8k evaluation with tools by Lagent and LangChain
* Avoid to use MMEngine new feature
* update document
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-09-22 15:28:22 +08:00
Tong Gao
681d3013de
[Feature] Log gold answer in prediction output ( #419 )
...
* [Feature] Log gold answer in prediction output
* support clp golden ans
* minor fix
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
2023-09-22 12:44:40 +08:00
Yike Yuan
97fdc51102
[Fix] Fix performance issue of visualglm. ( #424 )
...
* [Fix] Visualglm performance fixed.
* [Fix] Hide ckpt path.
2023-09-21 19:54:23 +08:00
Hubert
8803f7f7a6
[Feat] support antropics evals dataset ( #422 )
...
* [Feat] support anthropics ai risk dataset
* [Feat] support anthropics evals dataset
* [Feat] support anthropics evals dataset
2023-09-20 18:36:44 +08:00
Leymore
ae0cd8752f
[Feature] Use local accuracy from hf implements ( #416 )
...
* use local accuracy from hf implements
* add load from hf fallback
2023-09-20 16:35:22 +08:00
Zequn Liu
ff2c15a09f
[fix] summarizer debug logger ( #417 )
2023-09-20 15:29:26 +08:00
Yike Yuan
bd50bad8b5
[Feat] Support mm models on public dataset and fix several issues. ( #412 )
...
* [Feat] Add public dataset support for visualglm, qwenvl, and flamingo
* [Fix] MMBench related changes.
* [Fix] Openflamingo inference.
* [Fix] Hide ckpt path.
* [Fix] Pre-commit.
---------
Co-authored-by: Haodong Duan <dhd.efz@gmail.com>
2023-09-19 19:08:44 +08:00
Yuanhan Zhang
7c2726c23b
[Model] Yhzhang/add mlugowl llamaadapter ( #405 )
...
* refine gitignore
* [Feature]: Add minigpt-4
* [Feature]: Add mm local runner
* [Feature]: Add instructblip
* add otter and llama-adapter
* add owl
* add llama2-adapter and owl
* lint
* [Feature]: Add minigpt-4
* [Feature]: Add instructblip
* add otter and llama-adapter
* add owl
* add llama2-adapter and owl
* lint
* lint
* update
* lint
* lint
* add __init__.py
* update
* update
* update
* update
* [Feature]: Add minigpt-4
* [Feature]: Add mm local runner
* [Feature]: Add instructblip
* add otter and llama-adapter
* add owl
* add llama2-adapter and owl
* lint
* [Feature]: Add minigpt-4
* [Feature]: Add instructblip
* add otter and llama-adapter
* add owl
* add llama2-adapter and owl
* lint
* lint
* update
* lint
* lint
* add __init__.py
* update
* update
* update
* update
* optimize mmbench dataset args
* update
* update
* run commit hook
---------
Co-authored-by: liuyuan <3463423099@qq.com>
Co-authored-by: kennymckormick <dhd@pku.edu.cn>
Co-authored-by: kennymckormick <dhd.efz@gmail.com>
2023-09-19 14:21:26 +08:00
