Ma Zerun
6aaf3b91ec
[Feature] Support chat style inferencer. ( #643 )
...
* [Feature] Support chat style inferencer.
* [Fix] use new prompt
* [Fix] use new prompt
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
2023-11-30 14:00:06 +08:00
Yang Yong
522241a8c9
[Fix] Fix lightllmapi list bug ( #635 )
2023-11-24 14:24:13 +08:00
Songyang Zhang
5202456b4c
[API] Update API ( #624 )
...
* update api
* update generation_kwargs impl
* update api
* refactor
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-23 15:06:20 +08:00
Fengzhe Zhou
d4d1330a5a
[Sync] Fix cmnli, fix vicuna meta template, fix longbench postprocess and other minor fixes ( #625 )
2023-11-23 14:05:59 +08:00
Kevin Wang
c0785e53d8
[Feature] support download from modelscope ( #534 )
...
* [Feature] download from modelscope
* [Feature] download from modelscope
* minor fix
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
2023-11-22 15:32:21 +08:00
Songyang Zhang
721a45c68f
[Bug] Update api with generation_kargs ( #614 )
...
* update api
* update generation_kwargs impl
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-22 10:02:57 +08:00
Lyu Han
eb56fd6d16
Integrate turbomind python api ( #484 )
...
* integrate turbomind python api
* update
* update user guide
* update
* fix according to reviewer's comments
* fix error
* fix linting
* update user guide
* remove debug log
---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>
2023-11-21 22:34:46 +08:00
Songyang Zhang
d925748266
[Feature] Support 360API and FixKRetriever for CSQA dataset ( #601 )
...
* [Feature] Support 360API and FixKRetriever for CSQA dataset
* Update API
* Update API
* [Feature] Support 360API and FixKRetriever for CSQA dataset
* Update API
* Update API
* rm mathbench
* fix_lint
* Update opencompass/models/bytedance_api.py
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
* update
* update
* update
---------
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
2023-11-21 20:25:47 +08:00
Yang Yong
d3b0d5c4ce
[Feature] Support Lightllm API ( #613 )
...
* [Feature] Support Lightllm api
* formatting & renaming
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-21 19:18:40 +08:00
Yuan Feng
7199acc25d
Add support for DataCanvas Alaya LM ( #612 )
...
* Support for Alaya
* Remove useless requirements
2023-11-21 17:51:30 +08:00
Hubert
91fba2c2e9
[Feat] support humaneval and mbpp pass@k ( #598 )
...
* [Feat] support pass@ k
* [Feat] support pass@k
* [Feat] support pass@k
* [Feat] support pass@k
* [Feat] support pass@k
* [Feat] support pass@k docs
* update naming
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-16 21:22:06 +08:00
Fengzhe Zhou
d3de5c41fb
[Sync] update model configs ( #574 )
2023-11-13 15:15:34 +08:00
Songyang Zhang
9e42cb163b
[Feature] Update xunfei api ( #572 )
...
* update xunfei api
* fix lint
* avoid warning
2023-11-10 22:46:06 +08:00
Hubert
9f8a721313
[Fix] fix registry error with internal ( #551 )
...
* [Fix] fix conflict with internal
* [Fix] fix conflict with internal
2023-11-07 20:01:23 +08:00
Hubert
bb2ecf416e
[Feat] Support cibench ( #538 )
...
* [Feat] support cidataset
* [Feat] support cidataset
* [Feat] support cidataset
* [Feat] support cidataset
* minor fix
* minor fix
* minor fix
* minor fix
* minor fix
* minor fix
* rename cibench
* rename cibench
* rename cibench
* rename cibench
* minor fix
* minor fix
* minor fix
2023-11-07 19:11:44 +08:00
Songyang Zhang
239c2a346e
[Feature] Add support for MiniMax API ( #548 )
...
* update requirement
* update requirement
* update with minimax
* update api model
* Update readme
* fix error
---------
Co-authored-by: zhangsongyang <zhangsongyang@pjlab.org.cn>
2023-11-06 21:57:32 +08:00
Hubert
1ccdfaa623
[Feat] support xunfei api ( #547 )
2023-11-06 19:29:26 +08:00
Fengzhe Zhou
dbb20b8270
[Sync] update ( #517 )
2023-10-27 20:31:22 +08:00
Fengzhe Zhou
df07391ed8
[Fix] Enforce do_sample=False in HF model ( #506 )
...
* update hf model wrapper
* patch llama
---------
Co-authored-by: bot <bot@bot.com>
2023-10-27 16:54:19 +08:00
Fengzhe Zhou
e3d4901bed
[Feat] Add _set_model_kwargs_torch_dtype for HF model ( #507 )
...
* add _set_model_kwargs_torch_dtype for hf models
* add logger
2023-10-27 11:45:41 +08:00
Hubert
b3f5d9e421
[Feat] support math/gms8k agent config ( #494 )
...
* support math agent
* support gsm8k agent
* support gsm8k agent
* minor fix
* minor fix
* minor fix
* Update configs/eval_codeagent.py
2023-10-25 23:05:15 +08:00
Hubert
ac3a2c4501
[Feat] local api speed up with fixed concurrent users ( #497 )
...
* [Feat] local api speed up
* fix lint
* fix lint
* minor fix
* add example api
2023-10-25 21:12:20 +08:00
Lyu Han
6738247142
Integrate turbomind inference via its RPC API instead of its python API ( #414 )
...
* support tis
* integrate turbomind inference via its RPC API instead of its python API
* update guide
* update ip address spec
* update according to reviewer's comments
2023-10-07 10:27:48 +08:00
philipwangOvO
3bb3d330eb
[Sync] Update LongEval ( #443 )
2023-09-27 16:32:40 +08:00
Ma Zerun
0f2c388280
Support GSM8k evaluation with tools by Lagent and LangChain ( #277 )
...
* Support GSM8k evaluation with tools by Lagent and LangChain
* Avoid to use MMEngine new feature
* update document
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-09-22 15:28:22 +08:00
Tong Gao
681d3013de
[Feature] Log gold answer in prediction output ( #419 )
...
* [Feature] Log gold answer in prediction output
* support clp golden ans
* minor fix
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
2023-09-22 12:44:40 +08:00
Leymore
49c467458f
[Feature] Update llama2 ( #372 )
2023-09-08 12:47:56 +08:00
Tong Gao
b11838f80a
[Feature] Update claude2 postprocessor ( #365 )
...
* [Feature] Update claude2 config
* [Feature] Update claude2 postprocessor
2023-09-07 11:26:26 +08:00
Leymore
e810974068
[Fix] Fix when missing both pad and eos token ( #287 )
...
* fix when missing both pad and eos token
* update pad_token_id impl
2023-08-31 16:53:39 +08:00
Tong Gao
9058be07b8
[Feature] Simplify entry script ( #204 )
...
* [Feature] Simply entry script
* update
2023-08-25 17:36:30 +08:00
Tong Gao
f480b72703
[Feature] Support model-bound prediction postprocessor, use it in Claude ( #268 )
...
* [Feature] Support model-bound text postprocessor, add claude as an example
* update
* update
* minor fix
---------
Co-authored-by: zhoufengzhe <zhoufengzhe@pjlab.org.cn>
2023-08-25 16:12:21 +08:00
Tong Gao
60c2d3d76b
[Feature] Add Claude support ( #253 )
...
* [Feature] Add Claude support
* [Feature] Add Claude support
* Update opencompass/models/claude_api.py
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
* raise import erorr
---------
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
2023-08-24 14:29:45 +08:00
LZHgrla
77745a84ea
[Fix] Fix bugs for PeftModel generate ( #252 )
...
* fix bugs
* fix typo
2023-08-24 14:07:33 +08:00
Leymore
ff5ab92331
[Feature] Add llama2 native implements ( #235 )
...
* add llama2 native implements
* rename configs/eval_llama_7b.py
---------
Co-authored-by: zhoufengzhe <zhoufengzhe@pjlab.org.cn>
2023-08-23 11:33:25 +08:00
philipwangOvO
3b29aaee2b
[Fix] bin_trim ( #237 )
...
Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>
2023-08-21 15:44:49 +08:00
philipwangOvO
655a807f4b
[Dataset] LongBench ( #236 )
...
Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>
2023-08-21 14:15:20 +08:00
Zaida Zhou
f256abffd3
[Enhancement] Skip invalid keys to avoid requesting API ( #184 )
...
* Skip invalid keys to avoid requesting API
* get expected key
* print warning info
2023-08-10 18:41:43 +08:00
Tong Gao
312095de9d
[Fix] meta template & unit tests ( #170 )
2023-08-10 16:49:13 +08:00
Songyang Zhang
3f36db3b06
[Feature] Support turbomind ( #166 )
...
* support turbomind
* update doc
* Update docs/en/advanced_guides/evaluation_turbomind.md
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
* Update docs/zh_cn/advanced_guides/evaluation_turbomind.md
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
* Update docs/zh_cn/advanced_guides/evaluation_turbomind.md
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
* Update docs/en/advanced_guides/evaluation_turbomind.md
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
* update
---------
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
2023-08-10 16:25:11 +08:00
Haodong Duan
d5d4f47371
[API] Refine OpenAI ( #175 )
2023-08-09 12:38:57 +08:00
Zaida Zhou
af436f5951
[Feature] Calculate max_out_len without hard code for OpenAI model ( #158 )
...
* calulate max_out_len without hard code
* set default value
* update configs
* Update configs/eval_gpt3.5.py
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
---------
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
2023-08-08 15:16:56 +08:00
Tong Gao
bbdedc6c95
[Enhancement] Optimize OpenAI models ( #128 )
...
* [Feature] Enhance OpenAI API, add example config for GPT evaluation
2023-08-03 14:55:16 +08:00
Leymore
d862f570aa
[Feature] Add SC ( #126 )
...
* add self-consistency
* add CoT method Self-Consistency
* fix typo error and update openicl_eval
* add tydiQA-GoldP task
* fix sc
* rename gsm8k_sc
* fix sc
* add self-consistency doc
* refine sc
---------
Authored-by: liushz <qq1791167085@163.com>
2023-07-28 17:29:37 +08:00
Haodong Duan
46c9645753
[Feature] Allow explicitly setting the temperature for API model ( #121 )
...
* allow explicitly setting the temperature
* update
2023-07-28 11:28:15 +08:00
gowithme
57fcfc975a
[Feature] Support intern lanuage model ( #51 )
...
* support internLM
* support internLM
* simplify intern model files
* update storage_manager
* support internLM
* Modify the file organization structure
* support internLM
* support internLM
* support internLM
* support internLM
* change some details
2023-07-27 18:49:36 +08:00
Leymore
eea8b04417
[Feature] Add llama-2 models ( #81 )
...
* add llama-2 models
* update docs
---------
Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>
2023-07-19 19:51:29 +08:00
LZH
26e2f171f4
[Feature] Support load PEFT adapter for HuggingFace model ( #74 )
...
* support peft for HuggingFace model
* add docstring
2023-07-18 16:21:43 +08:00
Tong Gao
1e44541730
[Enhancement] Test linting in CI and fix existing linting errors ( #69 )
...
* [Enhancement] Test linting in CI
* fix linting
2023-07-17 15:59:10 +08:00
Tong Gao
7ee5a86fee
[Feature] Enhance OpenAI API, add example config for GPT evaluation ( #53 )
...
* [Feature] Enhance OpenAI API, add example config for GPT evaluation
* fix
2023-07-12 16:43:46 +08:00
mzr1996
04dd01a235
Update configs and code
2023-07-05 11:45:08 +08:00
