jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
503 Commits 11 Branches 35 Tags 66 MiB
41196c48ae
Commit Graph

93 Commits

Author SHA1 Message Date
Ke Bao
81d0e4d793
[Feature] Add lmdeploy tis python backend model (#1014) 
* add lmdeploy tis python backend model

* fix pr check

* update
 2024-04-23 14:27:11 +08:00
Fengzhe Zhou
8c85edd1cd
[Sync] deprecate old mbpps (#1064) 2024-04-19 20:49:46 +08:00
Robin Chen
c172401323
[Fix] Fixed repeated loading of VLLM (#1051) 
* [fix]Fixed the issue caused by the repeated loading of VLLM model during task segmentation.

* [fix] avoid TypeError: VLLM.__init__() got an unexpected keyword argument 'tokenizer_only'

* restore .pre-commit-config.yaml

* restore opencompass/tasks/openicl_infer.py

---------

Co-authored-by: IcyFeather <mengzhuo.happy@gmail.com>
Co-authored-by: Leymore <zfz-960727@163.com>
 2024-04-17 20:36:08 +08:00
Fengzhe Zhou
7a41951dda
[Fix] logger.error -> logger.debug in OpenAI wrapper (#1050) 
* logger.error -> logger.info in OpenAI

* logger.info -> logger.debug in OpenAI
 2024-04-15 21:08:13 +08:00
Fengzhe Zhou
b39f501563
[Sync] update taco (#1030) 2024-04-09 17:50:23 +08:00
bittersweet1999
02e7eec911
[Feature] Support AlpacaEval_V2 (#1006) 
* support alpacaeval_v2

* support alpacaeval

* update docs

* update docs
 2024-03-28 16:49:04 +08:00
Ke Bao
e415ddf96a
[Fix] Fix turbomind_tis (#992) 2024-03-22 15:50:12 +08:00
Fengzhe Zhou
bdd85358cc
[Sync] update 20240308 (#953) 2024-03-11 22:34:19 +08:00
Yang Yong
3829be87b1
Fix LightllmApi ppl test (#951) 2024-03-08 12:04:44 +08:00
Yang Yong
107e022cf4
Support prompt template for LightllmApi. Update LightllmApi token bucket. (#945) 2024-03-06 15:33:53 +08:00
RunningLeon
c54a5d3b0f
Support get_ppl for TurbomindModel (#878) 
* update ppl for turbomindmodel

* update api_server

* rename config and set thread_safe for pytorch engine if possible
 2024-03-06 11:44:19 +08:00
Fengzhe Zhou
b03d5dc531
[Sync] Sync Internal (#941) 2024-03-04 14:42:36 +08:00
bittersweet1999
001e77fea2
[Feature] add support for gemini (#931) 
* add gemini

* add gemini

* add gemini
 2024-02-28 19:38:34 +08:00
RunningLeon
32ba0b074e
Support lmdeploy pytorch engine (#875) 
* add lmdeploy pytorch model

* fix

* speed up encoding and decoding

* fix

* change tokenizer
 2024-02-22 03:46:07 -03:00
Yang Yong
b6e21ece38
Support LightllmApi input_format (#888) 2024-02-19 10:02:59 +08:00
Fengzhe Zhou
d34ba11106
[Sync] Merge branch 'dev' into zfz/update-keyset-demo (#876) 2024-02-05 23:29:10 +08:00
RunningLeon
4c87e777d8
[Feature] Add end_str for turbomind (#859) 
* fix

* update

* fix internlm1

* fix docs

* remove sys
 2024-02-01 22:31:14 +08:00
Songyang Zhang
cdca59ff49
[Fix] Update Zhipu API and Fix issue min_out_len issue of API models (#847) 
* Update zhipu api and fix min_out_len issue of API class

* Update example

* Update example
 2024-01-28 14:52:43 +08:00
Songyang Zhang
8ed022b4c4
Update Sensetime API (#844) 2024-01-26 16:40:49 +08:00
Hubert
4aa74565e2
[Feat] minor update agent related (#839) 
* [Feat] update cibench

* [Feat] Support CIBench

* [Feat] Support CIBench

* [Feat] Support CIBench

* [Feat] Support CIBench
 2024-01-26 14:15:51 +08:00
Songyang Zhang
793e32c9cc
[Feature] Update API implementation (#834) 2024-01-24 13:35:21 +08:00
Yang Yong
f09a2ff418
Add LightllmApi KeyError log & Update doc (#816) 
* Add LightllmApi KeyError log

* Update LightllmApi doc
 2024-01-18 22:23:38 +08:00
RunningLeon
61fe873c89
[Fix] Fix turbomind and update docs (#808) 
* update

* update docs

* add engine_config and gen_config in eval_config

* update

* fix

* fix

* fix

* fix docstr

* fix url
 2024-01-18 14:41:35 +08:00
Fengzhe Zhou
b4afe3e7c1
[Sync] Add InternLM2 Keyset Evaluation Demo (#807) 
Co-authored-by: zhangyifan1 <zhangyifan1@pjlab.org.cn>
 2024-01-17 13:48:12 +08:00
RunningLeon
0836aec67b
[Feature] Update evaluate turbomind (#804) 
* update

* fix

* fix

* fix
 2024-01-17 11:09:50 +08:00
notoschord
d3a0ddc3ef
[Feature] Add support for Nanbeige API (#786) 
Co-authored-by: notoschord <wangzekai@kanzhun.com>
 2024-01-11 13:54:27 +08:00
bittersweet1999
5679edb490
add temperature in alles (#787) 2024-01-11 03:57:24 +00:00
Fengzhe Zhou
32f40a8f83
[Sync] Sync with internal codes 2023.01.08 (#777) 2024-01-08 14:07:24 +00:00
jiangjin1999
8194199d79
[Feature] *_batch_generate* function, add the MultiTokenEOSCriteria (#772) 
* jiangjin1999: in the _batch_generate function, add the MultiTokenEOSCriteria feature to speed up inference.

* jiangjin1999: in the _batch_generate function, add the MultiTokenEOSCriteria feature to speed up inference.

---------

Co-authored-by: jiangjin08 <jiangjin08@MBP-2F32S5MD6P-0029.local>
Co-authored-by: jiangjin08 <jiangjin08@a.sh.vip.dianping.com>
 2024-01-08 16:40:02 +08:00
Chris Liu
3eb225a5e6
[Feature] Support LLaMA2-Accessory (#732) 
* Support LLaMA2-Accessory

* remove strip

* clear imports

* reformat

* fix lint

* fix lint

* update readme

* update readme

* update readme

* update readme
 2024-01-02 20:48:51 +08:00
HUANG Fei
ba027eeeac
[Feature] Add support of qwen api (#735) 2024-01-02 20:47:12 +08:00
Hubert
327951087f
[Feat] update code config (#749) 
* [Feat] update code dataset

* [Feat] update code dataset

* [Feat] update code dataset
 2023-12-29 18:46:34 +08:00
Yang Yong
54345c56b7
Update LightllmApi and Fix mmlu bug (#738) 
* Update LightllmApi and Fix mmlu bug

* checkout mmlu_gen_a484b3.py

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-12-27 13:49:08 +08:00
AllentDan
336d8d76ff
add turbomind restful api support (#693) 
* add turbomind restful api support

* config

* top_p 0.8

* top_k = 1
 2023-12-24 01:40:00 +08:00
RunningLeon
e34c552282
[Feature] Update configs for evaluating chat models like qwen, baichuan, llama2 using turbomind backend (#721) 
* add llama2 test

* fix

* test qwen chat-7b

* test w4

* add baichuan2

* update

* update

* update configs and docs

* update
 2023-12-21 18:22:17 +08:00
Hubert
eda72e756e
[Fix] minor fix openai (#711) 2023-12-18 15:45:31 +08:00
Songyang Zhang
637628a70f
[Doc] Update Doc for Alignbench (#707) 
* update alignmentbench

* update alignmentbench

* update doc

* update

* update
 2023-12-15 15:07:25 +08:00
Songyang Zhang
bfe4aa2af5
[Fix] Update alignmentbench (#704) 
* update alignmentbench

* update alignmentbench

* update alignmentbench
 2023-12-14 18:24:21 +08:00
Hubert
e78857ac36
[Sync] minor test (#683) 2023-12-11 17:42:53 +08:00
Songyang Zhang
e25c5f9525
[Enhancement] Update API Interface and Mixtral (#681) 
* [Enhancement] Update API interface

* [Enhancement] Update API interface

* Update mixtral

* Update readme
 2023-12-10 13:29:26 +08:00
Hubert
9eb5cadcac
[Feat] update gsm8k and math agent config (#652) 
* [Feat] update gsm8k and math agent config

* minor fix
 2023-12-01 15:08:38 +08:00
Ma Zerun
6aaf3b91ec
[Feature] Support chat style inferencer. (#643) 
* [Feature] Support chat style inferencer.

* [Fix] use new prompt

* [Fix] use new prompt

---------

Co-authored-by: yingfhu <yingfhu@gmail.com>
 2023-11-30 14:00:06 +08:00
Yang Yong
522241a8c9
[Fix] Fix lightllmapi list bug (#635) 2023-11-24 14:24:13 +08:00
Songyang Zhang
5202456b4c
[API] Update API (#624) 
* update api

* update generation_kwargs impl

* update api

* refactor

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-11-23 15:06:20 +08:00
Fengzhe Zhou
d4d1330a5a
[Sync] Fix cmnli, fix vicuna meta template, fix longbench postprocess and other minor fixes (#625) 2023-11-23 14:05:59 +08:00
Kevin Wang
c0785e53d8
[Feature] support download from modelscope (#534) 
* [Feature] download from modelscope

* [Feature] download from modelscope

* minor fix

---------

Co-authored-by: yingfhu <yingfhu@gmail.com>
 2023-11-22 15:32:21 +08:00
Songyang Zhang
721a45c68f
[Bug] Update api with generation_kargs (#614) 
* update api

* update generation_kwargs impl

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-11-22 10:02:57 +08:00
Lyu Han
eb56fd6d16
Integrate turbomind python api (#484) 
* integrate turbomind python api

* update

* update user guide

* update

* fix according to reviewer's comments

* fix error

* fix linting

* update user guide

* remove debug log

---------

Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>
 2023-11-21 22:34:46 +08:00
Songyang Zhang
d925748266
[Feature] Support 360API and FixKRetriever for CSQA dataset (#601) 
* [Feature] Support 360API and FixKRetriever for CSQA dataset

* Update API

* Update API

* [Feature] Support 360API and FixKRetriever for CSQA dataset

* Update API

* Update API

* rm mathbench

* fix_lint

* Update opencompass/models/bytedance_api.py

Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>

* update

* update

* update

---------

Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
 2023-11-21 20:25:47 +08:00
Yang Yong
d3b0d5c4ce
[Feature] Support Lightllm API (#613) 
* [Feature] Support Lightllm api

* formatting & renaming

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-11-21 19:18:40 +08:00