jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
855 Commits 11 Branches 35 Tags 66 MiB
fd6fbf01a2
Commit Graph

31 Commits

Author SHA1 Message Date
Myhs_phz
68a9838907
[Feature] Add list of supported datasets at html page (#1850) 
* feat dataset-index.yml and stat.py

* fix

* fix

* fix

* feat url of paper and config file

* doc all supported dataset list

* docs zh and en

* docs README zh and en

* docs new_dataset

* docs new_dataset
 2025-02-14 16:17:30 +08:00
Lyu Han
4fde41036f
[Feature] Update TurboMindModel by integrating lmdeploy pipeline API (#1556) 
* integrate lmdeploy's pipeline api

* fix linting

* update user guide

* rename

* update

* update

* update

* rollback class name

* update

* remove unused code

* update

* update

* use pipeline

* fix ci check

* compatibility

* compatibility

* remove concurrency

* update

* fix table content

* update
 2024-10-14 15:33:40 +08:00
liushz
e076dc5acf
[Fix] Fix openai api tiktoken bug for api server (#1433) 
* Fix openai api tiktoken

* Fix openai api tiktoken

---------

Co-authored-by: liushz <liuhongwei@pjlab.rog.cn>
 2024-08-20 22:02:14 +08:00
liushz
e5ee1647fb
Add doc for accelerator function (#1252) 
* Add Math Evaluation with Judge Model Evaluator

* Add Math Evaluation with Judge Model Evaluator

* Add Math Evaluation with Judge Model Evaluator

* Add Math Evaluation with Judge Model Evaluator

* Fix Llama-3 meta template

* Fix MATH with JudgeLM Evaluation

* Fix MATH with JudgeLM Evaluation

* Fix MATH with JudgeLM Evaluation

* Fix MATH with JudgeLM Evaluation

* Update acclerator

* Update MathBench

* Update accelerator

* Add Doc for accelerator

* Add Doc for accelerator

* Add Doc for accelerator

* Add Doc for accelerator

---------

Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
 2024-06-24 14:53:51 +08:00
Songyang Zhang
063f5f5f49
[Update] Update performance of common benchmarks (#1109) 
* [Update] Update performance of common benchmarks

* [Update] Update performance of common benchmarks

* [Update] Update performance of common benchmarks
 2024-04-30 00:09:08 +08:00
Haodong Duan
3a232db471
[Deperecate] Remove multi-modal related stuff (#1072) 
* Remove MultiModal

* update index.rst

* update README

* remove mmbench codes

* update news

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2024-04-26 21:20:14 +08:00
Mo Li
acae560911
Added support for multi-needle testing in needle-in-a-haystack test (#802) 
* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py

Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

* add English version support

* change NeedleInAHaystackDataset to dynamic loading

* change NeedleInAHaystackDataset to dynamic loading

* fix needleinahaystack test eval bug

* fix needleinahaystack config bug

* Added support for multi-needle testing in needle-in-a-haystack test

* Optimize the code for plotting in the needle-in-a-haystack test.

* Correct the typo in the dataset parameters.

* update needleinahaystack test docs

---------

Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>
 2024-01-17 13:47:34 +08:00
Fengzhe Zhou
3a68083ecc
[Sync] update configs (#734) 2023-12-25 21:59:16 +08:00
liyucheng09
05bbce8b08
[Feature] Add Data Contamination Analysis (#639) 
* add contamination analysis to ceval

* fix bugs

* add contamination docs

* to pass CI check

* update

---------

Co-authored-by: zhangyifan1 <zhangyifan1@pjlab.org.cn>
Co-authored-by: Leymore <zfz-960727@163.com>
 2023-12-08 10:00:11 +08:00
Fengzhe Zhou
d949e3c003
[Feature] Add circular eval (#610) 
* refactor default, add circular summarizer

* add circular

* update impl

* update doc

* minor update

* no more to be added
 2023-11-23 16:45:47 +08:00
Hubert
8c1483e3ce
[Docs] update ds1000 code eval docs (#618) 2023-11-22 13:37:53 +08:00
Yang Yong
d3b0d5c4ce
[Feature] Support Lightllm API (#613) 
* [Feature] Support Lightllm api

* formatting & renaming

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-11-21 19:18:40 +08:00
Songyang Zhang
239c2a346e
[Feature] Add support for MiniMax API (#548) 
* update requirement

* update requirement

* update with minimax

* update api model

* Update readme

* fix error

---------

Co-authored-by: zhangsongyang <zhangsongyang@pjlab.org.cn>
 2023-11-06 21:57:32 +08:00
Wei Jueqi
b62842335d
[Doc] Update Subjective docs (#510) 
* rename

* add en subdoc

* fix name

* fix writing

* update

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-10-27 16:27:24 +08:00
Tong Gao
767c12a660
[Docs] update get_started (#435) 
* [Docs] update get_started

* [Docs] Refactor get_started

* update

* add zh FAQ

* add cn doc

* update

* fix dead links

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-10-07 11:49:40 +08:00
Yixiao Fang
524579b5af
[Docs] Add multimodal docs (#334) 
* add multimodal docs

* fix lint

---------

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
 2023-09-22 18:58:29 +08:00
philipwangOvO
f57c0702f7
[Docs] Readme in longeval (#389) 
* [Docs] Readme in longeval

* [Docs] Readme in longeval

* [Docs] Readme in longeval

* [Docs] Readme in longeval

* [Docs] Readme in longeval

* [Docs] Readme in longeval

* [Docs] Readme in longeval
 2023-09-18 17:06:00 +08:00
Hubert
a11cb45c83
[Feat] implementation for support promptbench (#239) 
* [Feat] support adv_glue dataset for adversarial robustness

* reorg files

* minor fix

* minor fix

* support prompt bench demo

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix
 2023-09-15 15:06:53 +08:00
Tong Gao
4d89533fbc
[Docs] Add FAQ (#384) 
* [Docs] Add FAQ

* [Docs] Add FAQ
 2023-09-12 12:11:38 +08:00
Songyang Zhang
a05daab911
[Doc] Update Overview (#242) 
* Update news

* update overview

* add framework

* update index

* update

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-09-07 14:21:39 +08:00
Leymore
c0e58632ca
[Doc] Add summarizer doc (#231) 
* add summarizer doc

* update

* update doc

* Apply suggestions from code review

---------

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
 2023-08-23 11:18:01 +08:00
Ezra-Yu
17ccaa5980
[Feat] Add codegeex2 and Humanevalx (#210) 
* add codegeex2

* add humanevalx dataset

* add evaluator

* update evaluator

* update configs

* update clean code

* update configs

* fix lint

* remove sleep

* fix lint

* update docs

* fix lint
 2023-08-17 11:03:16 +08:00
Tong Gao
e464265cf8
[Docs] Update contribution guide & toc, improve user experience (#188) 
* [Docs] Update contribution guide & toc

* update

* Update docs/en/notes/contribution_guide.md

Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* update

* update

---------

Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>
 2023-08-11 11:36:09 +08:00
liushz
ed248af136
[Fix] Fix some sc errors (#177) 
* Update sc

* Update sc doc

* Apply suggestions from code review

Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>

---------

Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
 2023-08-10 16:40:32 +08:00
Songyang Zhang
3f36db3b06
[Feature] Support turbomind (#166) 
* support turbomind

* update doc

* Update docs/en/advanced_guides/evaluation_turbomind.md

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

* Update docs/zh_cn/advanced_guides/evaluation_turbomind.md

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

* Update docs/zh_cn/advanced_guides/evaluation_turbomind.md

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

* Update docs/en/advanced_guides/evaluation_turbomind.md

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

* update

---------

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
 2023-08-10 16:25:11 +08:00
Ezra-Yu
e9b7b8ab02
[DOC] Add metric doc (#118) 
* update

* update

* update metric docs

* update index.rst

* update metrics
 2023-08-01 11:47:04 +08:00
Tong Gao
fd57786954
[Docs] Polish docs (#43) 
* [Docs] Polish docs

* apply suggestions

* apply suggestions
 2023-07-13 09:07:53 +08:00
Tong Gao
30a988a620
[Docs] Update dataset docs (#19) 
* [Docs] Update dataset docs

* [Docs] Update dataset docs
 2023-07-06 15:47:09 +08:00
Hubert
7f8eee4725
[Docs] add en docs (#15) 
* add en docs

* update

---------

Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>
 2023-07-06 12:58:44 +08:00
Ezra-Yu
83dac269bd
update docs (#14) 
* update docs

* update docs

* update docs
 2023-07-06 12:41:17 +08:00
gaotongxiao
7d346000bb initial commit 2023-07-04 21:34:55 +08:00