jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
333 Commits 11 Branches 35 Tags 66 MiB
4780b39eda
Commit Graph

333 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Hubert
4780b39eda
[Sync] format (#690) 
Co-authored-by: Leymore <zfz-960727@163.com>
 2023-12-12 14:03:45 +08:00
bittersweet1999
3e77175720
[Fix] Hotfix for Subjective Evaluation (#686) 2023-12-12 09:22:08 +08:00
bittersweet1999
465308e430
[Feature] Add Subjective Evaluation (#680) 
* new version of subject

* fixed draw

* fixed draw

* fixed draw

* done

* done

* done

* done

* fixed lint
 2023-12-11 22:22:11 +08:00
Hubert
4f0b373a0a
[Fix] fix docstring (#684) 2023-12-11 19:12:01 +08:00
Hubert
e78857ac36
[Sync] minor test (#683) 2023-12-11 17:42:53 +08:00
Jingming
dd4318f6ab
[Feature] enhance the ability of humaneval_postprocess (#676) 
* [Feature] enhance the ability of humaneval_postprocess

* refactor

* [Feature] Keep the old version of the function and realize the new function in humaneval_postprocess_v2.

* Update opencompass/datasets/humaneval.py

---------

Co-authored-by: Leymore <zfz-960727@163.com>
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
 2023-12-11 14:39:56 +08:00
Hubert
1029119e39
[Feat] support pr merge test ci (#669) 
* [Feat] support ci

* [Feat] support ci

* [Feat] support ci

* [Feat] support ci

* init docs

* init docs

* init docs
 2023-12-11 14:12:04 +08:00
Haodong Duan
6a928b996a
[Doc] Update README (#682) 2023-12-10 21:27:46 +08:00
Songyang Zhang
e25c5f9525
[Enhancement] Update API Interface and Mixtral (#681) 
* [Enhancement] Update API interface

* [Enhancement] Update API interface

* Update mixtral

* Update readme
 2023-12-10 13:29:26 +08:00
Xiaoming Shi
1bf85949ef
[Feature] Add medbench (#678) 
* update medbench

* medbench update

* format medbench

* format

---------

Co-authored-by: 施晓明 <PJLAB\shixiaoming@pjnl104220118l.pjlab.org>
Co-authored-by: Leymore <zfz-960727@163.com>
 2023-12-09 16:05:46 +08:00
Jingming
7cb53a95fa
[Fix] fix bug on standart_deviation summarizer (#675) 2023-12-08 13:38:07 +08:00
liyucheng09
05bbce8b08
[Feature] Add Data Contamination Analysis (#639) 
* add contamination analysis to ceval

* fix bugs

* add contamination docs

* to pass CI check

* update

---------

Co-authored-by: zhangyifan1 <zhangyifan1@pjlab.org.cn>
Co-authored-by: Leymore <zfz-960727@163.com>
 2023-12-08 10:00:11 +08:00
Fengzhe Zhou
3a354bd1da
add qwen and deepseek configs (#672) 2023-12-07 20:29:00 +08:00
bittersweet1999
1c95790fdd
New subjective judgement (#660) 
* TabMWP

* TabMWP

* fixed

* fixed

* fixed

* done

* done

* done

* add new subjective judgement

* add new subjective judgement

* add new subjective judgement

* add new subjective judgement

* add new subjective judgement

* modified to a more general way

* modified to a more general way

* final

* final

* add summarizer

* add new summarize

* fixed

* fixed

* fixed

---------

Co-authored-by: caomaosong <caomaosong@pjlab.org.cn>
 2023-12-06 13:28:33 +08:00
rolellm
e10f1c9139
added rolebench dataset. (#633) 
* added rolebench

* 修改了不合理的变量名

* 修改了评论中的变量名
 2023-12-01 22:54:42 +08:00
liushz
f4bbff6537
[Feature] Update MathBench CodeInterpreter & fix MathBench Bug (#657) 
* Update MathBench CodeInterpreter & fix MathBench Bug

* Fix errors

* update

---------

Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
Co-authored-by: Fengzhe Zhou <zfz-960727@163.com>
 2023-12-01 22:27:24 +08:00
Hubert
9eb5cadcac
[Feat] update gsm8k and math agent config (#652) 
* [Feat] update gsm8k and math agent config

* minor fix
 2023-12-01 15:08:38 +08:00
liushz
a331c9abfd
[Feature] Add wikibench dataset (#655) 
* Add WikiBench

* Add WikiBench

* format

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-12-01 14:56:54 +08:00
liushz
e019c831fe
[Feature] Add Chinese version: commonsenseqa, crowspairs and nq (#144) 
* add Chinese version: csqa crowspairs nq

* Update cn_data

* Update cn_data

* update format

---------

Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
Co-authored-by: Leymore <zfz-960727@163.com>
 2023-11-30 15:33:02 +08:00
Ma Zerun
6aaf3b91ec
[Feature] Support chat style inferencer. (#643) 
* [Feature] Support chat style inferencer.

* [Fix] use new prompt

* [Fix] use new prompt

---------

Co-authored-by: yingfhu <yingfhu@gmail.com>
 2023-11-30 14:00:06 +08:00
Fengzhe Zhou
5933c04fda
fix hellaswag_ppl_47bff9 (#648) 2023-11-29 16:51:44 +08:00
Hubert
e9e75fb4eb
[Fix] remove colossalai dependency (#645) 2023-11-28 14:09:44 +08:00
Fengzhe Zhou
e20d654c18
[Sync] Bump version to 0.1.9 (#644) 2023-11-28 11:42:43 +08:00
Hubert
d4af31bab4
[Feat] support zhipu post process (#642) 
* [Feat] support zhipu post

* [Feat] support zhipu post

* [Feat] support zhipu post
 2023-11-27 19:57:36 +08:00
liushz
6d0d78986c
[Feature] Add GSM_Hard dataset (#619) 
* Add SVAMP dataset

* Add SVAMP dataset

* Add SVAMP dataset

* Add gsm_hard dataset

* Add gsm_hard dataset

* format

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-11-27 17:40:34 +08:00
Fengzhe Zhou
9083dea683
[Sync] some renaming (#641) 2023-11-27 16:06:49 +08:00
Yggdrasill7D6
68c4c1ef86
[Fix] fix typo in README (#637) 2023-11-24 17:49:04 +08:00
Yang Yong
522241a8c9
[Fix] Fix lightllmapi list bug (#635) 2023-11-24 14:24:13 +08:00
Hubert
1884912674
[Bug] fix icl eval with nested list (#632) 2023-11-24 13:43:26 +08:00
Songyang Zhang
81b67e8d9e
[Doc] Update README (#629) 
* update readme

* fix typo

* Update README.md

---------

Co-authored-by: liushz <qq1791167085@163.com>
 2023-11-24 11:24:00 +08:00
Fengzhe Zhou
79f6449d85
[Doc] Update FAQ (#628) 
* update faq

* Update docs/zh_cn/get_started/faq.md

* Update docs/en/get_started/faq.md

* Update docs/zh_cn/get_started/faq.md

---------

Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>
 2023-11-23 18:19:17 +08:00
Fengzhe Zhou
d949e3c003
[Feature] Add circular eval (#610) 
* refactor default, add circular summarizer

* add circular

* update impl

* update doc

* minor update

* no more to be added
 2023-11-23 16:45:47 +08:00
Songyang Zhang
5202456b4c
[API] Update API (#624) 
* update api

* update generation_kwargs impl

* update api

* refactor

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-11-23 15:06:20 +08:00
Fengzhe Zhou
d4d1330a5a
[Sync] Fix cmnli, fix vicuna meta template, fix longbench postprocess and other minor fixes (#625) 2023-11-23 14:05:59 +08:00
Songyang Zhang
5329724b65
[Doc] Update README and requirements. (#622) 
* update readme

* update doc
 2023-11-22 19:16:54 +08:00
Kevin Wang
c0785e53d8
[Feature] support download from modelscope (#534) 
* [Feature] download from modelscope

* [Feature] download from modelscope

* minor fix

---------

Co-authored-by: yingfhu <yingfhu@gmail.com>
 2023-11-22 15:32:21 +08:00
liushz
048775192b
[Feature] Add SVAMP dataset (#604) 
* Add SVAMP dataset

* Add SVAMP dataset

* Add SVAMP dataset
 2023-11-22 14:54:39 +08:00
Hubert
8c1483e3ce
[Docs] update ds1000 code eval docs (#618) 2023-11-22 13:37:53 +08:00
Fengzhe Zhou
fb30b7c7a2
[Fix] Fix gen inferencer (#615) 2023-11-22 12:04:31 +08:00
Songyang Zhang
721a45c68f
[Bug] Update api with generation_kargs (#614) 
* update api

* update generation_kwargs impl

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-11-22 10:02:57 +08:00
Lyu Han
eb56fd6d16
Integrate turbomind python api (#484) 
* integrate turbomind python api

* update

* update user guide

* update

* fix according to reviewer's comments

* fix error

* fix linting

* update user guide

* remove debug log

---------

Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>
 2023-11-21 22:34:46 +08:00
Songyang Zhang
d925748266
[Feature] Support 360API and FixKRetriever for CSQA dataset (#601) 
* [Feature] Support 360API and FixKRetriever for CSQA dataset

* Update API

* Update API

* [Feature] Support 360API and FixKRetriever for CSQA dataset

* Update API

* Update API

* rm mathbench

* fix_lint

* Update opencompass/models/bytedance_api.py

Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>

* update

* update

* update

---------

Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
 2023-11-21 20:25:47 +08:00
Yang Yong
d3b0d5c4ce
[Feature] Support Lightllm API (#613) 
* [Feature] Support Lightllm api

* formatting & renaming

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-11-21 19:18:40 +08:00
Yuan Feng
7199acc25d
Add support for DataCanvas Alaya LM (#612) 
* Support for Alaya

* Remove useless requirements
 2023-11-21 17:51:30 +08:00
liushz
dbacd36379
Add aritch to mathbench (#607) 2023-11-20 19:40:41 +08:00
liushz
c9c5c5d92e
Mathbench update postprocess (#600) 
* Update mathbench

* Update mathbench
 2023-11-20 16:48:55 +08:00
Jingming
5e75e29711
[Feature] Add multi-prompt generation demo (#568) 
* [Feature] Add multi-prompt generation demo

* [Fix] change form in winogrande_gen_XXX.py

* [Fix] make multi prompt demo more directly

* [Fix] fix bug

* [Fix] minor fix

---------

Co-authored-by: yingfhu <yingfhu@gmail.com>
 2023-11-20 16:16:37 +08:00
Hubert
91fba2c2e9
[Feat] support humaneval and mbpp pass@k (#598) 
* [Feat] support pass@ k

* [Feat] support pass@k

* [Feat] support pass@k

* [Feat] support pass@k

* [Feat] support pass@k

* [Feat] support pass@k docs

* update naming

---------

Co-authored-by: Leymore <zfz-960727@163.com>
 2023-11-16 21:22:06 +08:00
Raymond Zhang
c0acd06b05
[Feature] Add FinanceIQ dataset (#596) 2023-11-16 17:47:57 +08:00
Yu
8160cb84e3
update word spell (#594) 2023-11-15 15:23:58 +08:00