Ezra-Yu
e9b7b8ab02
[DOC] Add metric doc ( #118 )
...
* update
* update
* update metric docs
* update index.rst
* update metrics
2023-08-01 11:47:04 +08:00
Songyang Zhang
d860b61d04
[Enhancement] Update README.md ( #119 )
...
* Update README.md
* update README_zh-CN.md
* update get_started
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-07-31 18:26:46 +08:00
Leymore
262ab794fb
[Docs] Update prompt docs ( #46 )
...
* [Docs] Update prompt docs
* update
* [Docs] Prompt docs (#112 )
* update docs
* update
* update
* Update en prompt template
* Update en prompt doc
* fix
* fix
---------
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
2023-07-29 00:46:13 +08:00
Anakin Skywalker
e04f88424d
edit doc ( #125 )
2023-07-28 17:33:51 +08:00
Leymore
d862f570aa
[Feature] Add SC ( #126 )
...
* add self-consistency
* add CoT method Self-Consistency
* fix typo error and update openicl_eval
* add tydiQA-GoldP task
* fix sc
* rename gsm8k_sc
* fix sc
* add self-consistency doc
* refine sc
---------
Authored-by: liushz <qq1791167085@163.com>
2023-07-28 17:29:37 +08:00
Haodong Duan
538b439302
[Fix] Fix seed in HFEvaluator ( #122 )
2023-07-28 11:29:01 +08:00
Haodong Duan
46c9645753
[Feature] Allow explicitly setting the temperature for API model ( #121 )
...
* allow explicitly setting the temperature
* update
2023-07-28 11:28:15 +08:00
Tong Gao
80ce18f860
[Docs] Update issue templates for proper guidance to discussions ( #116 )
2023-07-27 19:38:41 +08:00
gowithme
57fcfc975a
[Feature] Support intern lanuage model ( #51 )
...
* support internLM
* support internLM
* simplify intern model files
* update storage_manager
* support internLM
* Modify the file organization structure
* support internLM
* support internLM
* support internLM
* support internLM
* change some details
2023-07-27 18:49:36 +08:00
vansin
8a4d0867ab
Doc: add twitter link ( #111 )
2023-07-27 17:19:35 +08:00
Ezra-Yu
d1ec6047af
[Doc] Update Readme and Fix failed links ( #108 )
...
* update reame and fix failed links
* update
* update review
2023-07-27 17:15:25 +08:00
Hubert
aa13067735
[Feat] add auto assignee bot ( #105 )
...
* [Feat] add auto assignee bot
* minor fix
2023-07-26 16:43:31 +08:00
Hubert
b7184e9db5
[Refactor] Update crows-pairs evaluation ( #98 )
...
* [Refactor] Update crows-pairs evaluation
* [Refactor] Update crows-pairs evaluation
* minor
2023-07-26 11:21:32 +08:00
Haodong Duan
4b0aa80466
[Fix] MMBench Doc Fix ( #96 )
...
* update
* update
* fix lint
2023-07-25 10:43:22 +08:00
Tong Gao
3715be6595
[Fix] Fix llama configs ( #72 )
...
Co-authored-by: Leymore <zfz-960727@163.com>
2023-07-25 10:21:31 +08:00
Haonan Li
e9cdb24ddd
[Feature] Add CMMLU dataset ( #91 )
...
* add CMMLU
* debug cmmlu
* add slurm args `qos`
* fix format: space before comment
* remove unused variable
* change the location of `answer is`
---------
Co-authored-by: 李浩楠 <lihaonan@lihaonandeMacBook-Air.local>
Co-authored-by: 李浩楠 <haonan.li>
Co-authored-by: Leymore <zfz-960727@163.com>
2023-07-25 10:14:27 +08:00
Haodong Duan
6e885d668b
force utf-8 encoding for all non-dataset fileios ( #97 )
2023-07-25 10:06:01 +08:00
Leymore
3fe5ee096c
[Feature] Add heuristic size partitioner ( #63 )
...
* [Feature] Add heuristic size partitioner
* update
2023-07-20 11:53:24 +08:00
Leymore
eea8b04417
[Feature] Add llama-2 models ( #81 )
...
* add llama-2 models
* update docs
---------
Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>
2023-07-19 19:51:29 +08:00
Hubert
f83e125e5a
[Feat] Support CValues Responsibility dataset ( #78 )
...
* [Feat] support CValues
* minor fix
2023-07-18 18:45:15 +08:00
LZH
26e2f171f4
[Feature] Support load PEFT adapter for HuggingFace model ( #74 )
...
* support peft for HuggingFace model
* add docstring
2023-07-18 16:21:43 +08:00
liushz
f36c0496f3
[Feature] Add tydiqa-goldp ( #75 )
...
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
2023-07-18 14:54:35 +08:00
Hubert
29598e3619
[Feat] add falcon-40b ( #76 )
...
* [Feat] add falcon-40b
* minor fix
2023-07-18 14:40:16 +08:00
Tong Gao
311bf0daa7
[Fix] Fix CI ( #70 )
...
* [Fix] Fix CI
* [Fix] Fix CI
* [Fix] Fix CI
* update
2023-07-17 19:10:59 +08:00
Tong Gao
29006e39c0
[Fix] Fix circular import of PromptTemplate ( #71 )
2023-07-17 19:09:38 +08:00
Tong Gao
1e44541730
[Enhancement] Test linting in CI and fix existing linting errors ( #69 )
...
* [Enhancement] Test linting in CI
* fix linting
2023-07-17 15:59:10 +08:00
Leymore
9a16448905
[Fix] eval_llama_7b ( #68 )
2023-07-17 15:28:21 +08:00
Leymore
edb23d15d1
[Feature] Add baichuan13b model configs ( #60 )
...
* [Feature] Add baichuan13b
* update num_gpus
2023-07-17 14:38:12 +08:00
Leymore
1326aff77e
[Feature] Add logger info and remove dataset bugs ( #61 )
...
* Add logger info and remove dataset bugs
* fix typo
2023-07-17 14:26:30 +08:00
Tong Gao
77a1cc4486
[Docs] Update evaluation doc ( #39 )
2023-07-17 14:12:19 +08:00
Leymore
e19a0c1cf8
[Feature] add --dry-run option ( #59 )
2023-07-17 10:41:38 +08:00
vansin
840a8ebecb
[Doc] add discord and wechat link ( #64 )
2023-07-14 15:33:43 +08:00
Yuan Liu
f6c5a839c2
[Feature]: Add MMBench ( #56 )
2023-07-13 10:06:23 +08:00
Tong Gao
fd57786954
[Docs] Polish docs ( #43 )
...
* [Docs] Polish docs
* apply suggestions
* apply suggestions
2023-07-13 09:07:53 +08:00
Tong Gao
7ee5a86fee
[Feature] Enhance OpenAI API, add example config for GPT evaluation ( #53 )
...
* [Feature] Enhance OpenAI API, add example config for GPT evaluation
* fix
2023-07-12 16:43:46 +08:00
Hubert
f5103f93dd
[Feat] add bs for perspective api eval ( #50 )
...
* [Feat] add bs for perspective api eval
* fix according to comments
* fix according to comments
2023-07-12 16:26:01 +08:00
Tong Gao
014b24dd02
[Fix] Fix errors caused by MMEngine 0.8.1 ( #48 )
2023-07-12 11:22:16 +08:00
Hubert
c8f1d513b2
[Fix] fix clp inferencer ( #44 )
2023-07-11 14:54:39 +08:00
Leymore
50b658d234
[Fix] Update HF configs ( #42 )
2023-07-11 10:51:49 +08:00
Tong Gao
0625294e5f
[Fix] Fix OpenICLInferTask ( #41 )
2023-07-10 16:12:01 +08:00
Ezra-Yu
0c6fb6cf67
[Doc] Update logo icon ( #32 )
...
* update logo_icon and fix type in docs
* rebase:
* update get_started
* update .gitignore
* remove extra lines
* remove extra 'S'
* update
* update
* update docs
* update docs
* update docs
---------
Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>
2023-07-08 16:40:24 +08:00
Tong Gao
72e6fc7756
[Docs] Update readme ( #34 )
2023-07-08 10:42:30 +08:00
Ma Zerun
805293a9f2
Auto re-generate port number during retry ( #24 )
...
* Auto re-generate port number during retry
* Fix slurm command
2023-07-07 17:25:56 +08:00
Hubert
efdf116f18
[Fix] fix readme ( #31 )
2023-07-07 17:08:33 +08:00
Tau
92dc143111
[Docs] update leaderboard img ( #29 )
2023-07-07 17:05:31 +08:00
Hubert
e12f6eda0f
[Fix] fix license ( #27 )
2023-07-07 11:43:14 +08:00
Tong Gao
9584574046
[Docs] fix readme ( #21 )
2023-07-06 18:53:25 +08:00
gaotongxiao
81943d039a
[Docs] fix readme
2023-07-06 18:49:36 +08:00
Hubert
29801a03e7
[Fix] fix docs ( #20 )
2023-07-06 16:58:45 +08:00
Leymore
5f2e7c3469
Add models ( #18 )
...
* Add models
* Add comments
2023-07-06 16:02:39 +08:00
