Hubert
8c1483e3ce
[Docs] update ds1000 code eval docs ( #618 )
2023-11-22 13:37:53 +08:00
Lyu Han
eb56fd6d16
Integrate turbomind python api ( #484 )
...
* integrate turbomind python api
* update
* update user guide
* update
* fix according to reviewer's comments
* fix error
* fix linting
* update user guide
* remove debug log
---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>
2023-11-21 22:34:46 +08:00
Yang Yong
d3b0d5c4ce
[Feature] Support Lightllm API ( #613 )
...
* [Feature] Support Lightllm api
* formatting & renaming
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-21 19:18:40 +08:00
Hubert
91fba2c2e9
[Feat] support humaneval and mbpp pass@k ( #598 )
...
* [Feat] support pass@ k
* [Feat] support pass@k
* [Feat] support pass@k
* [Feat] support pass@k
* [Feat] support pass@k
* [Feat] support pass@k docs
* update naming
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-11-16 21:22:06 +08:00
Wei Jueqi
14e6fe6f13
Fix bugs in subjective evaluation ( #589 )
...
* rename
* fix sub bugs and update docs
* update
* update
2023-11-14 16:11:55 +08:00
Songyang Zhang
01a0f2f3c7
[Doc] Update README ( #582 )
2023-11-13 20:39:43 +08:00
Fengzhe Zhou
689ffe5b63
[Feature] Use dataset in local path ( #570 )
...
* update commonsenseqa
* update drop
* update flores_first100
* update gsm8k
* update humaneval
* update lambda
* update obqa
* update piqa
* update race
* update siqa
* update story_cloze
* update strategyqa
* update tydiqa
* update winogrande
* update doc
* update hellaswag
* fix obqa
* update collections
* update .zip name
2023-11-13 13:00:37 +08:00
Hubert
95e0da0173
[Docs] add humanevalx dataset link in config ( #559 )
...
* [Docs] add humanevalx dataset link in config
* [Docs] add humanevalx dataset link in config
* minor fix
2023-11-10 18:18:58 +08:00
Hubert
36360bdfc3
[Fix] fix filename typo ( #549 )
2023-11-07 14:00:26 +08:00
Songyang Zhang
239c2a346e
[Feature] Add support for MiniMax API ( #548 )
...
* update requirement
* update requirement
* update with minimax
* update api model
* Update readme
* fix error
---------
Co-authored-by: zhangsongyang <zhangsongyang@pjlab.org.cn>
2023-11-06 21:57:32 +08:00
Songyang Zhang
987a711232
[Doc] Update README and FAQ ( #535 )
...
* update readme
* update readme and faq
2023-11-02 15:16:37 +08:00
Fengzhe Zhou
dbb20b8270
[Sync] update ( #517 )
2023-10-27 20:31:22 +08:00
Wei Jueqi
b62842335d
[Doc] Update Subjective docs ( #510 )
...
* rename
* add en subdoc
* fix name
* fix writing
* update
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-10-27 16:27:24 +08:00
Hubert
44c8d6cc60
[Docs] update invalid link in docs ( #499 )
2023-10-25 13:15:42 +08:00
Leymore
d7ff933a73
[Fix] Use jieba rouge in lcsts ( #459 )
...
* use jieba rouge in lcsts
* use rouge_chinese
2023-10-09 10:10:33 +08:00
Tong Gao
119bfd1569
[Refactor] Move fix_id_list to Retriever ( #442 )
...
* [Refactor] Move fix_id_list to Retriever
* update
* move to base
* fix
2023-10-07 12:53:41 +08:00
Tong Gao
767c12a660
[Docs] update get_started ( #435 )
...
* [Docs] update get_started
* [Docs] Refactor get_started
* update
* add zh FAQ
* add cn doc
* update
* fix dead links
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-10-07 11:49:40 +08:00
Lyu Han
6738247142
Integrate turbomind inference via its RPC API instead of its python API ( #414 )
...
* support tis
* integrate turbomind inference via its RPC API instead of its python API
* update guide
* update ip address spec
* update according to reviewer's comments
2023-10-07 10:27:48 +08:00
Yixiao Fang
524579b5af
[Docs] Add multimodal docs ( #334 )
...
* add multimodal docs
* fix lint
---------
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
2023-09-22 18:58:29 +08:00
Tong Gao
3e980a9737
[Docs] Add intro figure to README ( #413 )
...
* [Docs] Add intro figure to README
* update
2023-09-19 20:19:35 +08:00
philipwangOvO
f57c0702f7
[Docs] Readme in longeval ( #389 )
...
* [Docs] Readme in longeval
* [Docs] Readme in longeval
* [Docs] Readme in longeval
* [Docs] Readme in longeval
* [Docs] Readme in longeval
* [Docs] Readme in longeval
* [Docs] Readme in longeval
2023-09-18 17:06:00 +08:00
Hubert
a11cb45c83
[Feat] implementation for support promptbench ( #239 )
...
* [Feat] support adv_glue dataset for adversarial robustness
* reorg files
* minor fix
* minor fix
* support prompt bench demo
* minor fix
* minor fix
* minor fix
* minor fix
* minor fix
* minor fix
* minor fix
* minor fix
2023-09-15 15:06:53 +08:00
Tong Gao
4d89533fbc
[Docs] Add FAQ ( #384 )
...
* [Docs] Add FAQ
* [Docs] Add FAQ
2023-09-12 12:11:38 +08:00
Tong Gao
b9b145c335
[Docs] Fix incorrect name in get_started ( #380 )
2023-09-11 16:10:09 +08:00
liushz
63ced828d8
Update get_started.md ( #377 )
2023-09-11 10:58:17 +08:00
Leymore
2c915218e8
[Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 ( #373 )
...
* add bag of new models: baichuan2, tigerbot, vicuna v1.5
* update
* re-organize models
* update readme
* update
2023-09-08 15:41:20 +08:00
Songyang Zhang
3871188c89
[Feat] Update URL ( #368 )
2023-09-07 17:29:50 +08:00
Songyang Zhang
a05daab911
[Doc] Update Overview ( #242 )
...
* Update news
* update overview
* add framework
* update index
* update
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-09-07 14:21:39 +08:00
Hubert
2c71b0f6f3
[Docs] update code evaluator docs ( #354 )
...
* [Docs] update code evaluator docs
* minor fix
* minor fix
2023-09-06 17:52:22 +08:00
Haodong Duan
b95aea75ce
[Doc] Update MMBench.md ( #336 )
...
Fix minor problem in the doc
2023-09-01 13:44:45 +08:00
Tong Gao
166022f568
[Docs] Update docs for new entry script ( #246 )
...
* update docs
* update docs
* update
* update en docs
* update
* update
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-08-31 16:43:55 +08:00
Leymore
c26ecdb1b0
[Feature] Add and apply update suffix tool ( #280 )
...
* add and apply update suffix tool
* add dataset suffix updater as precommit hook
* update workflow
* update scripts
* update ci
* update
* ci with py3.8
* run in serial
* update bbh
* use py 3.10
* update pre commit zh cn
2023-08-28 17:35:04 +08:00
Songyang Zhang
b2d602f42b
Update README.md ( #262 )
...
* Update README.md
* update news and readme
* update
* update
---------
Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>
2023-08-25 18:53:35 +08:00
liushz
02ce139bc6
[Feature] Add Tree-of-Thought method ( #173 )
...
* Add ToT method
* Update ToT
* Update ToT
* Update ToT
* Update ToT
* Update ToT
* Update ToT
* Update ToT
* Update chain_of_thought.md
* Update icl_tot_inferencer.py
---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
2023-08-23 12:23:05 +08:00
Leymore
c0e58632ca
[Doc] Add summarizer doc ( #231 )
...
* add summarizer doc
* update
* update doc
* Apply suggestions from code review
---------
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
2023-08-23 11:18:01 +08:00
Ezra-Yu
17ccaa5980
[Feat] Add codegeex2 and Humanevalx ( #210 )
...
* add codegeex2
* add humanevalx dataset
* add evaluator
* update evaluator
* update configs
* update clean code
* update configs
* fix lint
* remove sleep
* fix lint
* update docs
* fix lint
2023-08-17 11:03:16 +08:00
Ezra-Yu
d7cb39581a
update conf ( #212 )
2023-08-16 15:22:14 +08:00
Tong Gao
10cbc2b175
Bump version to 0.1.2 ( #190 )
2023-08-11 17:43:14 +08:00
Tong Gao
e464265cf8
[Docs] Update contribution guide & toc, improve user experience ( #188 )
...
* [Docs] Update contribution guide & toc
* update
* Update docs/en/notes/contribution_guide.md
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>
* update
* update
---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>
2023-08-11 11:36:09 +08:00
Songyang Zhang
99ae786598
[Feature] update news ( #186 )
...
* update news
* update
---------
Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>
2023-08-10 18:52:09 +08:00
liushz
ed248af136
[Fix] Fix some sc errors ( #177 )
...
* Update sc
* Update sc doc
* Apply suggestions from code review
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
2023-08-10 16:40:32 +08:00
Songyang Zhang
3f36db3b06
[Feature] Support turbomind ( #166 )
...
* support turbomind
* update doc
* Update docs/en/advanced_guides/evaluation_turbomind.md
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
* Update docs/zh_cn/advanced_guides/evaluation_turbomind.md
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
* Update docs/zh_cn/advanced_guides/evaluation_turbomind.md
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
* Update docs/en/advanced_guides/evaluation_turbomind.md
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
* update
---------
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
2023-08-10 16:25:11 +08:00
Yuan Liu
191a3f6f9d
[Feature]: Use multimodal ( #73 )
...
* [Feature]: Add minigpt-4
* [Feature]: Add mm local runner
* [Feature]: Add instructblip
* [Feature]: Delete redundant file
* [Feature]: Delete redundant file
* [Feature]: Add README to InstructBLIP
* [Feature]: Update MiniGPT-4
* [Fix]: Fix lint
* [Feature]add omnibenchmark readme (#49 )
* add omnibenchmark readme
* fix
* Update OmniMMBench.md
* Update OmniMMBench.md
* Update OmniMMBench.md
* [Fix]: Refine name (#54 )
* [Feature]: Unify out and err
* [Fix]: Fix lint
* [Feature]: Rename to mmbench and change weight path
* [Feature]: Delete Omni in instructblip
* [Feature]: Check the avaliablity of lavis
* [Fix]: Fix lint
* [Feature]: Refactor MM
* [Refactor]: Refactor path
* [Feature]: Delete redundant files
* [Refactor]: Delete redundant files
---------
Co-authored-by: Wangbo Zhao(黑色枷锁) <56866854+wangbo-zhao@users.noreply.github.com>
2023-08-03 11:07:50 +08:00
Ezra-Yu
e9b7b8ab02
[DOC] Add metric doc ( #118 )
...
* update
* update
* update metric docs
* update index.rst
* update metrics
2023-08-01 11:47:04 +08:00
Songyang Zhang
d860b61d04
[Enhancement] Update README.md ( #119 )
...
* Update README.md
* update README_zh-CN.md
* update get_started
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-07-31 18:26:46 +08:00
Leymore
262ab794fb
[Docs] Update prompt docs ( #46 )
...
* [Docs] Update prompt docs
* update
* [Docs] Prompt docs (#112 )
* update docs
* update
* update
* Update en prompt template
* Update en prompt doc
* fix
* fix
---------
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>
2023-07-29 00:46:13 +08:00
Anakin Skywalker
e04f88424d
edit doc ( #125 )
2023-07-28 17:33:51 +08:00
Leymore
d862f570aa
[Feature] Add SC ( #126 )
...
* add self-consistency
* add CoT method Self-Consistency
* fix typo error and update openicl_eval
* add tydiQA-GoldP task
* fix sc
* rename gsm8k_sc
* fix sc
* add self-consistency doc
* refine sc
---------
Authored-by: liushz <qq1791167085@163.com>
2023-07-28 17:29:37 +08:00
Haodong Duan
4b0aa80466
[Fix] MMBench Doc Fix ( #96 )
...
* update
* update
* fix lint
2023-07-25 10:43:22 +08:00
Leymore
3fe5ee096c
[Feature] Add heuristic size partitioner ( #63 )
...
* [Feature] Add heuristic size partitioner
* update
2023-07-20 11:53:24 +08:00
