Leymore
d7ff933a73
[Fix] Use jieba rouge in lcsts ( #459 )
...
* use jieba rouge in lcsts
* use rouge_chinese
2023-10-09 10:10:33 +08:00
Tong Gao
119bfd1569
[Refactor] Move fix_id_list to Retriever ( #442 )
...
* [Refactor] Move fix_id_list to Retriever
* update
* move to base
* fix
2023-10-07 12:53:41 +08:00
Hubert
d9f3e88dfe
[Fix] fix clp potential error and support bs>1 ( #439 )
...
* [Fix] fix clp potential error and support bs>1
* [Fix] fix clp potential error and support bs>1
* minor fix
* minor fix
2023-09-27 16:32:57 +08:00
Tong Gao
a1ea3c094a
[Sync] Initial support of subjective evaluation ( #421 )
...
Co-authored-by: Leymore <zfz-960727@163.com>
2023-09-22 15:42:31 +08:00
Ma Zerun
0f2c388280
Support GSM8k evaluation with tools by Lagent and LangChain ( #277 )
...
* Support GSM8k evaluation with tools by Lagent and LangChain
* Avoid to use MMEngine new feature
* update document
---------
Co-authored-by: Leymore <zfz-960727@163.com>
2023-09-22 15:28:22 +08:00
Tong Gao
681d3013de
[Feature] Log gold answer in prediction output ( #419 )
...
* [Feature] Log gold answer in prediction output
* support clp golden ans
* minor fix
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
2023-09-22 12:44:40 +08:00
Leymore
ae0cd8752f
[Feature] Use local accuracy from hf implements ( #416 )
...
* use local accuracy from hf implements
* add load from hf fallback
2023-09-20 16:35:22 +08:00
Hubert
a11cb45c83
[Feat] implementation for support promptbench ( #239 )
...
* [Feat] support adv_glue dataset for adversarial robustness
* reorg files
* minor fix
* minor fix
* support prompt bench demo
* minor fix
* minor fix
* minor fix
* minor fix
* minor fix
* minor fix
* minor fix
* minor fix
2023-09-15 15:06:53 +08:00
cdpath
722eb39526
fix potential oom issue ( #387 )
2023-09-12 10:41:03 +08:00
Leymore
880b34e759
[Fix] Quick lint fix ( #362 )
...
* add default value
* lint fix
* use None
2023-09-06 14:33:13 +08:00
Leymore
b8bf16e81c
[Fix] zero retriever add default value ( #361 )
2023-09-05 10:37:42 +08:00
Leymore
8774465a8f
[Enhancement] ignore ZeroRetriever error when id_list provided ( #340 )
2023-09-04 11:12:16 +08:00
Leymore
e810974068
[Fix] Fix when missing both pad and eos token ( #287 )
...
* fix when missing both pad and eos token
* update pad_token_id impl
2023-08-31 16:53:39 +08:00
liushz
02ce139bc6
[Feature] Add Tree-of-Thought method ( #173 )
...
* Add ToT method
* Update ToT
* Update ToT
* Update ToT
* Update ToT
* Update ToT
* Update ToT
* Update ToT
* Update chain_of_thought.md
* Update icl_tot_inferencer.py
---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
2023-08-23 12:23:05 +08:00
Hubert
5a9539f375
[Feat] add safety to collections ( #185 )
...
* [Feat] add safety to collections
* minor fix
2023-08-11 11:19:26 +08:00
liushz
ed248af136
[Fix] Fix some sc errors ( #177 )
...
* Update sc
* Update sc doc
* Apply suggestions from code review
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
2023-08-10 16:40:32 +08:00
Haodong Duan
d17a5b94fa
[Refine] Refine PR #122 ( #123 )
...
* update
* update
2023-08-03 14:54:38 +08:00
Yuan Liu
191a3f6f9d
[Feature]: Use multimodal ( #73 )
...
* [Feature]: Add minigpt-4
* [Feature]: Add mm local runner
* [Feature]: Add instructblip
* [Feature]: Delete redundant file
* [Feature]: Delete redundant file
* [Feature]: Add README to InstructBLIP
* [Feature]: Update MiniGPT-4
* [Fix]: Fix lint
* [Feature]add omnibenchmark readme (#49 )
* add omnibenchmark readme
* fix
* Update OmniMMBench.md
* Update OmniMMBench.md
* Update OmniMMBench.md
* [Fix]: Refine name (#54 )
* [Feature]: Unify out and err
* [Fix]: Fix lint
* [Feature]: Rename to mmbench and change weight path
* [Feature]: Delete Omni in instructblip
* [Feature]: Check the avaliablity of lavis
* [Fix]: Fix lint
* [Feature]: Refactor MM
* [Refactor]: Refactor path
* [Feature]: Delete redundant files
* [Refactor]: Delete redundant files
---------
Co-authored-by: Wangbo Zhao(黑色枷锁) <56866854+wangbo-zhao@users.noreply.github.com>
2023-08-03 11:07:50 +08:00
Tong Gao
8b163bd8e9
[Feature] Several enhancements ( #142 )
2023-08-01 18:19:49 +08:00
Tong Gao
c00179d46b
[Feature] Evaluating acc based on minimum edit distance, update SIQA ( #130 )
...
* [Feature] Support evaluating acc based on minimum edit distance, update SIQA
* update
2023-08-01 14:24:27 +08:00
Leymore
d862f570aa
[Feature] Add SC ( #126 )
...
* add self-consistency
* add CoT method Self-Consistency
* fix typo error and update openicl_eval
* add tydiQA-GoldP task
* fix sc
* rename gsm8k_sc
* fix sc
* add self-consistency doc
* refine sc
---------
Authored-by: liushz <qq1791167085@163.com>
2023-07-28 17:29:37 +08:00
Haodong Duan
538b439302
[Fix] Fix seed in HFEvaluator ( #122 )
2023-07-28 11:29:01 +08:00
Haodong Duan
6e885d668b
force utf-8 encoding for all non-dataset fileios ( #97 )
2023-07-25 10:06:01 +08:00
Tong Gao
311bf0daa7
[Fix] Fix CI ( #70 )
...
* [Fix] Fix CI
* [Fix] Fix CI
* [Fix] Fix CI
* update
2023-07-17 19:10:59 +08:00
Tong Gao
29006e39c0
[Fix] Fix circular import of PromptTemplate ( #71 )
2023-07-17 19:09:38 +08:00
Tong Gao
1e44541730
[Enhancement] Test linting in CI and fix existing linting errors ( #69 )
...
* [Enhancement] Test linting in CI
* fix linting
2023-07-17 15:59:10 +08:00
Hubert
f5103f93dd
[Feat] add bs for perspective api eval ( #50 )
...
* [Feat] add bs for perspective api eval
* fix according to comments
* fix according to comments
2023-07-12 16:26:01 +08:00
Hubert
c8f1d513b2
[Fix] fix clp inferencer ( #44 )
2023-07-11 14:54:39 +08:00
Leymore
86d5ec3d0f
Update configs ( #9 )
...
* Update implements
* Update
2023-07-06 12:27:41 +08:00
Leymore
c94cc94348
Add release contribution
2023-07-05 03:15:31 +00:00
tonysy
e6b5bdcb87
OpenCompass Public MR
2023-07-05 03:15:21 +00:00
Ezra-Yu
cbe9fe2cdb
Add Release Contraibution
2023-07-05 02:22:40 +00:00
cky
36f111100f
update datasets
2023-07-05 01:45:26 +00:00
mzr1996
3cfe73de3f
Support a batch of datasets.
2023-07-05 01:30:27 +00:00
yingfhu
fb11108723
[Feat] support opencompass
2023-07-04 22:11:33 +08:00
gaotongxiao
7d346000bb
initial commit
2023-07-04 21:34:55 +08:00
