OpenCompass/opencompass
jxd 12b84aeb3b
[Feature] Update CHARM Memeorziation (#1230)
* update gemini api and add gemini models

* add openai models

* update CHARM evaluation

* add CHARM memorization tasks

* add CharmMemSummarizer (output eval details for memorization-independent reasoning analysis

* update CHARM readme

---------

Co-authored-by: wujiang <wujiang@pjlab.org.cn>
2024-07-26 18:42:30 +08:00
..
cli [Fix] add bc for alignbench summarizer (#1306) 2024-07-12 11:06:20 +08:00
datasets [Feature] Update CHARM Memeorziation (#1230) 2024-07-26 18:42:30 +08:00
lagent Update CIBench (#1089) 2024-04-26 18:46:02 +08:00
metrics [Feat] Support multi-modal evaluation on MME benchmark. (#197) 2023-08-21 15:53:20 +08:00
models [Feature] Update CHARM Memeorziation (#1230) 2024-07-26 18:42:30 +08:00
openicl [Feature] Support inference ppl datasets (#1315) 2024-07-22 17:59:30 +08:00
partitioners update docs (#1318) 2024-07-25 18:44:25 +08:00
runners [Bug] fix bug: delete & (#1365) 2024-07-26 11:03:55 +08:00
summarizers [Feature] Update CHARM Memeorziation (#1230) 2024-07-26 18:42:30 +08:00
tasks [Fix] origin_prompt should be None in llm-compression task (#1225) 2024-07-26 11:46:02 +08:00
utils [Doc] quick start swap tabs (#1263) 2024-07-05 23:51:42 +08:00
__init__.py [Sync] bump version 0.2.6+local (#1294) 2024-07-06 00:44:06 +08:00
registry.py [Deperecate] Remove multi-modal related stuff (#1072) 2024-04-26 21:20:14 +08:00