jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[Feature] add global retriever config (#842)

Browse Source 
* add global retriever config

* give zero shot overwrite example

* give zero shot overwrite example

---------

Co-authored-by: Lei Fei <SENSETIME\leifei1@cn3114002087l.domain.sensetime.com>
Co-authored-by: Leymore <zfz-960727@163.com>
This commit is contained in:
hailsham 2024-02-07 00:30:20 +08:00 committed by GitHub
parent dd444685bb
commit e257254b00
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 14 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
configs/eval_mmlu_with_zero_retriever_overwritten.py Normal file
View File

@ -0,0 +1,14 @@
from mmengine.config import read_base
from opencompass.openicl.icl_retriever import ZeroRetriever
from copy import deepcopy
with read_base():
from .models.qwen.hf_qwen_7b_chat import models
from .datasets.mmlu.mmlu_gen_4d595a import mmlu_datasets # this is a dataset evaluated with 5-shot
datasets = []
for d in mmlu_datasets:
d = deepcopy(d)
d['infer_cfg']['retriever'] = dict(type=ZeroRetriever)
datasets.append(d)