jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
f2af49337d
OpenCompass/configs
History
Mo Li f2af49337d
[Feature] Add ATC Choice Version (#1019) 
* Squashed commit of the following:

commit c48ad194c3976dc63d1b60d8c8ab2d5ff9e1cbfe
Author: DseidLi <2568818204@qq.com>
Date:   Tue Apr 2 16:57:43 2024 +0800

    add atc_choice

commit 3ac6efea29619573e6fac8fa3cce464853dcead0
Merge: 2d4e559 8e3a9c3
Author: DseidLi <2568818204@qq.com>
Date:   Tue Apr 2 16:41:38 2024 +0800

    Merge branch 'atc_choice' into atc_add_choice

commit 8e3a9c396a3e5546d3faf584183f6fd60b974d5e
Merge: 150a036 0a6a03f
Author: DseidLi <2568818204@qq.com>
Date:   Tue Mar 26 04:47:07 2024 +0800

    Merge branch 'main' into atc_choice

    Conflicts:
    	configs/summarizers/needlebench.py
    	opencompass/datasets/needlebench/multi.py
    	opencompass/datasets/needlebench/origin.py
    	opencompass/datasets/needlebench/parallel.py

commit 150a036d6d990f26a57c974d1af83d88c31a0f9d
Merge: 8d6ac9a 940dd18
Author: DseidLi <2568818204@qq.com>
Date:   Wed Mar 20 03:49:08 2024 +0800

    Merge branch 'needlebench_fix' into atc_choice

commit 8d6ac9a1a43b1c9d0f0ea27e7d58968a203ea898
Author: DseidLi <2568818204@qq.com>
Date:   Wed Mar 20 03:41:49 2024 +0800

    optimize needlebench code

commit 940dd18a4270f24bc69edd2a780182c68918e1a9
Author: DseidLi <2568818204@qq.com>
Date:   Wed Mar 20 03:39:46 2024 +0800

    fix vllm

commit d8be6877bc41051f3edcc0421c462c834c0f1c9a
Merge: ecad78a 2527fda
Author: DseidLi <2568818204@qq.com>
Date:   Tue Mar 19 21:07:08 2024 +0800

    Merge remote-tracking branch 'origin/add_1M_dataset' into atc_choice

commit 2527fda8a5
Author: DseidLi <2568818204@qq.com>
Date:   Tue Mar 19 16:03:40 2024 +0800

    add model configs

commit 75425acdf8
Author: DseidLi <2568818204@qq.com>
Date:   Tue Mar 19 16:02:15 2024 +0800

    add prompt postion args

commit 367ba1ba61
Author: DseidLi <2568818204@qq.com>
Date:   Wed Feb 28 21:40:00 2024 +0800

    add Needlebench-1000K configs

commit ecad78af14c4bb00fe325779114b384c57ab30bf
Author: DseidLi <2568818204@qq.com>
Date:   Thu Mar 14 22:08:32 2024 +0800

    fix atc

commit 08772c0787b18872abadc9ffec3223941a5ee0c2
Merge: 9f3f8cf caf1cf8
Author: DseidLi <2568818204@qq.com>
Date:   Thu Mar 14 22:07:28 2024 +0800

    Merge branch 'main' into atc_choice

    Conflicts:
    	configs/datasets/needlebench/readme.md
    	configs/datasets/needlebench/readme_zh-CN.md
    	configs/summarizers/needlebench.py
    	opencompass/datasets/needlebench/atc.py
    	opencompass/summarizers/needlebench.py

commit 9f3f8cfb4452722734d334114ac1d14110e57406
Author: DseidLi <2568818204@qq.com>
Date:   Thu Mar 14 21:35:53 2024 +0800

    add atc-choice test

commit 52be7c1202376b4e09821188b826f1a805328129
Author: DseidLi <2568818204@qq.com>
Date:   Wed Mar 6 02:54:15 2024 +0800

    update needlebench randomseed and add vllm qwen14b

commit fc1effce596ae2e5ece4933e8cd34aef8e64a6f9
Merge: 4e747ed caf1cf8
Author: DseidLi <2568818204@qq.com>
Date:   Wed Mar 6 02:51:14 2024 +0800

    Merge branch 'main' into add_model_configs

commit 31834f9b23af3354ac3581ec86d693d0f05cdd1c
Merge: 7dabc82 120bf8b
Author: DseidLi <2568818204@qq.com>
Date:   Sun Mar 3 23:29:42 2024 +0800

    Merge branch 'main' of https://github.com/open-compass/opencompass into atc_choice

commit 4e747ed1988ddbcfcc7fff334601259ade72d363
Author: DseidLi <2568818204@qq.com>
Date:   Sun Mar 3 22:15:25 2024 +0800

    add internlm2-lmdeploy model and gemma configs

commit 7dabc828123d711c8cf834d6aab4137bb55e85ed
Author: DseidLi <2568818204@qq.com>
Date:   Sat Mar 2 17:26:15 2024 +0800

    add atc choice version -ZH

commit 996f8ae43d
Author: DseidLi <2568818204@qq.com>
Date:   Wed Feb 28 16:58:56 2024 +0800

    update readme for needlebench

commit f7266e873c
Author: DseidLi <2568818204@qq.com>
Date:   Wed Feb 28 16:44:53 2024 +0800

    move readme.md

commit 1c7375681d
Author: DseidLi <2568818204@qq.com>
Date:   Wed Feb 28 16:38:31 2024 +0800

    fix linting error

commit b6524f3ebf
Author: DseidLi <2568818204@qq.com>
Date:   Wed Feb 28 16:33:51 2024 +0800

    lint summarizer

commit c0d1190e39
Author: DseidLi <2568818204@qq.com>
Date:   Wed Feb 28 16:29:03 2024 +0800

    add needlebench intro, fix summarizer

commit 0965baf785
Author: DseidLi <2568818204@qq.com>
Date:   Mon Feb 26 13:31:26 2024 +0800

    fix bug in needlebench summarizer

commit 5d32b31eb8
Author: DseidLi <2568818204@qq.com>
Date:   Sat Feb 24 03:19:08 2024 +0800

    update act prompt

commit af82a7f085
Merge: 32bf9fe 53fe788
Author: DseidLi <2568818204@qq.com>
Date:   Fri Feb 23 17:50:32 2024 +0800

    Merge remote-tracking branch 'upstream/main' into needlebench

commit 32bf9fe802
Author: DseidLi <2568818204@qq.com>
Date:   Fri Feb 23 17:31:32 2024 +0800

    simplify needlebench 32k, 128k, 200k for eval

commit a7cb025e05
Author: DseidLi <2568818204@qq.com>
Date:   Fri Feb 23 14:48:58 2024 +0800

    add needlebench

* fix summarizer

* remove repeated code

* remove chinese comments
2024-04-07 15:46:20 +08:00
..
api_examples [Fix] Update Zhipu API and Fix issue min_out_len issue of API models (#847) 2024-01-28 14:52:43 +08:00
datasets [Feature] Add ATC Choice Version (#1019) 2024-04-07 15:46:20 +08:00
models [Feature] Add ATC Choice Version (#1019) 2024-04-07 15:46:20 +08:00
multimodal docs: fix typos in markdown files (#530) 2023-11-01 16:16:16 +08:00
subjective [Feature] Support AlpacaEval_V2 (#1006) 2024-03-28 16:49:04 +08:00
summarizers [Feature] Add ATC Choice Version (#1019) 2024-04-07 15:46:20 +08:00
eval_alaya.py Add support for DataCanvas Alaya LM (#612) 2023-11-21 17:51:30 +08:00
eval_attack.py [Feat] implementation for support promptbench (#239) 2023-09-15 15:06:53 +08:00
eval_bluelm_32k_lveval.py [Feature] add lveval benchmark (#914) 2024-03-04 11:22:03 +08:00
eval_chat_agent_baseline.py [Feat] Update math/agent (#716) 2023-12-19 21:20:42 +08:00
eval_chat_agent.py [Feat] Update math/agent (#716) 2023-12-19 21:20:42 +08:00
eval_chat_cibench_api.py [Feat] minor update agent related (#839) 2024-01-26 14:15:51 +08:00
eval_chat_cibench.py [Sync] minor test (#683) 2023-12-11 17:42:53 +08:00
eval_chat_last.py [Feature] Support chat style inferencer. (#643) 2023-11-30 14:00:06 +08:00
eval_cibench.py [Feat] Support cibench (#538) 2023-11-07 19:11:44 +08:00
eval_circular.py [Feature] Add circular eval (#610) 2023-11-23 16:45:47 +08:00
eval_claude.py [Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373) 2023-09-08 15:41:20 +08:00
eval_code_passk_repeat_dataset.py [Feature] Support sanitized MBPP dataset (#745) 2023-12-27 22:17:23 +08:00
eval_code_passk.py [Feature] Support sanitized MBPP dataset (#745) 2023-12-27 22:17:23 +08:00
eval_codeagent.py [Feat] Support cibench (#538) 2023-11-07 19:11:44 +08:00
eval_codegeex2.py [Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373) 2023-09-08 15:41:20 +08:00
eval_contamination.py [Docs] Update contamination docs (#775) 2024-01-08 16:37:28 +08:00
eval_demo.py [Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373) 2023-09-08 15:41:20 +08:00
eval_ds1000_interpreter.py [Feat] Support cibench (#538) 2023-11-07 19:11:44 +08:00
eval_gpt3.5.py [Feature] Calculate max_out_len without hard code for OpenAI model (#158) 2023-08-08 15:16:56 +08:00
eval_gpt4.py [Enhancement] Add humaneval postprocessor for GPT models & eval config for GPT4, enhance the original humaneval postprocessor (#129) 2023-08-10 16:31:12 +08:00
eval_hf_llama2.py [Sync] Sync with internal codes 2023.01.08 (#777) 2024-01-08 14:07:24 +00:00
eval_hf_llama_7b.py [Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373) 2023-09-08 15:41:20 +08:00
eval_internlm2_chat_keyset.py [Sync] Add InternLM2 Keyset Evaluation Demo (#807) 2024-01-17 13:48:12 +08:00
eval_internlm2_keyset.py [Sync] Add InternLM2 Keyset Evaluation Demo (#807) 2024-01-17 13:48:12 +08:00
eval_internlm_7b.py [Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373) 2023-09-08 15:41:20 +08:00
eval_internlm_chat_lmdeploy_apiserver.py Support get_ppl for TurbomindModel (#878) 2024-03-06 11:44:19 +08:00
eval_internlm_chat_lmdeploy_pytorch.py Support lmdeploy pytorch engine (#875) 2024-02-22 03:46:07 -03:00
eval_internlm_chat_turbomind_tis.py [Fix] Fix turbomind_tis (#992) 2024-03-22 15:50:12 +08:00
eval_internlm_chat_turbomind.py [Feature] Add end_str for turbomind (#859) 2024-02-01 22:31:14 +08:00
eval_internlm_lmdeploy_apiserver.py Support get_ppl for TurbomindModel (#878) 2024-03-06 11:44:19 +08:00
eval_internlm_math_chat.py [Sync] Merge branch 'dev' into zfz/update-keyset-demo (#876) 2024-02-05 23:29:10 +08:00
eval_internlm_turbomind_tis.py Integrate turbomind python api (#484) 2023-11-21 22:34:46 +08:00
eval_internlm_turbomind.py [Fix] Fix turbomind and update docs (#808) 2024-01-18 14:41:35 +08:00
eval_internLM.py [Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373) 2023-09-08 15:41:20 +08:00
eval_lightllm.py Support prompt template for LightllmApi. Update LightllmApi token bucket. (#945) 2024-03-06 15:33:53 +08:00
eval_llama2_7b_lveval.py [Feature] add lveval benchmark (#914) 2024-03-04 11:22:03 +08:00
eval_llama2_7b.py [Feature] Add GPQA Dataset (#729) 2024-01-01 15:54:40 +08:00
eval_mixtral_8x7b.py [Fix] fix a bug on configs/eval_mixtral_8x7b.py (#706) 2023-12-15 14:15:32 +08:00
eval_mmlu_with_zero_retriever_overwritten.py [Feature] add global retriever config (#842) 2024-02-07 00:30:20 +08:00
eval_multi_prompt_demo.py [Feature] Add multi-prompt generation demo (#568) 2023-11-20 16:16:37 +08:00
eval_needlebench.py [Feature] Upgrade the needle-in-a-haystack experiment to Needlebench (#913) 2024-03-04 11:10:52 +08:00
eval_qwen_7b_chat_lawbench.py [Feature] Add lawbench (#460) 2023-10-13 06:51:36 -05:00
eval_qwen_7b_chat.py [Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373) 2023-09-08 15:41:20 +08:00
eval_qwen_7b.py [Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373) 2023-09-08 15:41:20 +08:00
eval_rwkv5_3b.py add rwkv-5-3b model (#666) 2023-12-12 18:15:19 +08:00
eval_subjective_alignbench.py [Feature] Add multi-model judge and fix some problems (#1016) 2024-04-02 11:52:06 +08:00
eval_subjective_alpacaeval_oc.py [Feature] Add multi-model judge and fix some problems (#1016) 2024-04-02 11:52:06 +08:00
eval_subjective_alpacaeval.py [Feature] Support AlpacaEval_V2 (#1006) 2024-03-28 16:49:04 +08:00
eval_subjective_compassarena.py [Feature] Add multi-model judge and fix some problems (#1016) 2024-04-02 11:52:06 +08:00
eval_subjective_creationbench.py [Fix] Quick fix (#995) 2024-03-22 19:54:19 +08:00
eval_subjective_judge_pandalm.py [Feature] Add multi-model judge and fix some problems (#1016) 2024-04-02 11:52:06 +08:00
eval_subjective_mtbench.py [Feature] Add multi-model judge and fix some problems (#1016) 2024-04-02 11:52:06 +08:00
eval_teval.py [Sync] Merge branch 'dev' into zfz/update-keyset-demo (#876) 2024-02-05 23:29:10 +08:00
eval_with_model_dataset_combinations.py [Sync] minor test (#683) 2023-12-11 17:42:53 +08:00