OpenCompass/opencompass/utils
Junnan Liu 73c80953c6
[Feature] Support Dataset Repeat and G-Pass Compute for Each Evaluator (#1886)
* support dataset repeat and g-pass compute for each evaluator

* fix pre-commit errors

* delete print

* delete gpassk_evaluator and fix potential errors

* change `repeat` to `n`

* fix `repeat` to `n` in openicl_eval

* update doc for multi-run and g-pass

* update latex equation in doc

* update eng doc for multi-run and g-pass

* update datasets.md

* update datasets.md

* fix multi-line equation

* fix multi-line equation

* fix multi-line equation

* fix multi-line equation

* fix multi-line equation

* fix multi-line equation

* fix multi-line equation in zh_cn user_guides

* mmodify pre-commit-zh-cn

* recover pre-commit and edit math expr in doc

* del [TIP]

* del cite tag in doc

* del extract_model param in livemathbench config
2025-02-26 19:43:12 +08:00
..
postprocessors [Feature] Add custom model postprocess function (#1519) 2024-09-18 14:40:51 +08:00
__init__.py [Update] Update Skywork/Qwen-QwQ (#1728) 2024-12-05 19:30:43 +08:00
abbr.py [Feature] Add multi-model judge and fix some problems (#1016) 2024-04-02 11:52:06 +08:00
auxiliary.py [Feat] support humaneval and mbpp pass@k (#598) 2023-11-16 21:22:06 +08:00
build.py [Feature] Support Dataset Repeat and G-Pass Compute for Each Evaluator (#1886) 2025-02-26 19:43:12 +08:00
collect_env.py [Feature] Update pip install (#1324) 2024-07-29 18:32:50 +08:00
datasets_info.py [Feature] Support OlympiadBench Benchmark (#1841) 2025-01-24 10:00:01 +08:00
datasets.py [Update] Update o1 eval prompt (#1806) 2025-01-07 00:14:32 +08:00
dependency.py [Feature]: Use multimodal (#73) 2023-08-03 11:07:50 +08:00
dict_postprocessors.py [Feature] Add Judgerbench and reorg subeval (#1593) 2024-10-15 16:36:05 +08:00
file.py fix output typing, change mutable list to immutable tuple (#989) 2024-04-26 23:07:34 +08:00
fileio.py [Update] Update Skywork/Qwen-QwQ (#1728) 2024-12-05 19:30:43 +08:00
lark.py [Feature] Several enhancements (#142) 2023-08-01 18:19:49 +08:00
logging.py [Enhance] Supress warning raised by get_logger (#353) 2023-09-04 15:27:08 +08:00
menu.py [Feat] Support local runner for windows (#515) 2023-10-27 17:16:22 +08:00
model_postprocessors.py [Update] Update MATH dataset with model judge (#1711) 2024-11-25 15:14:55 +08:00
network.py [Update] Update Skywork/Qwen-QwQ (#1728) 2024-12-05 19:30:43 +08:00
prompt.py Support wildbench (#1266) 2024-06-24 13:16:27 +08:00
run.py [Fix] Fix vllm max_seq_len parameter transfer (#1745) 2024-12-16 21:44:36 +08:00
text_postprocessors.py [Feature] Math Verify with model post_processor (#1881) 2025-02-20 19:32:12 +08:00
types.py [Sync] Initial support of subjective evaluation (#421) 2023-09-22 15:42:31 +08:00