Mo Li acae560911 Added support for multi-needle testing in needle-in-a-haystack test (#802) ... * Add NeedleInAHaystack Test * Apply pre-commit formatting * Update configs/eval_hf_internlm_chat_20b_cdme.py Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com> * add needle in haystack test * update needle in haystack test * update plot function in tools_needleinahaystack.py * optimizing needleinahaystack dataset generation strategy * modify minor formatting issues * add English version support * change NeedleInAHaystackDataset to dynamic loading * change NeedleInAHaystackDataset to dynamic loading * fix needleinahaystack test eval bug * fix needleinahaystack config bug * Added support for multi-needle testing in needle-in-a-haystack test * Optimize the code for plotting in the needle-in-a-haystack test. * Correct the typo in the dataset parameters. * update needleinahaystack test docs --------- Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>		2024-01-17 13:47:34 +08:00
..
api_examples	[Feature] Add support for Nanbeige API (#786)	2024-01-11 13:54:27 +08:00
datasets	Added support for multi-needle testing in needle-in-a-haystack test (#802)	2024-01-17 13:47:34 +08:00
models	[Sync] Sync with internal codes 2023.01.08 (#777)	2024-01-08 14:07:24 +00:00
multimodal	docs: fix typos in markdown files (#530)	2023-11-01 16:16:16 +08:00
summarizers	[Sync] Sync with internal codes 2023.01.08 (#777)	2024-01-08 14:07:24 +00:00
eval_alaya.py	Add support for DataCanvas Alaya LM (#612)	2023-11-21 17:51:30 +08:00
eval_attack.py	[Feat] implementation for support promptbench (#239)	2023-09-15 15:06:53 +08:00
eval_chat_agent_baseline.py	[Feat] Update math/agent (#716)	2023-12-19 21:20:42 +08:00
eval_chat_agent.py	[Feat] Update math/agent (#716)	2023-12-19 21:20:42 +08:00
eval_chat_cibench.py	[Sync] minor test (#683)	2023-12-11 17:42:53 +08:00
eval_chat_last.py	[Feature] Support chat style inferencer. (#643)	2023-11-30 14:00:06 +08:00
eval_cibench.py	[Feat] Support cibench (#538)	2023-11-07 19:11:44 +08:00
eval_circular.py	[Feature] Add circular eval (#610)	2023-11-23 16:45:47 +08:00
eval_claude.py	[Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373)	2023-09-08 15:41:20 +08:00
eval_code_passk_repeat_dataset.py	[Feature] Support sanitized MBPP dataset (#745)	2023-12-27 22:17:23 +08:00
eval_code_passk.py	[Feature] Support sanitized MBPP dataset (#745)	2023-12-27 22:17:23 +08:00
eval_codeagent.py	[Feat] Support cibench (#538)	2023-11-07 19:11:44 +08:00
eval_codegeex2.py	[Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373)	2023-09-08 15:41:20 +08:00
eval_contamination.py	[Docs] Update contamination docs (#775)	2024-01-08 16:37:28 +08:00
eval_demo.py	[Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373)	2023-09-08 15:41:20 +08:00
eval_ds1000_interpreter.py	[Feat] Support cibench (#538)	2023-11-07 19:11:44 +08:00
eval_gpt3.5.py	[Feature] Calculate max_out_len without hard code for OpenAI model (#158)	2023-08-08 15:16:56 +08:00
eval_gpt4.py	[Enhancement] Add humaneval postprocessor for GPT models & eval config for GPT4, enhance the original humaneval postprocessor (#129)	2023-08-10 16:31:12 +08:00
eval_hf_llama2.py	[Sync] Sync with internal codes 2023.01.08 (#777)	2024-01-08 14:07:24 +00:00
eval_hf_llama_7b.py	[Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373)	2023-09-08 15:41:20 +08:00
eval_internlm_7b.py	[Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373)	2023-09-08 15:41:20 +08:00
eval_internlm_chat_turbomind_api.py	add turbomind restful api support (#693)	2023-12-24 01:40:00 +08:00
eval_internlm_chat_turbomind_tis.py	Integrate turbomind python api (#484)	2023-11-21 22:34:46 +08:00
eval_internlm_chat_turbomind.py	[Feature] Update evaluate turbomind (#804)	2024-01-17 11:09:50 +08:00
eval_internlm_turbomind_api.py	add turbomind restful api support (#693)	2023-12-24 01:40:00 +08:00
eval_internlm_turbomind_tis.py	Integrate turbomind python api (#484)	2023-11-21 22:34:46 +08:00
eval_internlm_turbomind.py	[Feature] Update evaluate turbomind (#804)	2024-01-17 11:09:50 +08:00
eval_internLM.py	[Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373)	2023-09-08 15:41:20 +08:00
eval_lightllm.py	Update LightllmApi and Fix mmlu bug (#738)	2023-12-27 13:49:08 +08:00
eval_llama2_7b.py	[Feature] Add GPQA Dataset (#729)	2024-01-01 15:54:40 +08:00
eval_mixtral_8x7b.py	[Fix] fix a bug on configs/eval_mixtral_8x7b.py (#706)	2023-12-15 14:15:32 +08:00
eval_multi_prompt_demo.py	[Feature] Add multi-prompt generation demo (#568)	2023-11-20 16:16:37 +08:00
eval_needleinahaystack_turbomind.py	Added support for multi-needle testing in needle-in-a-haystack test (#802)	2024-01-17 13:47:34 +08:00
eval_needleinahaystack.py	Added support for multi-needle testing in needle-in-a-haystack test (#802)	2024-01-17 13:47:34 +08:00
eval_qwen_7b_chat_lawbench.py	[Feature] Add lawbench (#460)	2023-10-13 06:51:36 -05:00
eval_qwen_7b_chat.py	[Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373)	2023-09-08 15:41:20 +08:00
eval_qwen_7b.py	[Feaure] Add new models: baichuan2, tigerbot, vicuna v1.5 (#373)	2023-09-08 15:41:20 +08:00
eval_rwkv5_3b.py	add rwkv-5-3b model (#666)	2023-12-12 18:15:19 +08:00
eval_subjective_alignbench.py	reorganize subject files (#801)	2024-01-16 18:03:11 +08:00
eval_subjective_compare.py	[Fix] Update alignmentbench (#704)	2023-12-14 18:24:21 +08:00
eval_subjective_creationbench.py	reorganize subject files (#801)	2024-01-16 18:03:11 +08:00
eval_subjective_judge_pandalm.py	reorganize subject files (#801)	2024-01-16 18:03:11 +08:00
eval_subjective_score.py	[Fix] Update alignmentbench (#704)	2023-12-14 18:24:21 +08:00
eval_with_model_dataset_combinations.py	[Sync] minor test (#683)	2023-12-11 17:42:53 +08:00