mirror of
https://github.com/open-compass/opencompass.git
synced 2025-05-30 16:03:24 +08:00
acae560911
* Add NeedleInAHaystack Test * Apply pre-commit formatting * Update configs/eval_hf_internlm_chat_20b_cdme.py Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com> * add needle in haystack test * update needle in haystack test * update plot function in tools_needleinahaystack.py * optimizing needleinahaystack dataset generation strategy * modify minor formatting issues * add English version support * change NeedleInAHaystackDataset to dynamic loading * change NeedleInAHaystackDataset to dynamic loading * fix needleinahaystack test eval bug * fix needleinahaystack config bug * Added support for multi-needle testing in needle-in-a-haystack test * Optimize the code for plotting in the needle-in-a-haystack test. * Correct the typo in the dataset parameters. * update needleinahaystack test docs --------- Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>
94 lines
2.2 KiB
ReStructuredText
94 lines
2.2 KiB
ReStructuredText
Welcome to OpenCompass' documentation!
|
|
==========================================
|
|
|
|
Getting started with OpenCompass
|
|
-------------------------------
|
|
|
|
To help you quickly familiarized with OpenCompass, we recommend you to walk through the following documents in order:
|
|
|
|
- First read the GetStarted_ section set up the environment, and run a mini experiment.
|
|
|
|
- Then learn its basic usage through the UserGuides_.
|
|
|
|
- If you want to tune the prompts, refer to the Prompt_.
|
|
|
|
- If you want to customize some modules, like adding a new dataset or model, we have provided the AdvancedGuides_.
|
|
|
|
- There are more handy tools, such as prompt viewer and lark bot reporter, all presented in Tools_.
|
|
|
|
We always welcome *PRs* and *Issues* for the betterment of OpenCompass.
|
|
|
|
.. _GetStarted:
|
|
.. toctree::
|
|
:maxdepth: 1
|
|
:caption: Get Started
|
|
|
|
get_started/installation.md
|
|
get_started/quick_start.md
|
|
get_started/faq.md
|
|
|
|
.. _UserGuides:
|
|
.. toctree::
|
|
:maxdepth: 1
|
|
:caption: User Guides
|
|
|
|
user_guides/framework_overview.md
|
|
user_guides/config.md
|
|
user_guides/datasets.md
|
|
user_guides/models.md
|
|
user_guides/evaluation.md
|
|
user_guides/experimentation.md
|
|
user_guides/metrics.md
|
|
user_guides/summarizer.md
|
|
|
|
.. _Prompt:
|
|
.. toctree::
|
|
:maxdepth: 1
|
|
:caption: Prompt
|
|
|
|
prompt/overview.md
|
|
prompt/prompt_template.md
|
|
prompt/meta_template.md
|
|
prompt/chain_of_thought.md
|
|
|
|
|
|
.. _AdvancedGuides:
|
|
.. toctree::
|
|
:maxdepth: 1
|
|
:caption: Advanced Guides
|
|
|
|
advanced_guides/new_dataset.md
|
|
advanced_guides/custom_dataset.md
|
|
advanced_guides/new_model.md
|
|
advanced_guides/evaluation_turbomind.md
|
|
advanced_guides/evaluation_lightllm.md
|
|
advanced_guides/code_eval.md
|
|
advanced_guides/code_eval_service.md
|
|
advanced_guides/multimodal_eval.md
|
|
advanced_guides/prompt_attack.md
|
|
advanced_guides/longeval.md
|
|
advanced_guides/subjective_evaluation.md
|
|
advanced_guides/circular_eval.md
|
|
advanced_guides/contamination_eval.md
|
|
advanced_guides/needleinahaystack_eval.md
|
|
|
|
.. _Tools:
|
|
.. toctree::
|
|
:maxdepth: 1
|
|
:caption: Tools
|
|
|
|
tools.md
|
|
|
|
.. _Notes:
|
|
.. toctree::
|
|
:maxdepth: 1
|
|
:caption: Notes
|
|
|
|
notes/contribution_guide.md
|
|
|
|
Indexes & Tables
|
|
==================
|
|
|
|
* :ref:`genindex`
|
|
* :ref:`search`
|