OpenCompass/configs/multimodal/otter/README.md

# OTTER: Multi-modal In-context Instruction Tuning.

### Prepare the environment

```sh
pip install otter_ai
```

### Start evaluation

#### Slurm

```sh
cd $root
python run.py configs/multimodal/tasks.py --mm-eval --slurm -p $PARTITION
```

#### PyTorch


```sh
cd $root
python run.py configs/multimodal/tasks.py --mm-eval
```
[Feat] Add Otter to OpenCompass MMBench Evaluation (#232) * add otter model for opencompass mmbench * add docs * add readme docs * debug for otter opencomass eval * delete unused folders * change to default data path * remove unused files * remove unused files * update * update config file * flake8 lint formated and add prompt generator * add prompt generator to config * add a specific postproecss * add post processor * add post processor * add post processor * update according to suggestions * remove unused redefinition 2023-08-31 12:55:53 +08:00			`# OTTER: Multi-modal In-context Instruction Tuning.`

			`### Prepare the environment`

			```sh
[Feat] Support mm models on public dataset and fix several issues. (#412) * [Feat] Add public dataset support for visualglm, qwenvl, and flamingo * [Fix] MMBench related changes. * [Fix] Openflamingo inference. * [Fix] Hide ckpt path. * [Fix] Pre-commit. --------- Co-authored-by: Haodong Duan <dhd.efz@gmail.com> 2023-09-19 19:08:44 +08:00			`pip install otter_ai`
[Feat] Add Otter to OpenCompass MMBench Evaluation (#232) * add otter model for opencompass mmbench * add docs * add readme docs * debug for otter opencomass eval * delete unused folders * change to default data path * remove unused files * remove unused files * update * update config file * flake8 lint formated and add prompt generator * add prompt generator to config * add a specific postproecss * add post processor * add post processor * add post processor * update according to suggestions * remove unused redefinition 2023-08-31 12:55:53 +08:00			```

			`### Start evaluation`

			`#### Slurm`

			```sh
			`cd $root`
			`python run.py configs/multimodal/tasks.py --mm-eval --slurm -p $PARTITION`
			```

			`#### PyTorch`


			```sh
			`cd $root`
			`python run.py configs/multimodal/tasks.py --mm-eval`
			```