jiangzhs/OpenCompass
1
0
Fork 0
mirror of https://github.com/open-compass/opencompass.git synced 2025-05-30 16:03:24 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
a11cb45c83
OpenCompass/configs/multimodal/otter
History
Li Bo a4d6840739
[Feat] Add Otter to OpenCompass MMBench Evaluation (#232) 
* add otter model for opencompass mmbench

* add docs

* add readme docs

* debug for otter opencomass eval

* delete unused folders

* change to default data path

* remove unused files

* remove unused files

* update

* update config file

* flake8 lint formated and add prompt generator

* add prompt generator to config

* add a specific postproecss

* add post processor

* add post processor

* add post processor

* update according to suggestions

* remove unused redefinition
2023-08-31 12:55:53 +08:00
..
otter_9b_mmbench.py [Feat] Add Otter to OpenCompass MMBench Evaluation (#232) 2023-08-31 12:55:53 +08:00
README.md [Feat] Add Otter to OpenCompass MMBench Evaluation (#232) 2023-08-31 12:55:53 +08:00

README.md

OTTER: Multi-modal In-context Instruction Tuning.

Prepare the environment

cd opencompass/multimodal/models/otter
git clone https://github.com/Luodian/Otter.git

Then create a new conda environment and prepare the environement according to this doc

Start evaluation

Slurm

cd $root
python run.py configs/multimodal/tasks.py --mm-eval --slurm -p $PARTITION

PyTorch

cd $root
python run.py configs/multimodal/tasks.py --mm-eval