mirror of
https://github.com/open-compass/opencompass.git
synced 2025-05-30 16:03:24 +08:00
doc
This commit is contained in:
parent
60e3b40267
commit
d8a50ba5ff
@ -38,6 +38,12 @@ You can directly read existing results from the data station to avoid duplicate
|
||||
opencompass ... -sp '/your_path' --read-from-station
|
||||
```
|
||||
|
||||
### Command Combination
|
||||
1. Only upload the results under your latest working directory to the data station, without supplementing tasks that missing results:
|
||||
```bash
|
||||
opencompass ... -sp '/your_path' -r latest -m viz
|
||||
```
|
||||
|
||||
## Storage Format of the Data Station
|
||||
|
||||
In the data station, the evaluation results are stored as `json` files for each `model-dataset` pair. The specific directory form is `/your_path/dataset_name/model_name.json `. Each `json` file stores a dictionary corresponding to the results, including `predictions`, `results`, and `cfg`, here is an example:
|
||||
@ -47,8 +53,9 @@ Result = {
|
||||
'predictions': List[Dict],
|
||||
'results': Dict,
|
||||
'cfg': Dict = {
|
||||
'model': Dict,
|
||||
'dataset': Dict
|
||||
'models': Dict,
|
||||
'datasets': Dict,
|
||||
(Only subjective datasets)'judge_models': Dict
|
||||
}
|
||||
}
|
||||
```
|
||||
|
@ -38,6 +38,12 @@ opencompass ... -sp '/your_path' --station-overwrite
|
||||
opencompass ... -sp '/your_path' --read-from-station
|
||||
```
|
||||
|
||||
### 指令组合
|
||||
1. 仅向数据站上传最新工作目录下结果,不补充运行缺失结果的任务:
|
||||
```bash
|
||||
opencompass ... -sp '/your_path' -r latest -m viz
|
||||
```
|
||||
|
||||
## 数据站存储格式
|
||||
|
||||
在数据站中,评测结果按照每个`model-dataset`对的结果存储为`json`文件。具体的目录组织形式为`/your_path/dataset_name/model_name.json`。每个`json`文件都存储了对应结果的字典,包括`predictions`、`results`以及`cfg`三个子项,具体示例如下:
|
||||
@ -47,8 +53,9 @@ Result = {
|
||||
'predictions': List[Dict],
|
||||
'results': Dict,
|
||||
'cfg': Dict = {
|
||||
'model': Dict,
|
||||
'dataset': Dict
|
||||
'models': Dict,
|
||||
'datasets': Dict,
|
||||
(Only subjective datasets)'judge_models': Dict
|
||||
}
|
||||
}
|
||||
```
|
||||
|
Loading…
Reference in New Issue
Block a user