From 4d89533fbcef00176acde72028520c423c3edb39 Mon Sep 17 00:00:00 2001
From: Tong Gao <gaotongxiao@gmail.com>
Date: Tue, 12 Sep 2023 12:11:38 +0800
Subject: [PATCH] [Docs] Add FAQ (#384)

* [Docs] Add FAQ

* [Docs] Add FAQ
---
 docs/en/faq.md               | 22 ++++++++++++++++++++++
 docs/en/get_started.md       |  4 ++++
 docs/en/index.rst            |  1 +
 docs/zh_cn/conf.py           |  8 ++++++++
 docs/zh_cn/cp_origin_docs.sh |  9 +++++++++
 docs/zh_cn/get_started.md    |  4 ++++
 docs/zh_cn/index.rst         |  1 +
 7 files changed, 49 insertions(+)
 create mode 100644 docs/en/faq.md
 create mode 100755 docs/zh_cn/cp_origin_docs.sh

diff --git a/docs/en/faq.md b/docs/en/faq.md
new file mode 100644
index 00000000..ba020e0a
--- /dev/null
+++ b/docs/en/faq.md
@@ -0,0 +1,22 @@
+# FAQ
+
+## Network
+
+**Q1**: My tasks failed with error: `('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))` or `urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='cdn-lfs.huggingface.co', port=443)`
+
+A: Because of HuggingFace's implementation, OpenCompass requires network (especially the connection to HuggingFace) for the first time it loads some datasets and models. Additionally, it connects to HuggingFace each time it is launched. For a successful run, you may:
+
+- Work behind a proxy by specifying the environment variables `http_proxy` and `https_proxy`;
+- Use the cache files from other machines. You may first run the experiment on a machine that has access to the Internet, and then copy the cached files to the offline one. The cached files are located at `~/.cache/huggingface/` by default ([doc](https://huggingface.co/docs/datasets/cache#cache-directory)). When the cached files are ready, you can start the evaluation in offline mode:
+  ```python
+  HF_DATASETS_OFFLINE=1 TRANSFORMERS_OFFLINE=1 HF_EVALUATE_OFFLINE=1 python run.py ...
+  ```
+  With which no more network connection is needed for the evaluation. However, error will still be raised if the files any dataset or model is missing from the cache.
+
+**Q2**: My server cannot connect to the Internet, how can I use OpenCompass?
+
+Use the cache files from other machines, as suggested in the answer to **Q1**.
+
+**Q3**: In evaluation phase, I'm running into an error saying that `FileNotFoundError: Couldn't find a module script at opencompass/accuracy.py. Module 'accuracy' doesn't exist on the Hugging Face Hub either.`
+
+A: HuggingFace tries to load the metric (e.g. `accuracy`) as an module online, and it could fail if the network is unreachable. Please refer to **Q1** for guidelines to fix your network issue.
diff --git a/docs/en/get_started.md b/docs/en/get_started.md
index b03359b5..b518c6e1 100644
--- a/docs/en/get_started.md
+++ b/docs/en/get_started.md
@@ -196,6 +196,10 @@ python run.py configs/eval_demo.py -w outputs/demo
 
 Now let's go over the configuration file and the launch options used in this case.
 
+```{warning}
+OpenCompass usually assumes network is available. If you encounter network issues or wish to run OpenCompass in an offline environment, please refer to [FAQ - Network - Q1](./faq.md#network) for solutions.
+```
+
 ## Explanations
 
 ### Model list - `models`
diff --git a/docs/en/index.rst b/docs/en/index.rst
index c40ef420..6939c0bc 100644
--- a/docs/en/index.rst
+++ b/docs/en/index.rst
@@ -24,6 +24,7 @@ We always welcome *PRs* and *Issues* for the betterment of OpenCompass.
    :caption: Get Started
 
    get_started.md
+   faq.md
 
 .. _UserGuides:
 .. toctree::
diff --git a/docs/zh_cn/conf.py b/docs/zh_cn/conf.py
index f19f0daa..2585688a 100644
--- a/docs/zh_cn/conf.py
+++ b/docs/zh_cn/conf.py
@@ -219,3 +219,11 @@ autodoc_typehints = 'none'
 
 # The not found page
 notfound_template = '404.html'
+
+
+def builder_inited_handler(app):
+    subprocess.run(['./cp_origin_docs.sh'])
+
+
+def setup(app):
+    app.connect('builder-inited', builder_inited_handler)
diff --git a/docs/zh_cn/cp_origin_docs.sh b/docs/zh_cn/cp_origin_docs.sh
new file mode 100755
index 00000000..1e728323
--- /dev/null
+++ b/docs/zh_cn/cp_origin_docs.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+
+# Copy *.md files from docs/ if it doesn't have a Chinese translation
+
+for filename in $(find ../en/ -name '*.md' -printf "%P\n");
+do
+    mkdir -p $(dirname $filename)
+    cp -n ../en/$filename ./$filename
+done
diff --git a/docs/zh_cn/get_started.md b/docs/zh_cn/get_started.md
index a389d991..0571d06d 100644
--- a/docs/zh_cn/get_started.md
+++ b/docs/zh_cn/get_started.md
@@ -278,6 +278,10 @@ python run.py --models hf_llama_7b --datasets base_medium
 更多介绍可查看 [数据集配置](./user_guides/dataset_prepare.md)。
 ```
 
+```{warning}
+OpenCompass 在运行时通常需要依赖网络连接 HuggingFace 服务器 (https://huggingface.co/) 下载模型或数据集。如果连接遇到问题，或者需要离线运行评测，可以参考 [FAQ - Network - Q1](./faq.md#network)。
+```
+
 ### 启动评测
 
 配置文件准备完毕后，我们可以使用 debug 模式启动任务，以检查模型加载、数据集读取是否出现异常，如未正确读取缓存等。
diff --git a/docs/zh_cn/index.rst b/docs/zh_cn/index.rst
index eb5369de..686b19f6 100644
--- a/docs/zh_cn/index.rst
+++ b/docs/zh_cn/index.rst
@@ -25,6 +25,7 @@ OpenCompass 上手路线
    :caption: 开始你的第一步
 
    get_started.md
+   faq.md
 
 .. _教程:
 .. toctree::