[Docs] Add FAQ (#384)

* [Docs] Add FAQ

* [Docs] Add FAQ

docs/en/faq.md

@@ -0,0 +1,22 @@
+# FAQ
+
+## Network
+
+**Q1**: My tasks failed with error: `('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))` or `urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='cdn-lfs.huggingface.co', port=443)`
+
+A: Because of HuggingFace's implementation, OpenCompass requires network (especially the connection to HuggingFace) for the first time it loads some datasets and models. Additionally, it connects to HuggingFace each time it is launched. For a successful run, you may:
+
+- Work behind a proxy by specifying the environment variables `http_proxy` and `https_proxy`;
+- Use the cache files from other machines. You may first run the experiment on a machine that has access to the Internet, and then copy the cached files to the offline one. The cached files are located at `~/.cache/huggingface/` by default ([doc](https://huggingface.co/docs/datasets/cache#cache-directory)). When the cached files are ready, you can start the evaluation in offline mode:
+  ```python
+  HF_DATASETS_OFFLINE=1 TRANSFORMERS_OFFLINE=1 HF_EVALUATE_OFFLINE=1 python run.py ...
+  ```
+  With which no more network connection is needed for the evaluation. However, error will still be raised if the files any dataset or model is missing from the cache.
+
+**Q2**: My server cannot connect to the Internet, how can I use OpenCompass?
+
+Use the cache files from other machines, as suggested in the answer to **Q1**.
+
+**Q3**: In evaluation phase, I'm running into an error saying that `FileNotFoundError: Couldn't find a module script at opencompass/accuracy.py. Module 'accuracy' doesn't exist on the Hugging Face Hub either.`
+
+A: HuggingFace tries to load the metric (e.g. `accuracy`) as an module online, and it could fail if the network is unreachable. Please refer to **Q1** for guidelines to fix your network issue.

docs/en/get_started.md

@@ -196,6 +196,10 @@ python run.py configs/eval_demo.py -w outputs/demo
 
 Now let's go over the configuration file and the launch options used in this case.
 
+```{warning}
+OpenCompass usually assumes network is available. If you encounter network issues or wish to run OpenCompass in an offline environment, please refer to [FAQ - Network - Q1](./faq.md#network) for solutions.
+```
+
 ## Explanations
 
 ### Model list - `models`

docs/en/index.rst

@@ -24,6 +24,7 @@ We always welcome *PRs* and *Issues* for the betterment of OpenCompass.
    :caption: Get Started
 
    get_started.md
+   faq.md
 
 .. _UserGuides:
 .. toctree::

docs/zh_cn/conf.py

@@ -219,3 +219,11 @@ autodoc_typehints = 'none'
 
 # The not found page
 notfound_template = '404.html'
+
+
+def builder_inited_handler(app):
+    subprocess.run(['./cp_origin_docs.sh'])
+
+
+def setup(app):
+    app.connect('builder-inited', builder_inited_handler)

docs/zh_cn/cp_origin_docs.sh

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+
+# Copy *.md files from docs/ if it doesn't have a Chinese translation
+
+for filename in $(find ../en/ -name '*.md' -printf "%P\n");
+do
+    mkdir -p $(dirname $filename)
+    cp -n ../en/$filename ./$filename
+done

docs/zh_cn/get_started.md

@@ -278,6 +278,10 @@ python run.py --models hf_llama_7b --datasets base_medium
 更多介绍可查看 [数据集配置](./user_guides/dataset_prepare.md)。
 ```
 
+```{warning}
+OpenCompass 在运行时通常需要依赖网络连接 HuggingFace 服务器 (https://huggingface.co/) 下载模型或数据集。如果连接遇到问题，或者需要离线运行评测，可以参考 [FAQ - Network - Q1](./faq.md#network)。
+```
+
 ### 启动评测
 
 配置文件准备完毕后，我们可以使用 debug 模式启动任务，以检查模型加载、数据集读取是否出现异常，如未正确读取缓存等。

docs/zh_cn/index.rst

@@ -25,6 +25,7 @@ OpenCompass 上手路线
    :caption: 开始你的第一步
 
    get_started.md
+   faq.md
 
 .. _教程:
 .. toctree::