mirror of
https://github.com/open-compass/opencompass.git
synced 2025-05-30 16:03:24 +08:00
323 lines
17 KiB
Python
323 lines
17 KiB
Python
# flake8: noqa: E501
|
||
base_prompt_dict = {
|
||
'basic':
|
||
"""You will be presented with a causal graph in the following form: %s.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (%s, %s) in the above causal graph?
|
||
Option 1: %s
|
||
Option 2: %s
|
||
Option 3: %s
|
||
Answer (Option 1 or Option 2 or Option 3 ?):""",
|
||
'basic-CN':
|
||
"""给定如下因果图:%s。
|
||
问题:对于上述因果图中的有序变量对 (%s, %s),满足后门准则的最大变量集是哪个?
|
||
选项一:%s
|
||
选项二:%s
|
||
选项三:%s
|
||
答案(选项一或选项二或选项三?):
|
||
""",
|
||
'adversarial-ignore':
|
||
"""You will be presented with a causal graph in the following form: %s.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (%s, %s) in the above causal graph?
|
||
Option 1: %s
|
||
Option 2: %s
|
||
Option 3: %s
|
||
Answer (Option 1 or Option 2 or Option 3 ?):""",
|
||
'adversarial-ignore-CN':
|
||
"""给定如下因果图:%s。
|
||
问题:对于上述因果图中的有序变量对 (%s, %s),满足后门准则的最大变量集是哪个?
|
||
选项一:%s
|
||
选项二:%s
|
||
选项三:%s
|
||
答案(选项一或选项二或选项三?):
|
||
""",
|
||
'adversarial-doubt':
|
||
"""You will be presented with a causal graph in the following form: %s.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (%s, %s) in the above causal graph?
|
||
Option 1: %s
|
||
Option 2: %s
|
||
Option 3: %s
|
||
Answer (Option 1 or Option 2 or Option 3 ?):""",
|
||
'adversarial-doubt-CN':
|
||
"""给定如下因果图:%s。
|
||
问题:对于上述因果图中的有序变量对 (%s, %s),满足后门准则的最大变量集是哪个?
|
||
选项一:%s
|
||
选项二:%s
|
||
选项三:%s
|
||
答案(选项一或选项二或选项三?):
|
||
""",
|
||
'zero-shot-IcL':
|
||
"""Objective:
|
||
Given a causal graph with specified relationships between variables, your task is to identify the set of variables that satisfy the back-door criterion relative to an ordered pair of variables (X, Y) in that graph.
|
||
Background Information:
|
||
The back-door criterion is defined as follows:
|
||
1. The variable set Z must not contain any descendants of X.
|
||
2. The variable set Z must block every path from X to Y that has an arrow pointing to X.
|
||
Input:
|
||
- A textual description of the causal graph, detailing which variables cause which other variables.
|
||
- A question asking for the maximal set of variables that satisfy the back-door criterion for a specified ordered pair of variables (X, Y).
|
||
- Multiple choice options representing possible sets of variables that could satisfy the criterion.
|
||
Output:
|
||
- Answer to the question in the format "Option N", where N is 1, 2, or 3 based on the given options.
|
||
|
||
You will be presented with a causal graph in the following form: %s.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (%s, %s) in the above causal graph?
|
||
Option 1: %s
|
||
Option 2: %s
|
||
Option 3: %s
|
||
Answer (Option 1 or Option 2 or Option 3 ?):""",
|
||
'zero-shot-IcL-CN':
|
||
"""给定一个具有指定变量间关系的因果图,你的任务是找出相对于该图中有序的一对变量(X,Y),满足后门标准的一组变量。
|
||
背景信息:
|
||
后门准则的定义如下:
|
||
1. 变量集 Z 不能包含任何 X 的后代。
|
||
2. 变量集 Z 必须阻止每一条从 X 到 Y 的路径,这条路径必须有一个箭头指向 X。
|
||
输入
|
||
- 因果图的文字描述,详细说明哪些变量会导致哪些其他变量。
|
||
- 一个问题,要求找出满足指定有序变量对(X,Y)的后门标准的最大变量集。
|
||
- 代表可能满足该标准的变量集的多选选项。
|
||
输出:
|
||
- 问题答案,格式为 "选项 N",根据给定选项,N 为 一、二或 三。
|
||
|
||
给定如下因果图:%s。
|
||
问题:对于上述因果图中的有序变量对 (%s, %s),满足后门准则的最大变量集是哪个?
|
||
选项一:%s
|
||
选项二:%s
|
||
选项三:%s
|
||
答案(选项一或选项二或选项三?):""",
|
||
'one-shot-IcL':
|
||
"""Objective:
|
||
Given a causal graph with specified relationships between variables, your task is to identify the set of variables that satisfy the back-door criterion relative to an ordered pair of variables (X, Y) in that graph.
|
||
Background Information:
|
||
The back-door criterion is defined as follows:
|
||
1. The variable set Z must not contain any descendants of X.
|
||
2. The variable set Z must block every path from X to Y that has an arrow pointing to X.
|
||
Input:
|
||
- A textual description of the causal graph, detailing which variables cause which other variables.
|
||
- A question asking for the maximal set of variables that satisfy the back-door criterion for a specified ordered pair of variables (X, Y).
|
||
- Multiple choice options representing possible sets of variables that could satisfy the criterion.
|
||
Output:
|
||
- Answer to the question in the format "Option N", where N is 1, 2, or 3 based on the given options.
|
||
|
||
Example:
|
||
You will be presented with a causal graph in the following form: A causes D, A causes E, B causes E, C causes D, and D causes E.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (A, E) in the above causal graph?
|
||
Option 1: E, B
|
||
Option 2: B, C
|
||
Option 3: E, D
|
||
Answer (Option 1 or Option 2 or Option 3 ?): Option 2
|
||
|
||
New Input:
|
||
You will be presented with a causal graph in the following form: %s.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (%s, %s) in the above causal graph?
|
||
Option 1: %s
|
||
Option 2: %s
|
||
Option 3: %s
|
||
Answer (Option 1 or Option 2 or Option 3 ?):""",
|
||
'one-shot-IcL-CN':
|
||
"""给定一个具有指定变量间关系的因果图,你的任务是找出相对于该图中有序的一对变量(X,Y),满足后门标准的一组变量。
|
||
背景信息:
|
||
后门准则的定义如下:
|
||
1. 变量集 Z 不能包含任何 X 的后代。
|
||
2. 变量集 Z 必须阻止每一条从 X 到 Y 的路径,这条路径必须有一个箭头指向 X。
|
||
输入
|
||
- 因果图的文字描述,详细说明哪些变量会导致哪些其他变量。
|
||
- 一个问题,要求找出满足指定有序变量对(X,Y)的后门标准的最大变量集。
|
||
- 代表可能满足该标准的变量集的多选选项。
|
||
输出:
|
||
- 问题答案,格式为 "选项 N",根据给定选项,N 为 一、二或 三。
|
||
|
||
例子:
|
||
给定如下因果图:A导致D, A导致E, B导致E, C导致D, 以及D导致E。
|
||
问题:对于上述因果图中的有序变量对 (A, E),满足后门准则的最大变量集是哪个?
|
||
选项一:E, B
|
||
选项二:B, C
|
||
选项三:E, D
|
||
答案(选项一或选项二或选项三?):选项二
|
||
|
||
新输入:
|
||
给定如下因果图:%s。
|
||
问题:对于上述因果图中的有序变量对 (%s, %s),满足后门准则的最小变量集是哪个?
|
||
选项一:%s
|
||
选项二:%s
|
||
选项三:%s
|
||
答案(选项一或选项二或选项三?):""",
|
||
'three-shot-IcL':
|
||
"""Objective:
|
||
Given a causal graph with specified relationships between variables, your task is to identify the set of variables that satisfy the back-door criterion relative to an ordered pair of variables (X, Y) in that graph.
|
||
Background Information:
|
||
The back-door criterion is defined as follows:
|
||
1. The variable set Z must not contain any descendants of X.
|
||
2. The variable set Z must block every path from X to Y that has an arrow pointing to X.
|
||
Input:
|
||
- A textual description of the causal graph, detailing which variables cause which other variables.
|
||
- A question asking for the maximal set of variables that satisfy the back-door criterion for a specified ordered pair of variables (X, Y).
|
||
- Multiple choice options representing possible sets of variables that could satisfy the criterion.
|
||
Output:
|
||
- Answer to the question in the format "Option N", where N is 1, 2, or 3 based on the given options.
|
||
|
||
Example:
|
||
You will be presented with a causal graph in the following form: A causes D, A causes E, B causes E, C causes D, and D causes E.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (A, E) in the above causal graph?
|
||
Option 1: E, B
|
||
Option 2: B, C
|
||
Option 3: E, D
|
||
Answer (Option 1 or Option 2 or Option 3 ?): Option 2
|
||
|
||
You will be presented with a causal graph in the following form: A causes D, A causes C, B causes E, C causes D, and D causes E.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (B, E) in the above causal graph?
|
||
Option 1: D, C
|
||
Option 2: E, A
|
||
Option 3: A, C
|
||
Answer (Option 1 or Option 2 or Option 3 ?): Option 1
|
||
|
||
You will be presented with a causal graph in the following form: A causes E, A causes D, B causes D, B causes E, C causes E, and D causes E.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (A, E) in the above causal graph?
|
||
Option 1: B, D
|
||
Option 2: A, C
|
||
Option 3: B, C
|
||
Answer (Option 1 or Option 2 or Option 3 ?): Option 3
|
||
|
||
New Input:
|
||
You will be presented with a causal graph in the following form: %s.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (%s, %s) in the above causal graph?
|
||
Option 1: %s
|
||
Option 2: %s
|
||
Option 3: %s
|
||
Answer (Option 1 or Option 2 or Option 3 ?):""",
|
||
'three-shot-IcL-CN':
|
||
"""给定一个具有指定变量间关系的因果图,你的任务是找出相对于该图中有序的一对变量(X,Y),满足后门标准的一组变量。
|
||
背景信息:
|
||
后门准则的定义如下:
|
||
1. 变量集 Z 不能包含任何 X 的后代。
|
||
2. 变量集 Z 必须阻止每一条从 X 到 Y 的路径,这条路径必须有一个箭头指向 X。
|
||
输入
|
||
- 因果图的文字描述,详细说明哪些变量会导致哪些其他变量。
|
||
- 一个问题,要求找出满足指定有序变量对(X,Y)的后门标准的最大变量集。
|
||
- 代表可能满足该标准的变量集的多选选项。
|
||
输出:
|
||
- 问题答案,格式为 "选项 N",根据给定选项,N 为 一、二或 三。
|
||
|
||
例子:
|
||
给定如下因果图:A导致D, A导致E, B导致E, C导致D, 以及D导致E。
|
||
问题:对于上述因果图中的有序变量对 (A, E),满足后门准则的最大变量集是哪个?
|
||
选项一:E, B
|
||
选项二:B, C
|
||
选项三:E, D
|
||
答案(选项一或选项二或选项三?):选项二
|
||
|
||
|
||
给定如下因果图:A导致D, A导致C, B导致E, C导致D, 以及D导致E。
|
||
问题:对于上述因果图中的有序变量对 (B, E),满足后门准则的最小变量集是哪个?
|
||
选项一:D, C
|
||
选项二:E, A
|
||
选项三:A, C
|
||
答案(选项一或选项二或选项三?):选项一
|
||
|
||
给定如下因果图:A导致E, A导致D, B导致D, B导致E, C导致E, 以及D导致E。
|
||
问题:对于上述因果图中的有序变量对 (A, E),满足后门准则的最小变量集是哪个?
|
||
选项一:B, D
|
||
选项二:A, C
|
||
选项三:B, C
|
||
答案(选项一或选项二或选项三?):选项三
|
||
|
||
新输入:
|
||
给定如下因果图:%s。
|
||
问题:对于上述因果图中的有序变量对 (%s, %s),满足后门准则的最小变量集是哪个?
|
||
选项一:%s
|
||
选项二:%s
|
||
选项三:%s
|
||
答案(选项一或选项二或选项三?):""",
|
||
'zero-shot-CoT':
|
||
"""You will be presented with a causal graph in the following form: %s.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (%s, %s) in the above causal graph? Let's think step by step.
|
||
Option 1: %s
|
||
Option 2: %s
|
||
Option 3: %s
|
||
Answer (Option 1 or Option 2 or Option 3 ?):""",
|
||
'zero-shot-CoT-CN':
|
||
"""给定如下因果图:%s。
|
||
问题:对于上述因果图中的有序变量对 (%s, %s),满足后门准则的最大变量集是哪个?请逐步思考。
|
||
选项一:%s
|
||
选项二:%s
|
||
选项三:%s
|
||
答案(选项一或选项二或选项三?):""",
|
||
'manual-CoT':
|
||
"""Here are three examples of finding the maximal backdoor adjustment set using chain of thought, and a question to answer. Note A is unobserved in the following questions.
|
||
|
||
You will be presented with a causal graph in the following form: A causes B, A causes D, A causes E, B causes E, B causes D, B causes C, C causes D, C causes E, D causes F, and D causes E.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (D, F) in the above causal graph?
|
||
Option 1: D, A
|
||
Option 2: D, F
|
||
Option 3: B, C
|
||
Answer (Option 1 or Option 2 or Option 3 ?): Since E is a descendant of D and A is unobserved, and there is no path between D and F containing an arrow pointing into D, thus the maximal backdoor adjust set is {B, C}. Therefore, the answer is Option 3.
|
||
|
||
You will be presented with a causal graph in the following form: A causes G, A causes C, B causes F, B causes D, B causes E, B causes G, C causes E, D causes F, D causes G, E causes G, and E causes F.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (A, E) in the above causal graph?
|
||
Option 1: E, F
|
||
Option 2: F, A
|
||
Option 3: B, D
|
||
Answer (Option 1 or Option 2 or Option 3 ?): Since C, G, E and F are all the descendants of A, and there is no path between A and E with an arrow pointing into A, thus the maximal backdoor adjustment set is {B, D}. Therefore, the answer is Option 3.
|
||
|
||
You will be presented with a causal graph in the following form: A causes H, A causes C, A causes E, A causes G, A causes B, B causes D, C causes D, C causes E, D causes F, D causes G, and F causes H.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (B, D) in the above causal graph?
|
||
Option 1: C, E
|
||
Option 2: E, D
|
||
Option 3: C, H
|
||
Answer (Option 1 or Option 2 or Option 3 ?): Since F, G and H are all the descendants of B, and C and E block all the paths from B to D with an arrow pointing into B, thus the maximal backdoor adjustment set is {C,E}. Therefore, the answer is Option 1.
|
||
|
||
You will be presented with a causal graph in the following form: %s.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (%s, %s) in the above causal graph?
|
||
Option 1: %s
|
||
Option 2: %s
|
||
Option 3: %s
|
||
Answer (Option 1 or Option 2 or Option 3 ?):""",
|
||
'manual-CoT-CN':
|
||
"""如下为两个使用思维链进行推理的判断最大后门变量集合的示例,和一个需要回答的问题。注意在以下的问题设定中,A为未观测到的变量。
|
||
|
||
给定如下因果图:A导致B, A导致D, A导致E, B导致E, B导致D, B导致C, C导致D, C导致E, D导致F, 以及D导致E。
|
||
问题:对于上述因果图中的有序变量对 (D, F),满足后门准则的最大变量集是哪个?
|
||
选项一:D, A
|
||
选项二:D, F
|
||
选项三:B, C
|
||
答案(选项一或选项二或选项三?):由于E是D的后代,A为未观测到的变量,D和F之间没有箭头指向D的路径,所以最大后门调整集是 {B,C}。因此答案为选项三。
|
||
|
||
给定如下因果图:A导致G, A导致C, B导致F, B导致D, B导致E, B导致G, C导致E, D导致F, D导致G, E导致G, 以及E导致F。
|
||
问题:对于上述因果图中的有序变量对 (A, E),满足后门准则的最大变量集是哪个?
|
||
选项一:E, F
|
||
选项二:F, A
|
||
选项三:B, D
|
||
答案(选项一或选项二或选项三?):由于C、G、E和F都是A的后代,A和E之间没有箭头指向A的路径,所以最大后门调整集是 {B,D}。因此答案为选项三。
|
||
|
||
给定如下因果图:%s。
|
||
问题:对于上述因果图中的有序变量对 (%s, %s),满足后门准则的最大变量集是哪个?
|
||
选项一:%s
|
||
选项二:%s
|
||
选项三:%s
|
||
答案(选项一或选项二或选项三?):""",
|
||
'explicit-function':
|
||
"""You are a helpful assistant for adjustment set analysis (back-door criterion).
|
||
You will be presented with a causal graph in the following form: %s.
|
||
Question: Which is the maximal set of variables that satisfies the back-door criterion relative to an ordered pair of variables (%s, %s) in the above causal graph?
|
||
Option 1: %s
|
||
Option 2: %s
|
||
Option 3: %s
|
||
Answer (Option 1 or Option 2 or Option 3 ?):""",
|
||
'explicit-function-CN':
|
||
"""你是一个用于调整集分析(后门准则)的得力助手。
|
||
给定如下因果图:%s。
|
||
问题:对于上述因果图中的有序变量对 (%s, %s),满足后门准则的最大变量集是哪个?
|
||
选项一:%s
|
||
选项二:%s
|
||
选项三:%s
|
||
答案(选项一或选项二或选项三?):
|
||
""",
|
||
}
|
||
|
||
|
||
def get_prompt(task_name, prompt_style, item, prompt_style_str=''):
|
||
base = base_prompt_dict[prompt_style]
|
||
prompt = prompt_style_str + base % (item['edges'], item['treatment'],
|
||
item['outcome'], item['option1'],
|
||
item['option2'], item['option3'])
|
||
return prompt
|