Skip to content

Commit f96566a

Browse files
authored
Add PIAF (#774)
* add original prompts * update metrics
1 parent fb4c27d commit f96566a

File tree

1 file changed

+80
-0
lines changed

1 file changed

+80
-0
lines changed
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
dataset: piaf
2+
templates:
3+
4bac0d14-ac52-442c-9364-ea7add071af4: !Template
4+
answer_choices: null
5+
id: 4bac0d14-ac52-442c-9364-ea7add071af4
6+
jinja: 'Extract from the passage the answer to this question: {{question}}
7+
8+
Passage about {{title}}: {{context}} ||| {{answers[''text''][0]}}'
9+
metadata: !TemplateMetadata
10+
choices_in_prompt: false
11+
metrics:
12+
- Accuracy
13+
- Squad
14+
original_task: true
15+
name: extract_the_answer
16+
reference: ''
17+
723aa38b-c671-457c-96ee-bf449184f57f: !Template
18+
answer_choices: null
19+
id: 723aa38b-c671-457c-96ee-bf449184f57f
20+
jinja: '{{title}}
21+
22+
{{context}}
23+
24+
Q: {{question}}
25+
26+
27+
A: ||| {{answers.text[0]}}'
28+
metadata: !TemplateMetadata
29+
choices_in_prompt: false
30+
metrics:
31+
- Accuracy
32+
- Squad
33+
original_task: true
34+
name: context_follow_q
35+
reference: ''
36+
9f4bf5cd-070c-46a1-9098-2ad1ef63d806: !Template
37+
answer_choices: null
38+
id: 9f4bf5cd-070c-46a1-9098-2ad1ef63d806
39+
jinja: "Given the following passage answer the question that follows: \n{{context}}\n\
40+
{{question}} ||| {{answers.text[0]}}"
41+
metadata: !TemplateMetadata
42+
choices_in_prompt: false
43+
metrics:
44+
- Accuracy
45+
- Squad
46+
original_task: true
47+
name: given_passage_answer
48+
reference: ''
49+
bdeaa285-287f-44de-9bff-49dbf533be89: !Template
50+
answer_choices: null
51+
id: bdeaa285-287f-44de-9bff-49dbf533be89
52+
jinja: '{{context}}
53+
54+
55+
Given the above context, {{question}} ||| {{answers.text[0]}}'
56+
metadata: !TemplateMetadata
57+
choices_in_prompt: false
58+
metrics:
59+
- Accuracy
60+
- Squad
61+
original_task: true
62+
name: Given_above_context
63+
reference: ''
64+
ec70d9f8-edd5-4b8c-b18a-a5a7fb962b9a: !Template
65+
answer_choices: null
66+
id: ec70d9f8-edd5-4b8c-b18a-a5a7fb962b9a
67+
jinja: 'After reading the following paragraph, please answer the question that
68+
follows:
69+
70+
{{context}}
71+
72+
{{question}} ||| {{answers.text[0]}}'
73+
metadata: !TemplateMetadata
74+
choices_in_prompt: false
75+
metrics:
76+
- Accuracy
77+
- Squad
78+
original_task: true
79+
name: after_reading
80+
reference: ''

0 commit comments

Comments
 (0)