Skip to content

adding GEM/webnlg prompts #743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
May 8, 2022
114 changes: 114 additions & 0 deletions promptsource/templates/GEM/web_nlg/en/templates.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
dataset: GEM/web_nlg
subset: en
templates:
38342608-5cd7-4ce7-b2e1-905ecd7f4c80: !Template
answer_choices: null
id: 38342608-5cd7-4ce7-b2e1-905ecd7f4c80
jinja: "{{input | join(\"; \")}}\nThe above is a collection of relations. Write\
\ descriptive English that contains this information.\n{% for i in references\
\ %}\n ||| {{ i }} \n{% endfor %}"
metadata: !TemplateMetadata
choices_in_prompt: false
metrics:
- BLEU
- ROUGE
original_task: true
name: implicit-graph-description
reference: ''
3e41305c-5461-4cf3-853d-8a6fb5747623: !Template
answer_choices: null
id: 3e41305c-5461-4cf3-853d-8a6fb5747623
jinja: "I will verbalize an abstract representation of a sentence in natural language.\
\ To do so, I will first show the representation and then the natural language.\
\ The text needs to include all of the information in the representation.\n\n\
{{input | join(\", \")}} {% for i in references %}\n ||| {{ i }} \n{% endfor\
\ %}"
metadata: !TemplateMetadata
choices_in_prompt: false
metrics:
- BLEU
- ROUGE
original_task: true
name: PALM_prompt
reference: used-in-PALM
Comment on lines +18 to +33

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do we know this prompt was used in palm?
all i could find in their paper is the below ..
Screenshot 2022-08-15 at 22 11 45

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe @sebastianGehrmann shared some NLG prompts for PALM.

426b682e-e801-4e8d-9ac3-5b676c9d3da2: !Template
answer_choices: null
id: 426b682e-e801-4e8d-9ac3-5b676c9d3da2
jinja: "A semantic triple is the atomic data entity in the Resource Description\
\ Framework (RDF) data model. As its name indicates, a triple is a set of three\
\ entities that codifies a statement about semantic data in the form of subject\u2013\
predicate\u2013object expressions. (e.g., \"Bob | is | 35\", or \"Bob | knows\
\ | John\"). \n\nA graph can be formed from a set of these triples. An example\
\ is {{input | join(\", \")}}. \n\nWrite grammatical text expressing all the\
\ relations succinctly and fluently.\n{% for i in references %}\n ||| {{ i\
\ }} \n{% endfor %}\n\n"
metadata: !TemplateMetadata
choices_in_prompt: false
metrics:
- ROUGE
- BLEU
original_task: true
name: very-explicit-description
reference: ''
471c94de-2db2-48bf-a61f-89aedad19f56: !Template
answer_choices: null
id: 471c94de-2db2-48bf-a61f-89aedad19f56
jinja: "A semantic triple is a set of three entities that codifies a statement\
\ about semantic data in the form of subject\u2013predicate\u2013object expressions.\
\ (e.g., \"Bob | is | 35\", or \"Bob | knows | John\"). \n\nA graph to do with\
\ {{category}} can be formed from a set of these triples. An example is {{input\
\ | join(\", \")}}. \n\nWrite grammatical text expressing all the relations\
\ succinctly and fluently.\n{% for i in references %}\n ||| {{ i }} \n{% endfor\
\ %}\n\n"
metadata: !TemplateMetadata
choices_in_prompt: false
metrics:
- ROUGE
- BLEU
original_task: false
name: use-category
reference: ''
9415bd8a-685f-4fa4-803a-f09bd47d4603: !Template
answer_choices: null
id: 9415bd8a-685f-4fa4-803a-f09bd47d4603
jinja: "I am taking this tabular data, where each row is separated by a \"|\"\
\ and expresses a relation between an object and a predicate : {{input | join(\"\
, \")}}. \n\nNow, I will produce a description of the tabular data using English sentences. {%\
\ for i in references %}\n ||| {{ i }} \n{% endfor %}"
metadata: !TemplateMetadata
choices_in_prompt: false
metrics:
- ROUGE
- BLEU
original_task: true
name: non-explicit-description
reference: Data-to-Text
af519494-2a6a-4d83-a440-ee6d24df00e3: !Template
answer_choices: null
id: af519494-2a6a-4d83-a440-ee6d24df00e3
jinja: "I will take the following graph formed from relations (triples) separated\
\ by \"|\": {{input | join(\", \")}}. \n\nThen, I will verbalise the triple\
\ set into plain text. {% for i in references %}\n ||| {{ i }} \n{% endfor\
\ %}"
metadata: !TemplateMetadata
choices_in_prompt: false
metrics:
- BLEU
- ROUGE
original_task: true
name: explicit-graph-description
reference: 'Explicit Graph Description: verbalization.'
afeec167-f75f-4687-a775-1efde7d04780: !Template
answer_choices: null
id: afeec167-f75f-4687-a775-1efde7d04780
jinja: "{{input | join(\", \")}}. \n\nThe above is a set of subject | predicate\
\ | object expressions separated by commas: \nWrite all the information in proper\
\ sentences. {% for i in references %}\n ||| {{ i }} \n{% endfor %}"
metadata: !TemplateMetadata
choices_in_prompt: false
metrics:
- ROUGE
- BLEU
original_task: true
name: explicit-graph-description2
reference: " subject\u2013predicate\u2013object "
Loading