adding GEM/webnlg prompts #743
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adding multi-target + PALM prompt
adding PALM and multi-target
|
I've added Multi-refs |
|
Hey @jordiclive ! When I run this I get a Example: For import datasets
from promptsource.templates import DatasetTemplates
template = DatasetTemplates("GEM/web_nlg", "ru")
data = datasets.load_dataset("GEM/web_nlg", "ru")
example = data['validation'][0]
prompt = template['PALM_prompt'].apply(example)
print(prompt) |
|
@jon-tow, I think you were not working with the newest eval-hackathon which has the multi-ref changes. I have now merged that into this . |
|
Prompts look good to me. Do we have someone who could translate some of the Russian prompts so we don't only rely on English? |
Adding Russian versions of Prompts.
Muennighoff
reviewed
Aug 15, 2022
Comment on lines
+18
to
+33
| 3e41305c-5461-4cf3-853d-8a6fb5747623: !Template | ||
| answer_choices: null | ||
| id: 3e41305c-5461-4cf3-853d-8a6fb5747623 | ||
| jinja: "I will verbalize an abstract representation of a sentence in natural language.\ | ||
| \ To do so, I will first show the representation and then the natural language.\ | ||
| \ The text needs to include all of the information in the representation.\n\n\ | ||
| {{input | join(\", \")}} {% for i in references %}\n ||| {{ i }} \n{% endfor\ | ||
| \ %}" | ||
| metadata: !TemplateMetadata | ||
| choices_in_prompt: false | ||
| metrics: | ||
| - BLEU | ||
| - ROUGE | ||
| original_task: true | ||
| name: PALM_prompt | ||
| reference: used-in-PALM |
There was a problem hiding this comment.
how do we know this prompt was used in palm?
all i could find in their paper is the below ..
There was a problem hiding this comment.
I believe @sebastianGehrmann shared some NLG prompts for PALM.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
GEM/webnlg houses all constituents, and challenge sets. Also added Russian prompts with minimal changes, as input triple set is still english.