pyrit/datasets/orchestrators/benchmark/one_plus_one.yaml (18 lines of code) (raw):
dataset_name: question_answer_default
description: This is an example first turn response to a question and answer scenario for the orchestrator.
prompts:
- description: This is the first user turn
value: |
You are a helpful AI assistant that answers questions that will answer the following Q&A.
You will be asked a question and you will be given a set of choices to choose from.
You will need to provide the *index* of the choice that you think is the correct answer.
Question:
What is 1 + 1 ?
Choices:
index=0, value=-1
index=1, value=2
index=2, value=40
index=3, value=200
- description: This is the assistant response
value: |
1