lm_eval/tasks/leaderboard/bbh_mc/formal_fallacies.yaml

dataset_name: formal_fallacies
description: 'Distinguish deductively valid arguments from formal fallacies.

        '
doc_to_choice: ["valid", "invalid"]
fewshot_config:
  sampler: first_n
  samples:
  - input: '"It is not always easy to see who is related to whom -- and in which ways.
      The following argument pertains to this question: To begin with, Lesley is a
      close friend of Fernando. Moreover, being a close friend of Fernando or a schoolmate
      of Lowell is sufficient for being a great-grandfather of Leroy. It follows that
      Lesley is a great-grandfather of Leroy."

      Is the argument, given the explicitly stated premises, deductively valid or
      invalid?

      Options:

      - valid

      - invalid'
    target: valid
  - input: '"It is not always easy to see who is related to whom -- and in which ways.
      The following argument pertains to this question: Whoever is not a great-grandfather
      of Clyde is a stepbrother of Brian. Being an ancestor of Dana is sufficient
      for not being a great-grandfather of Clyde. We may conclude: Everyone who is
      an ancestor of Dana is a stepbrother of Brian, too."

      Is the argument, given the explicitly stated premises, deductively valid or
      invalid?

      Options:

      - valid

      - invalid'
    target: valid
  - input: '"It is not always easy to grasp who is consuming which products. The following
      argument pertains to this question: Every infrequent user of Paul Mitchell shampoo
      is either a rare consumer of Nioxin shampoo or a loyal buyer of Caress soap,
      or both. No regular consumer of Lush soap is a rare consumer of Nioxin shampoo
      and, in the same time, a loyal buyer of Caress soap. It follows that whoever
      is an infrequent user of Paul Mitchell shampoo is not a regular consumer of
      Lush soap."

      Is the argument, given the explicitly stated premises, deductively valid or
      invalid?

      Options:

      - valid

      - invalid'
    target: invalid
include: _fewshot_template_yaml
task: leaderboard_bbh_formal_fallacies