（空气呼啸） (air whooshing)
- 如何预处理成对的句子？ - How to preprocess pairs of sentences?
我们已经看到了如何标记单个句子 We have seen how to tokenize single sentences
并将它们一起批处理 and batch them together
在 “Batching inputs together” 视频中。 in the "Batching inputs together" video.
如果你觉得这段代码不熟悉， If this code looks unfamiliar to you,
请务必再次检查该视频！ be sure to check that video again!
在这里，我们将专注于任务 Here, we will focus on tasks
对句子对进行分类。 that classify pairs of sentences.
例如，我们可能想要分类 For instance, we may want to classify
两个文本是否是释义。 whether two texts are paraphrases or not.
这是一个例子 Here is an example taken
来自 Quora Question Pairs 数据集， from the Quora Question Pairs dataset,
它侧重于识别重复的问题。 which focuses on identifying duplicate questions.
在第一对中，两个问题是重复的； In the first pair, the two questions are duplicates;
在第二对中，他们不是。 in the second, they are not.
另一个分类问题 Another classification problem
是当我们想知道是否有两个句子 is when we want to know if two sentences
逻辑上相关，或反之 are logically related or not,
一个称为自然语言推理或 NLI 的问题。 a problem called Natural Language Inference or NLI.
在这个取自 MultiNLI 数据集的例子中， In this example taken from the MultiNLI dataset,
对于每个可能的标签，我们都有一对句子： we have a pair of sentences for each possible label:
矛盾，中性或蕴涵， contradiction, neutral or entailment,
这是第一句话的奇特表达方式 which is a fancy way of saying the first sentence
暗示第二种。 implies the second.
所以对句子对进行分类 So classifying pairs of sentences
是一个值得研究的问题。 is a problem worth studying.
事实上，在 GLUE 基准测试中， In fact, in the GLUE benchmark,
这是文本分类的学术基准， which is an academic benchmark for text classification,
10 个数据集中有 8 个专注于任务 eight of the 10 datasets are focused on tasks
使用成对的句子。 using pairs of sentences.
这就是为什么像 BERT 这样的模型经常被预训练的原因 That's why models like BERT are often pretrained
具有双重目标： with a dual objective:
在语言建模目标之上， on top of the language modeling objective,
他们通常有一个与句子对相关的目标。 they often have an objective related to sentence pairs.
例如，在预训练期间， For instance, during pretraining,
BERT 显示成对的句子 BERT is shown pairs of sentences
并且必须预测随机屏蔽 token 的值 and must predict both the value of randomly masked tokens
以及第二句是否 and whether the second sentence
从第一个开始。 follows from the first or not.
幸运的是，来自 Transformers 库的 tokenizer Fortunately, the tokenizer from the Transformers library
有一个很好的 API 来处理成对的句子： has a nice API to deal with pairs of sentences:
你只需要将它们作为两个参数传递 you just have to pass them as two arguments
到 tokenizer 。 to the tokenizer.
在输入 ID 之上 On top of the input IDs
和我们已经研究过的注意力掩码， and the attention mask we studied already,
它返回一个名为 token 类型 ID 的新字段， it returns a new field called token type IDs,
它告诉模型哪些 token which tells the model which tokens
属于第一句 belong to the first sentence
哪些属于第二句。 and which ones belong to the second sentence.
放大一点，这里是输入 ID， Zooming in a little bit, here are the input IDs,
与它们对应的 token 对齐， aligned with the tokens they correspond to,
它们各自的 token 类型 ID 和注意掩码。 their respective token type ID and attention mask.
我们可以看到 tokenizer 还添加了特殊 token We can see the tokenizer also added special tokens
所以我们有一个 CLS token ，来自第一句话的 token ， so we have a CLS token, the tokens from the first sentence,
一个 SEP token ，第二句话中的 token ， a SEP token, the tokens from the second sentence,
和最终的 SEP token 。 and a final SEP token.
如果我们有几对句子， If we have several pairs of sentences,
我们可以将它们标记在一起 we can tokenize them together
通过传递第一句话的列表， by passing the list of first sentences,
然后是第二句话的列表 then the list of second sentences
以及我们已经研究过的所有关键字参数， and all the keyword arguments we studied already,
像 padding=True 。 like padding=True.
放大结果， Zooming in at the result,
我们可以看到 tokenizer 是如何添加填充的 we can see how the tokenizer added padding
对于第二对句子， to the second pair of sentences,
使两个输出的长度相同。 to make the two outputs the same length.
它还可以正确处理 token 类型 IDS It also properly dealt with token type IDS
和两个句子的注意力掩码。 and attention masks for the two sentences.
然后就可以通过我们的模型了！ This is then all ready to pass through our model!
（空气呼啸） (air whooshing)