（空气呼啸） (air whooshing)
（笑脸弹出） (smiley face popping)
（空气呼啸） (air whooshing)
使用 PyTorch 编写你自己的训练循环。 Write your own training loop with PyTorch.
在本视频中，我们将了解 In this video, we'll look at
我们如何进行与训练器视频中相同的微调， how we can do the same fine-tuning as in the Trainer video,
但不依赖那个课程。 but without relying on that class.
这样，你就可以根据你的需要轻松自定义 This way, you'll be able to easily customize each step
训练循环的每个步骤。 to the training loop to your needs.
这个也很有用 This is also very useful
对于手动调试 to manually debug something
Trainer API 出现的问题。 that went wrong with the Trainer API.
在我们深入研究代码之前， Before we dive into the code,
这是训练循环的草图。 here is a sketch of a training loop.
我们获取一批训练数据并将其提供给模型。 We take a batch of training data and feed it to the model.
有了标签，我们就可以计算损失。 With the labels, we can then compute a loss.
这个数字本身没有用， That number is not useful in its own,
它是用于计算 but is used to compute
我们模型权重的成分， the ingredients of our model weights,
即，损失关于 that is the derivative of the loss
每个模型权重的导数。 with respect to each model weight.
然后优化器使用这些梯度 Those gradients are then used by the optimizer
更新模型权重， to update the model weights,
让他们变得更好一点。 and make them a little bit better.
然后我们重复这个过程 We then repeat the process
使用一批新的训练数据。 with a new batch of training data.
如果有任何不清楚的地方， If any of this isn't clear,
不要犹豫，复习一下 don't hesitate to take a refresher
在你最喜欢的深度学习课程上。 on your favorite deep learning course.
我们将在这里再次使用 GLUE MRPC 数据集， We'll use the GLUE MRPC data set here again,
我们已经看到了如何预先提出数据 and we've seen how to prepropose the data
使用具有动态填充的数据集库。 using the Datasets library with dynamic padding.
查看下面的视频链接 Check out the videos link below
如果你还没有看过它们。 if you haven't seen them already.
完成后，我们只需要定义 PyTorch DataLoaders With this done, we only have to define PyTorch DataLoaders
这将负责转换 which will be responsible to convert
我们数据集的元素到批次数据中。 the elements of our dataset into batches.
我们将 DataColletorForPadding 用作整理函数， We use our DataColletorForPadding as a collate function,
并打乱训练集的次序 and shuffle the training set
确保我们不会每个纪元 to make sure we don't go over the samples
以相同的顺序遍历样本。 in the same order at a epoch*.
要检查一切是否按预期工作， To check that everything works as intended,
我们尝试获取一批数据，并对其进行检查。 we try to grab a batch of data, and inspect it.
就像我们的数据集元素一样，它是一个字典， Like our dataset elements, it's a dictionary,
但这里的值不是一个整数列表 but this times the values are not a single list of integers
而是形状为批量大小乘以序列长度的张量。 but a tensor of shape batch size by sequence length.
下一步是在我们的模型中发送训练数据。 The next step is to send the training data in our model.
为此，我们需要实际创建一个模型。 For that, we'll need to actually create a model.
如模型 API 视频中所示， As seen in the Model API video,
我们使用 from_pretrained 方法， we use the from_pretrained method,
并将标签数量调整为这个数据集 and adjust the number of labels to the number of classes
拥有的类别数量，这里是 2。 we have on this data set, here two.
再次确保一切顺利， Again to be sure everything is going well,
我们将我们抓取的批次传递给我们的模型， we pass the batch we grabbed to our model,
并检查没有错误。 and check there is no error.
如果提供了标签， If the labels are provided,
Transformers 库的模型 the models of the Transformers library
总是直接返回损失。 always returns a loss directly.
我们将能够做损失的反向传播 We will be able to do loss.backward ()
以计算所有梯度， to compute all the gradients,
然后需要一个优化器来完成训练步骤。 and will then need an optimizer to do the training step.
我们在这里使用 AdamW 优化器， We use the AdamW optimizer here,
这是具有适当权重衰减的 Adam 变体， which is a variant of Adam with proper weight decay,
但你可以选择任何你喜欢的 PyTorch 优化器。 but you can pick any PyTorch optimizer you like.
使用之前的损失， Using the previous loss,
并使用 loss.backward() 计算梯度， and computing the gradients with loss.backward (),
我们检查我们是否可以无误 we check that we can do the optimizer step
执行优化器步骤。 without any error.
之后不要忘记将梯度归零， Don't forget to zero your gradient afterwards,
或者在下一步， or at the next step,
它们将被添加到你计算的梯度中。 they will get added to the gradients you computed.
我们已经可以编写我们的训练循环， We could already write our training loop,
但我们还要再做两件事 but we will add two more things
使它尽可能好。 to make it as good as it can be.
第一个是学习率调度器， The first one is a learning rate scheduler,
逐步将我们的学习率降低到零。 to progressively decay our learning rate to zero.
Transformers 库中的 get_scheduler 函数 The get_scheduler function from the Transformers library
只是一个方便的功能 is just a convenience function
轻松构建这样的调度器。 to easily build such a scheduler.
你可以再次使用 You can again use
取而代之的是任何 PyTorch 学习率调度器。 any PyTorch learning rate scheduler instead.
最后，如果我们想要我们的培训 Finally, if we want our training
花几分钟而不是几个小时， to take a couple of minutes instead of a few hours,
我们需要使用 GPU。 we will need to use a GPU.
第一步是得到一个， The first step is to get one,
例如通过使用协作笔记本。 for instance by using a colab notebook.
然后你需要实际发送你的模型， Then you need to actually send your model,
并使用 torch 设备对其进行训练数据。 and training data on it by using a torch device.
仔细检查以下代码 Double-check the following lines
为你打印一个 CUDA 设备。 print a CUDA device for you.
或准备将你的训练减少到一个多小时。 or be prepared for your training to less, more than an hour.
我们现在可以把所有东西放在一起。 We can now put everything together.
首先，我们将模型置于训练模式 First, we put our model in training mode
这将激活训练行为 which will activate the training behavior
对于某些层，例如 Dropout。 for some layers, like Dropout.
然后遍历我们选择的纪元数， Then go through the number of epochs we picked,
以及我们训练数据加载器中的所有数据。 and all the data in our training dataloader.
然后我们完成我们已经看到的所有步骤； Then we go through all the steps we have seen already;
将数据发送到 GPU， send the data to the GPU,
计算模型输出， compute the model outputs,
尤其是损失。 and in particular the loss.
使用损失来计算梯度， Use the loss to compute gradients,
然后使用优化器进行训练。 then make a training step with the optimizer.
在我们的调度器中更新学习率 Update the learning rate in our scheduler
对于下一次迭代， for the next iteration,
并将优化器的梯度归零。 and zero the gradients of the optimizer.
一旦完成， Once this is finished,
我们可以很容易地评估我们的模型 we can evaluate our model very easily
使用数据集库中的指标。 with a metric from the Datasets library.
首先，我们将模型置于评估模式， First, we put our model in evaluation mode,
停用像 Dropout 这样的层， to deactivate layers like Dropout,
然后遍历评估数据加载器中的所有数据。 then go through all the data in the evaluation data loader.
正如我们在训练器视频中看到的那样， As we have seen in the Trainer video,
模型输出逻辑斯蒂， the model outputs logits,
我们需要应用 argmax 函数 and we need to apply the argmax function
将它们转化为预测。 to convert them into predictions.
然后度量对象有一个 add_batch 方法， The metric object then has an add_batch method,
我们可以用来向它发送那些中间预测。 we can use to send it those intermediate predictions.
一旦评估循环完成， Once the evaluation loop is finished,
我们只需要调用计算方法 we just have to call the compute method
得到我们的最终结果。 to get our final results.
恭喜，你现在已经微调了一个模型 Congratulations, you have now fine-tuned a model
全靠你自己。 all by yourself.
（空气呼啸） (air whooshing)