（空气呼啸） (air whooshing)
- 在这段视频中，我们将看到 - In this video, we're going to see
如何加载和微调预训练模型。 how to load and fine tune a pre-trained model.
它非常快。 It's very quick.
如果你看过我们关于 pipeline 的视频， And if you've watched our pipeline videos,
我将在下面链接之，过程非常相似。 which I'll link below, the process is very similar.
不过这一次，我们将使用迁移学习 This time, though, we're going to be using transfer learning
并自己做一些训练， and doing some training ourselves,
而不是仅仅加载模型并按原样使用它。 rather than just loading a model and using it as is.
所以要了解更多关于迁移学习的信息， So to learn more about transfer learning,
前往 “What is transfer learning?” 视频， head to the 'What is transfer learning?' video,
我们也会在下面链接它。 and we'll link that below as well.
但现在，让我们看看这段代码。 But for now, let's look at this code.
首先，我们选择我们想要开始的模型。 To start, we pick which model we want to start with.
在这种情况下，我们将使用著名的， In this case, we're going to use the famous,
原始的 BERT， the original BERT,
作为我们今天训练的基础。 as the foundation for our training today.
但这条怪物般的一行是什么， But what is this monstrosity line,
这个 “TFAutoModelForSequenceClassification”？ this 'TFAutoModelForSequenceClassification'?
这意味着什么？ What does that mean?
好吧，TF 代表 TensorFlow。 Well, the TF stands for TensorFlow.
剩下的意味着， And the rest means,
采用语言模型， take a language model,
并在上面贴上一个序列分类头 and stick a sequence classification head onto it
如果它还没有的话。 if it doesn't have one already.
所以这行代码加载了 BERT， So this line of code loads BERT,
这是一个通用语言模型， which is a general purpose language model,
它加载重量、架构和所有 it loads at weights, architecture, and all
然后在上面添加一个新的序列分类头 and then adds a new sequence classification head onto it
具有随机初始化的权重。 with randomly initialized weights.
所以这个方法需要知道两件事。 So this method needs to know two things.
首先，它需要知道模型的名称 Firstly, it needs to know the name of the model
你想要加载的架构和权重。 you wanted to load, the architecture and weights for.
其次，它需要知道有多少类 And secondly, it needs to know how many classes
你的问题是有的，因为这将决定大小， your problem has, because that will determine the size,
对输出头中的神经元数量。 the number of neurons in the output head.
所以如果你想跟随数据 So if you want to follow along with the data
来自我们有关数据集的视频，我将在下面链接， from our datasets videos, which I'll link below,
那么你将有两个类别，积极的和消极的， then you'll have two classes, positive and negative,
因此 num_labels 等于二。 and thus num_labels equals two.
但是这个编译行呢？ But what about this compile line?
好吧，如果你熟悉 Keras， Well, if you're familiar with Keras,
你可能已经看过了。 you've probably seen this already.
但如果不是，这是 Keras 中的核心方法之一 But if not, this is one of the core methods in Keras
你会一次又一次地看到。 that you're gonna see again, and again.
你总是需要在训练之前编译你的模型。 You always need to compile your model before you train it.
编译需要知道两件事。 And compile needs to know two things.
首先，它需要知道损失函数， Firstly, it needs to know the loss function,
这就是你要优化的内容。 which is what you're trying to optimize.
所以在这里，我们导入 SparseCategoricalCrossentropy So here, we import the SparseCategoricalCrossentropy
损失函数。 loss function.
所以这是一点点，但它是标准的损失函数 So that's a mouthful, but it's the standard loss function
对于任何正在执行分类任务的神经网络。 for any neural network that's doing a classification task.
它基本上鼓励网络 It basically encourages the network
为正确的类输出大的值， to output large values for the right class,
以及为错误的类别输出低值。 and low values for the wrong classes.
请注意，你可以将损失函数指定为字符串， Note that you can specify the loss function as a string,
就像我们对优化器所做的那样。 like we did with the optimizer.
但这里有一个问题， But there's a risk there,
人们容易掉入一个非常普遍的陷阱， there's a very common trap people fall into,
就是默认情况下，这种损失假设 which is that by default, this loss assumes
输出是 softmax 层之后的概率。 the output is probabilities after a softmax layer.
但是我们的模型实际输出了什么 But what our model has actually output
是 softmax 之前的值， is the values before the softmax,
通常称为 logits，有时称为 logits。 often called the logits, sometimes logits.
没有人十分确定如何发音。 No one's quite sure how to pronounce that one.
但你可能以前见过这些 But you probably seen these before
在关于 pipeline 的视频中。 in the video about pipelines.
所以如果你弄错了，你的模型就不会训练 So if you get this wrong, your model won't train
弄清楚原因会很烦人。 and it'll be very annoying to figure out why.
在以后的视频中，我们会看到 In future videos, we're gonna see
如何使用模型的内部损失计算， how to use the model's internal loss computations,
这样你就不必自己指定损失 so that you don't have to specify the loss yourself
而且你不必担心这些细节。 and you don't have to worry about these details.
但是现在，请记住将 from_logits 设置为 true。 But for now, remember to set from_logits to true.
编译需要知道的第二件事 The second thing compile needs to know
是你想要的优化器。 is the optimizer you want.
在我们的例子中，我们使用 adam ， In our case, we use adam,
这是一种标准的优化器 which is sort of the standard optimizer
用于现代深度学习。 for deep learning these days.
你可能想要改变的一件事 The one thing you might want to change
是学习率。 is the learning rate.
为此，我们需要导入实际的优化器 And to do that, we'll need to import the actual optimizer
而不仅仅是通过字符串调用它。 rather than just calling it by string.
但我们会在另一个视频中讨论这个， But we'll talk about that in another video,
我将在下面链接。 which I'll link below.
现在，让我们尝试训练模型。 For now, let's just try training the model.
那么，你如何训练模型呢？ Well, so how do you train the model?
同样，如果你以前使用过 Keras， Again, if you've used Keras before,
这一切对你来说都很熟悉。 this is all going to be very familiar to you.
但如果没有，让我们快速看看 But if not, let's very quickly look
我们在这里做什么。 at what we're doing here.
fit 几乎是 Keras 模型的核心方法。 fit is pretty much the central method for Keras models.
它告诉模型根据我们传入的数据进行训练。 It tells the model to train on the data we're passing in.
所以这里我们传递我们制作的数据集 So here we pass the datasets we made
在上一节中， in the previous section,
数据集包含我们的输入和标签。 the dataset contains both our inputs and our labels.
所以我们不需要指定单独的标签， So we don't need to specify separate labels,
当我们调用 fit 时。 when we're calling fit.
然后我们对 validation_data 做同样的事情。 Then we do the same thing with the validation_data.
然后我们可以，如果我们愿意，我们可以指定细节， And then we can if we want, we can specify details,
比如训练的次数 like the number of epochs for training
你可以传递一些其他参数给 fit 。 where there's some other arguments you can pass to fit.
但最后，你只需将所有这些传递给 model.fit But in the end, you just pass all of this to model.fit
然后你让它运行。 and you let it run.
如果一切顺利， If everything works out,
你应该看到一个小训练栏 you should see a little training bar
随着损失的减少而进步。 progressing along as your loss goes down.
就是这样。 And that's it.
在运行时， While that's running,
你知道，你可以打电话给你的老板并告诉他们 you know, you can call your boss and tell them
你现在是高级 NLP 机器学习工程师 you're a senior NLP machine learning engineer now
你会想要下个季度的薪水审查。 and you're gonna want a salary review next quarter.
这几行代码真的够用了 These few lines of code are really all it takes
应用大量的力量 to apply the power of a massive
预训练语言问题， pre-trained language problem,
大规模的预训练语言模型，不好意思， massive pre-trained language model, excuse me,
到你的 NLP 问题。 to your NLP problem.
但我们能做得更好吗？ But could we do better than this?
我的意思是，我们当然可以。 I mean, we certainly could.
具有一些更高级的 Keras 功能 With a few more advanced Keras features
就像一个经过调整的、预定的学习率， like a tuned, scheduled learning rate,
我们可以获得更低的损失 we can get an even lower loss
以及一个更准确、更有用的模型。 and an even more accurate, more useful model.
训练模型后，我们如何处理模型？ And what do we do with our model after we train it?
所以所有这些都将包含在视频中 So all of this is going to be covered in the videos
即将推出，敬请期待。 that are coming up, so stay tuned.
（空气呼啸） (air whooshing)