Welcome to the Hugging Face tasks series.
In this video, we will take a look at the Text Summarization task.
Summarization is a task of producing a shorter version
of a document while preserving the relevant and important information in the document.
Summarization models take a document to be summarized and output the summarized text.
This task is evaluated on the ROUGE score. It’s based on the overlap between the produced sequence
and the correct sequence. You might see this as ROUGE-1,
which is the overlap of single tokens and ROUGE-2, the overlap of subsequent token pairs. ROUGE-N
refers to the overlap of n subsequent tokens. Here we see an example of how overlaps take place.
An example dataset used for this task is called Extreme Summarization, XSUM. This
dataset contains texts and their summarized versions.
You can use summarization models to summarize research papers which
would enable researchers to easily pick papers for their reading list.
For more information about the Summarization task, check out the Hugging Face course.