The challenging task of summary evaluation: an overview
Please use this identifier to cite or link to this item:
http://hdl.handle.net/10045/71549
Title: | The challenging task of summary evaluation: an overview |
---|---|
Authors: | Lloret, Elena | Plaza Morales, Laura | Aker, Ahmet |
Research Group/s: | Procesamiento del Lenguaje y Sistemas de Información (GPLSI) |
Center, Department or Service: | Universidad de Alicante. Departamento de Lenguajes y Sistemas Informáticos |
Keywords: | Text summarization | Evaluation | Content evaluation | Readability | Task-based evaluation |
Knowledge Area: | Lenguajes y Sistemas Informáticos |
Issue Date: | 2-Sep-2017 |
Publisher: | Springer Science+Business Media B.V. |
Citation: | Language Resources & Evaluation. 2017. doi:10.1007/s10579-017-9399-2 |
Abstract: | Evaluation is crucial in the research and development of automatic summarization applications, in order to determine the appropriateness of a summary based on different criteria, such as the content it contains, and the way it is presented. To perform an adequate evaluation is of great relevance to ensure that automatic summaries can be useful for the context and/or application they are generated for. To this end, researchers must be aware of the evaluation metrics, approaches, and datasets that are available, in order to decide which of them would be the most suitable to use, or to be able to propose new ones, overcoming the possible limitations that existing methods may present. In this article, a critical and historical analysis of evaluation metrics, methods, and datasets for automatic summarization systems is presented, where the strengths and weaknesses of evaluation efforts are discussed and the major challenges to solve are identified. Therefore, a clear up-to-date overview of the evolution and progress of summarization evaluation is provided, giving the reader useful insights into the past, present and latest trends in the automatic evaluation of summaries. |
Sponsor: | This research is partially funded by the European Commission under the Seventh (FP7 - 2007- 2013) Framework Programme for Research and Technological Development through the SAM (FP7-611312) project; by the Spanish Government through the projects VoxPopuli (TIN2013-47090-C3-1-P) and Vemodalen (TIN2015-71785-R), the Generalitat Valenciana through project DIIM2.0 (PROMETEOII/2014/001), and the Universidad Nacional de Educación a Distancia through the project “Modelado y síntesis automática de opiniones de usuario en redes sociales” (2014-001-UNED-PROY). |
URI: | http://hdl.handle.net/10045/71549 |
ISSN: | 1574-020X (Print) | 1574-0218 (Online) |
DOI: | 10.1007/s10579-017-9399-2 |
Language: | eng |
Type: | info:eu-repo/semantics/article |
Rights: | © Springer Science+Business Media B.V. 2017 |
Peer Review: | si |
Publisher version: | http://dx.doi.org/10.1007/s10579-017-9399-2 |
Appears in Collections: | Research funded by the EU INV - GPLSI - Artículos de Revistas |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
2017_Lloret_etal_LangResources&Evaluation_final.pdf | Versión final (acceso restringido) | 837,79 kB | Adobe PDF | Open Request a copy |
2017_Lloret_etal_LangResources&Evaluation_preprint.pdf | Preprint (acceso abierto) | 1,93 MB | Adobe PDF | Open Preview |
Items in RUA are protected by copyright, with all rights reserved, unless otherwise indicated.