Accueil du site Accueil du site Adhésion Contact Plan du site

Principles of Evaluation in Natural Language Processing

Patrick Paroubek*, Stéphane Chaudiron**, Lynette Hirschman***

* LIMSI - CNRS
B.P. 133, 91403 Orsay Cedex, France
pap@limsi.fr

** GERiiCO, Université Charles-de-Gaulle (Lille 3)
B.P. 60149, 59653 Villeneuve d’Ascq Cedex, France
stephane.chaudiron@univ-lille3.fr

*** The MITRE Corporation
202 Burlington Rd., Bedford, MA, USA
lynette@mitre.org


In this special issue of TAL, we look at the fundamental principles underlying evaluation in natural language processing. We adopt a global point of view that goes beyond the horizon of a single evaluation campaign or a particular protocol. After a brief review of history and terminology, we will address the topic of a gold standard for natural language processing, of annotation quality, of the amount of data, of the difference between technology evaluation and usage evaluation, of dialog systems, and of standards, before concluding with a short discussion of the articles in this special issue and some prospective remarks.


Télécharger:
Fichier PDF
Patrick Paroubek, Stéphane Chaudiron, Lynette Hirschman
173.8 ko

TAL Volume 48 2007 . 1. Principes de l’évaluation en Traitement Automatique des Langues

Date de dernière mise à jour : 14 septembre 2010, auteur : Rédacteurs en chef.