Keywords: language technology evaluation, quantitative evaluation, black box evaluation, generic evaluation protocol, language technology testing, predictive evaluation, performance assessment, part-of-speech tagger, linguistic resources, multilingual resources, language engineering Contact Person: Martin Rajman Phone: (+41 21) 693-5277 E-mail: Martin.Rajman@epfl.ch homepage: http://www.limsi.fr/TLP/ELSE/
One of the ambitious goals of ELSE is to identify what is realistically possible in the domain of Language Engineering evaluation, taking account of particularities of the various sub-domains, available linguistic resources (lexicons, corpora), existing players and current practices.
ELSE will first identify a baseline grain size for the concepts and data needed to implement an evaluation scheme compatible with various approaches and allowing re-use in different language and different application contexts. Such a minimal set will provide a common reference for system developers and researchers to compare their approaches, and for decision-makers to assess the merits of existing systems and strategies in Language Engineering.