Home
A first approach to structure the parameter- and tool-space of semantic entity annotation systems was published by Cornolti et al. [1]. However, the BAT-framework is hard to setup and does not allow for an easy comparison of tools and datasets.
GERBIL [2] offers an easy-to-use web-based platform for the agile comparison of annotators using multiple datasets and uniform measuring approaches. To add a tool to GERBIL, all the end user has to do is to provide a URL to a REST interface to its tool which abides by a given specification. The integration and benchmarking of the tool against user-specified datasets is then carried out automatically by the GERBIL platform. Currently, out platform provides results for 9 annotators (more coming up soon). Internally, GERBIL is based on the Natural Language Programming Interchange Format (NIF) and provides Java classes for implementing APIs for datasets and annotators to NIF. Furthermore, all 11 datasets and annotators are described in RDF. GERBIL will provide persistent URIs for your experiments that are publishable and allow for a new level of reproducibility.
GERBIL will not return annotated text to a user. If a user wants to annotate his texts, he or she must use the tool of his or her choice.