Combining available datasets for building named entity recognition models of Croatian and Slovene

  • Nikola Ljubešić University of Zagreb, Faculty of Humanities and Social Sciences, Department of Information and Communication Sciences
  • Marija Stupar
  • Tereza Jurić University of Zagreb, Faculty of Humanities and Social Sciences, Department of Information and Communication Sciences
  • Željko Agić University of Zagreb, Faculty of Humanities and Social Sciences, Department of Information and Communication Sciences
Keywords: named entity recognition, distributional similarity, Croatian language, Slovene language

Abstract

The paper presents efforts in developing freely available models for named entity recognition and classification in Croatian and Slovene text. Our experiments focus on the most informative set of linguistic features taking into account the availability of language tools and resources for the languages in question. Besides the classic linguistic features, distributional similarity features calculated from large unannotated monolingual corpora are exploited as well. We performed two batches on experiments, the first one on a self-built dataset on which the optimal set of features is sought, and a second batch with additional, much larger datasets obtained at a later point on which we verify the findings from the first batch. On the initial dataset using distributional information improves the results for 7-8 points in F1 while adding morphological information improves the results for additional 3-4 points in both languages. The second batch of experiments shows that morphosyntactic and distributional information lose importance as the dataset size significantly increases. The best performing models that use distributional information only, along with test sets for comparison with existing and future systems are made publicly available for both academic and non-academic use.

Downloads

Download data is not yet available.

References

Ljubešić, N., Stupar, M., Jurić, T., Agić, Ž. (2013): Combining Available Datasets for Building Named Entity Recognition Models of Croatian and Slovene. Slovenščina 2.0, 1 (2): 35–57.
Published
2013-12-01
How to Cite
LjubešićN., StuparM., JurićT., & Agić Željko. (2013). Combining available datasets for building named entity recognition models of Croatian and Slovene. Slovenščina 2.0: Empirical, Applied and Interdisciplinary Research, 1(2), 35-57. https://doi.org/10.4312/slo2.0.2013.2.35-57