Abstract
Word alignment plays a crucial role in several NLP tasks, such as lexicon injection and cross-lingual label projection. The evaluation of word alignment systems relies heavily on manually-curated datasets, which are not always available, especially in mid-and low-resource languages. In order to address this limitation, we propose XL-WA, a novel entirely manually-curated evaluation benchmark for word alignment covering 14 language pairs. We illustrate the creation process of our benchmark and compare statistical and neural approaches to word alignment in both language-specific and zero-shot settings, thus investigating the ability of state-of-the-art models to generalize on unseen language pairs. We release our new benchmark at: https://github.com/SapienzaNLP/XL-WA.
- F. Martelli, A.S. Bejgu, C. Campagnano, J. Čibej, R. Costa, A. Gantar, J. Kallas, S. Peneva Koeva, K. Koppel, S. Krek, M. Langemets, V. Lipp, S. Nimb, S. Olsen, B.S. Pedersen, V. Quochi, A. Salgado, L. Simon, C. Tiberius, R. Ureña-Ruiz, R. Navigli. 2023. XL-WA: a Gold Evaluation Benchmark for Word Alignment in 14 Language Pairs. Procedings of the Ninth Italian Conference on Computational Linguistics (CLiC-it 2023), Venezia, Italia. CLiC-it.