Intrinsic Evaluation of Contextualized Word Embeddings

Contextualized word embeddings outperform static word embeddings in a range of downstream tasks. However, in the dictionary-based word translation task, the typically conducted intrinsic evaluation of word embedding, contextualized word embeddings underperform the static word embeddings. Having only dictionary, researchers could not investigate into the dynamic nature of word embeddings with surrounding words. Thus, we made the first step to contextualize a word translation task.

People

Naoki Otani

PhD

Yue Yin

MLT 2021