A proposed methodology for studying the historical trajectory of words’ meaning through Tsallis entropy

The availability of historical textual corpora has led to the study of words’ frequency along the historical time line, as representing the public's focus of attention over time. However, studying of the dynamics of words’ meaning is still in its infancy. In this paper, we propose a methodology for studying the historical trajectory of words’ meaning through Tsallis entropy. First, we present the idea that the meaning of a word may be studied through the entropy of its embedding. Using two historical case studies, we show that this entropy measure is correlated with the intensity in which a word is used. More importantly, we show that using Tsallis entropy with a superadditive entropy index may provide a better estimation of a word's frequency of use than using Shannon entropy. We explain this finding as resulting from an increasing redundancy between the words that comprise the semantic field of the target word and develop a new measure of redundancy between words. Using this measure, which relies on the Tsallis version of the Kullback–Leibler divergence, we show that the evolving meaning of a word involves the dynamics of increasing redundancy between components of its semantic field. The proposed methodology may enrich the toolkit of researchers who study the dynamics of word senses.