期刊
ENTROPY
卷 24, 期 2, 页码 -出版社
MDPI
DOI: 10.3390/e24020259
关键词
information theoretic models; synthetic language; entropy; Zipf-Mandelbrot-Li law; language models; behavior prediction
资金
- University of Queensland and Trusted Autonomous Systems Defence Cooperative Research Centre [2019002828]
This paper discusses the importance of identifying probabilistic symbols and introducing symbolization methods for entropy-based models and synthetic languages. New symbolization algorithms are proposed and demonstrated with real-world data.
An important aspect of using entropy-based models and proposed synthetic languages, is the seemingly simple task of knowing how to identify the probabilistic symbols. If the system has discrete features, then this task may be trivial; however, for observed analog behaviors described by continuous values, this raises the question of how we should determine such symbols. This task of symbolization extends the concept of scalar and vector quantization to consider explicit linguistic properties. Unlike previous quantization algorithms where the aim is primarily data compression and fidelity, the goal in this case is to produce a symbolic output sequence which incorporates some linguistic properties and hence is useful in forming language-based models. Hence, in this paper, we present methods for symbolization which take into account such properties in the form of probabilistic constraints. In particular, we propose new symbolization algorithms which constrain the symbols to have a Zipf-Mandelbrot-Li distribution which approximates the behavior of language elements. We introduce a novel constrained EM algorithm which is shown to effectively learn to produce symbols which approximate a Zipfian distribution. We demonstrate the efficacy of the proposed approaches on some examples using real world data in different tasks, including the translation of animal behavior into a possible human language understandable equivalent.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据