The importance of Term Weighting in semantic understanding of text: A review of techniques

In this paper we review a wide spectrum of techniques which have been proposed in literature to enable acceptable recognition of language and text by machines. We discuss many techniques which have been proposed by researchers in the field of term weighting and explore the mathematical foundations of these methods. Term weighting schemes have broadly been classified as supervised and statistical methods and we present numerous examples from both categories to highlight the difference in approaches between the two broad categories. We pay particular attention to the Vector Space Model and its variants which form the basis of many of the other methods which have been discussed in the paper.

Automated summary

This paper reviews a wide range of techniques proposed in the literature for machine recognition of language and text. It discusses the term weighting techniques proposed by researchers, exploring the mathematical foundations of these methods. The term weighting schemes are broadly classified as supervised and statistical methods, and the paper presents numerous examples to highlight the differences between the two categories. The Vector Space Model and its variants, which serve as the basis for many other methods discussed in the paper, are given particular attention.