Factored language model

The factored language model (FLM) is an extension of conventional Language model. In an FLM, each word is viewed as a vector of k factors: $w_{i}=\{f_{i}^{1},...,f_{i}^{k}\}$. An FLM provides the probabilistic model $P(f|f_{i},...,f_{N})$ where the prediction of factor $f$ is based on $N$ parents $\{f_{1},...,f_{N}\}$. For an example, if $w$ represents word token and $t$ represents Part of speech tag for English, the model $P(w_{i}|w_{i-2},w_{i-1},t_{i-1})$ gives a model for predicting current work token based on traditional Ngram model as well as Part of speech tag of the previous word.

A main advantage of factored language models is they allow users to put in linguistic knowledge such as explicitly model the relationship between word tokens and Part of speech in English, or morphological information (stems, root, etc.) in Arabic.

Like N-gram models, smoothing techniques are necessary in parameter estimation. In particular, generalized backing-off is used in training an FLM.

References

• J Bilmes and K Kirchhoff (2003). "Factored Language Models and Generalized Parallel Backoff" (PDF). Human Language Technology Conference. {{cite conference}}: Unknown parameter |booktitle= ignored (|book-title= suggested) (help)