Markov property

In probability theory, the terms Markov property and Markov-type property refer to two closely related properties of a stochastic process. Their namesake is the Russian mathematician Andrey Markov.

A stochastic process has the Markov property if the conditional probability distribution of future states of the process depend only upon the present state and a fixed number of past states; that is, future states are conditionally independent of past states older than a fixed number of past states. A process with this property is called Markovian or a Markov process. The articles Markov chain and Continuous-time Markov process explore this property in greater detail.

A stochastic process has a Markov-type property if the process's random variables determine a set of probabilities can be factored in a way that yields the Markov property. Useful in applied research, members of such classes defined by their mathematics or area of application are referred to as Markov random fields., and occur in many situations. The Ising model is a prototypical example.

Definition

If one has a system composed of a set of random variables $\{X_{k},k=1,\ldots ,n\}$, then in general, the probability of a given random variable $X_{j}$ being in a state $x_{j}$ is written as

$P[X_{j}=x_{j}|\{X_{k}\},k\neq j].$

That is, in general, the probability of $X_{j}$ being in a state $x_{j}$ depends on the values of all of the other random variables $\{X_{k}\}$. If, instead, one has that this probability only depends on some, but not all of these, then one says that the collection has the Markov property. Letting $N_{j}$ denote the subset of $\{X_{k}\}$ on which $X_{j}$ depends, one then writes this limited dependence as

$P[X_{j}=x_{j}|\{X_{k}\},k\neq j]=P[X_{j}=x_{j}|\{X_{k}\},k\in N_{j}].$

Any collection of random variables having this property is referred to as a Markov network. The set $N_{j}$ is sometimes referred to as the neighbors of $X_{j}$; alternately, it is the Markov blanket of $X_{j}$.

The probability distribution of a Markov network can always be written as a Gibbs distribution, that is, as

$P[X_{j}=x_{j}|\{X_{k}\},k\in N_{j}]={\frac {1}{Z[\lambda _{1},\dots ,\lambda _{n}]}}\exp \left(-\lambda _{j}E_{j}(N_{j})\right)$

for an appropriate energy function E defined on the subset $N_{j}$. The normalizing constant $Z[\lambda _{1},\dots ,\lambda _{n}]$ is known as the partition function.

Markov networks are commonly seen in maximum entropy methods, since the Gibbs measure also has the property of being the unique stochastic measure that maximizes the entropy for a given energy functional.