Master theorem (analysis of algorithms)

In the analysis of algorithms, the master theorem, which is a specific case of the Akra-Bazzi theorem, provides a cookbook solution in asymptotic terms for recurrence relations of types that occur in practice. It was popularized by the canonical algorithms textbook Introduction to Algorithms by Cormen, Leiserson, Rivest, and Stein, which introduces and proves it in sections 4.3 and 4.4, respectively. Nevertheless, not all recurrence relations can be solved with the use of the master theorem.

Generic Form

Given a relation of the form:

Failed to parse (Conversion error. Server ("https://wikimedia.org/enwiki/api/rest_") reported: "Cannot get mml. TeX parse error: Undefined control sequence \emph"): {\displaystyle T(n)=aT\left({\frac {n}{b}}\right)+f(n)\;\;\;\;{\emph {where}}\;\;a\geq 1,b>1}

     a  = the number of subproblems in the recursion,
     1/b  = the portion of the original problem represented by each sub-problem
     f(n) = the cost of dividing the problem + the cost of merging the solution

It is possible to determine an asymptotic tight bound according to these three cases:

Case 1

Generic Form

If it is true that:

$f(n)\in O\left(n^{\log _{b}a-\epsilon }\right)$ for some constant $\epsilon >0$

it follows that:

$T(n)\in \Theta \left(n^{\log _{b}a}\right).$

Example

$T(n)=8T\left({\frac {n}{2}}\right)+1000n^{2}$

As you can see in the formula above the variables get the following values:

$a=8$, $b=2$, $f(n)=1000n^{2}$, $\log _{b}a=\log _{2}8=3$

Now you have to check that the following equation holds:

$f(n)\in O\left(n^{\log _{b}a-\epsilon }\right)$

If you insert the Values from above, you get:

$1000n^{2}\in O\left(n^{3-\epsilon }\right)$

If you choose $\epsilon$ = 1, you get:

$1000n^{2}\in O\left(n^{3-1}\right)=O\left(n^{2}\right)$

Since this equation holds, the first case of the master theorem applies to the given recurrence relation, thus resulting in the conclusion:

$T(n)\in \Theta \left(n^{\log _{b}a}\right).$

If you insert the values from above, you finally get:

$T(n)\in \Theta \left(n^{3}\right).$

Thus the given recurrence relation T(n) was in Θ(n³)

Case 2

Generic Form

If it is true that:

$f(n)\in \Theta \left(n^{\log _{b}a}\right)$

it follows that:

$T(n)\in \Theta \left(n^{\log _{b}a}\log(n)\right).$

Example

$T(n)=2T\left({\frac {n}{2}}\right)+10n$

As you can see in the formula above the variables get the following values:

$a=2$, $b=2$, $f(n)=10n$, $\log _{b}a=\log _{2}2=1$

Now you have to check that the following equation holds:

$f(n)\in \Theta \left(n^{\log _{b}a}\right)$

If you insert the Values from above, you get:

$10n\in \Theta \left(n^{1}\right)=\Theta \left(n\right)$

Since this equation holds, the second case of the master theorem applies to the given recurrence relation, thus resulting in the conclusion:

$T(n)\in \Theta \left(n^{\log _{b}a}\log(n)\right).$

If you insert the values from above, you finally get:

$T(n)\in \Theta \left(n\log(n)\right).$

Thus the given recurrence relation T(n) was in Θ(nlog(n))

Case 3

Generic Form

If it is true that:

$f(n)\in \Omega \left(n^{\log _{b}a+\epsilon }\right)$ for a $\epsilon >0$

and if it is also true that:

$af\left({\frac {n}{b}}\right)\leq cf(n)$ for a $c<1$ and sufficiently large n

it follows that:

$T(n)\in \Theta (f(n)).$

Example

$T(n)=2T\left({\frac {n}{2}}\right)+n^{2}$

As you can see in the formula above the variables get the following values:

$a=2$, $b=2$, $f(n)=n^{2}$, $\log _{b}a=\log _{2}2=1$

Now you have to check that the following equation holds:

$f(n)\in \Omega \left(n^{\log _{b}a+\epsilon }\right)$

If you insert the Values from above, and choose $\epsilon$ = 1, you get:

$n^{2}\in \Omega \left(n^{1+1}\right)=\Omega \left(n^{2}\right)$

Since this equation holds, you have to check the second Condition, namely if it is true that:

$af\left({\frac {n}{b}}\right)\leq cf(n)$

If you insert once more the values from above, you get:

$2\left({\frac {n}{2}}\right)^{2}$$\leq cn^{2}\Leftrightarrow {\frac {1}{2}}n^{2}\leq cn^{2}$

If you choose $c={\frac {1}{2}}$, it is true that:

${\frac {1}{2}}n^{2}\leq {\frac {1}{2}}n^{2}$ $\forall n\geq 1$

So it follows:

$T(n)\in \Theta (f(n)).$

If you insert once more the necessary values, you get:

$T(n)\in \Theta (n^{2}).$

Thus the given recurrence relation T(n) was in Θ(n²), what complies with the f(n) of the original formula.

References

• Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms, Second Edition. MIT Press and McGraw-Hill, 2001. ISBN 0262032937. Sections 4.3 (The master method) and 4.4 (Proof of the master theorem), pp.73–90.