In the last post, we stated two fundamental theorems that establish existence of global minimizers of some continuous function , where . Today, we want to discuss the technicalities of establishing these theorems.

Let's start with the first theorem, which will actually enable the proof of the second theorem.

**Theorem 1 [Extreme Value Theorem]:** If is a compact set and is a continuous function on , then has a global minimizer on .

*Proof:* We shall divide the proof into two parts.

[*Part 1*: is bounded below on ] We shall prove this by contradiction. Suppose that is *not* bounded below on . Then, we can define a sequence in , by picking, for every , a vector such that . Since is compact, and thus bounded and closed, we know from the Bolzano-Weierstrass theorem that there exists a convergent subsequence of , whose limit we denote by . Note that we know that is indeed in due the fact that the latter is closed, *i.e.*, contains all its limit points. From the continuity of , we know that converges to the real number . However, since for all , we also have that diverges to . This is a contradiction, which means that is bounded below on .

[*Part 2*: achieves its infimum on ] Since is bounded below, it has a greatest lower bound (commonly known as the infimum), which we shall denote as . What we want to show is that there exists a vector such that . Since is the infimum, for any , we know that is not a lower bound for . We can then define a sequence in by picking such that . Then, we have that

for all , which means that the sequence converges to as . Then we know from the Bolzano-Weierstrass theorem that there exists a subsequence , which converges to some , and additionally, since is closed, we know that . From the continuity of the function , we have that converges to . But, is a subsequence of that converges to , so we must have . Therefore, we can conclude that attains its infimum at .

To establish the second theorem, we first prove that coercivity implies compactness, which is the special case of the following result.

**Lemma 1 [coercivity compactness]**: If is a continuous coercive function defined on all of , then for every the set

is compact.

*Proof for *: First note that, since is continuous, the set is closed. Thus, we only need to show that is bounded. We shall do this by contradiction. Suppose that there is a constant such that is unbounded. Then, there must exist a sequence in such that . But then, by coercivity of , we must also have , which contradicts the fact that by definition of , we have that for all . Therefore, the set must be bounded.

We can now finally establish the following important result.

**Theorem 2 [Extreme Value Theorem]:** If is a continuous coercive function defined on all of , then has a global minimizer.

*Proof:* Let a constant be chosen so that the set is non-empty. From the definition of coercivity, we know that is compact. From the Theorem 1 above, we know that has at least one global minimizer on , and as the set of global minimizers on is a global minimizer on , we get the result.

**References:**

[1] Wikipedia, "Extreme value theorem," https://en.wikipedia.org/wiki/Extreme_value_theorem.

[2] Class notes by James V. Burke on "Nonlinear Optimization", https://www.math.washington.edu/~burke/crs/408/notes/nlp/unoc.pdf

It's a nice post about extreme value theorem. I really like it because it's helpful. It's really appreciable. Thanks for sharing it.