• Real Analysis
• set
  • complement of set $S$
  • cardinality
    • finite set
    • infinite set
      • countable set
• real number
  • Archimedian property
• function
  • inverse function
• sequence
  • sequence convergence
  • sequence diverge to infinity
  • bounded sequence
  • limit theorem
  • Cauchy sequence
  • subsequence
  • limit point
  • Bolzano-Weierstrass Theorem
• continuity
  • continuous function on closed interval
    • intermediate value theorem
  • uniform continuity
  • Lipschitz continuity
• integral
  • partition
  • upper sum/ lower sum
  • Riemann integrability
  • Riemann integral
  • Riemann sum
• differentiation
  • continuously differentiable
  • Rolle’s theorem
  • mean value theorem
  • Taylor’s theorem
• sequence of function
  • sequence of function pointwise convergence
  • sequence of function uniform convergence
  • supremum norm
    • function converge in the sup norm
    • Cauchy sequence in the sup norm
    • integral equation
• calculus of variation
  • functional
    • Euler equation
• metric space
  • metric
  • sequence of point in metric space converge
  • equivalent metric
  • Cauchy sequence of point in metric space
    • complete metric space
    • contraction
      • fixed point of contraction
      • contraction mapping principle
• series of function
  • limit superior and limit inferior
    • limit superior
    • limit inferior
  • partial sum
  • series convergence test
    • series absolute convergence
    • comparison test
    • root test
    • ratio test
  • series of function converge
    • Weierstrass M-test
    • integral of uniformly convergent series of function
    • derivative of uniformly convergent series of function
• power series
  • convergence of power series
  • property of $f(x)$ in power series

Real Analysis

set

complement of set $S$

$S\subseteq X$

$$S^c:=\{x\in X|x\notin S\}$$

cardinality

$|S|=|T|⟺\exists f:S\to T$ one-to-one correspondence

finite set

empty or has cardinality of $\{1,2,\cdots ,n\}$ for some $n$

infinite set

countable set

has cardinality of $\mathbb{N}$

• $T$ countable, $S$ infinite and $S\subseteq T\Rightarrow S$ countable
• $S,T$ countable $\Rightarrow S\times T$ countable

real number

• real number are complete
  every Cauchy sequence in $\mathbb{R}$ converge to $\mathbb{R}$
• real number satisfy the Archimedian property

Archimedian property

$a>0,b>0\Rightarrow \exists n\in \mathbb{Z},na>b$

function

function $f$ from $S$ to $T$ is subset $F\subseteq S\times T$

$t$ is the value of $f$ at $s$:

$$s\stackrel{f}{\to }t$$

or

$$t=f(s)$$

• $Dom(f)$ domain
• $Ran(f)$ range
• onto: $Ran(f)=T$
• one-to-one: $\forall t\in Ran(f),\exists !s\in S,f(s)=t$
• one-to-one correspondence: onto and one-to-one

inverse function

for one-to-one function $f$, inverse function $f^{-1}$ of $f⟺$

$$\begin{gathered} f(f^{-1}(t))=t\forall t\in Dom(f) \\ {f}^{-1}(f(s))=s\forall s\in S \end{gathered}$$

sequence

$\{a_n{\}}_{n=1}^{\infty }$ or $\{a_n\}$

$$a_n:\mathbb{N}\to \mathbb{R}$$

see also Sequence and Series

sequence convergence

$$\underset{n\to \infty }{\lim }{a}_n=a\text{or}{a}_n\to a$$

or $\{a_n\}$ converge to limit $a\iff$

$\forall \epsilon >0,\exists$ integer $N(\epsilon )$, so that $\forall n\ge N$,

$$|a_n-a|\le \epsilon$$

sequence diverge to infinity

$\forall M,\exists N$, so that $\forall n\ge N$,

$$a_n\ge M$$

bounded sequence

$a_n$ is bounded iff

$\exists M$ so that $\forall n$

$$|a_n|\le M$$

• bounded monotone sequence converge

limit theorem

• bounded sequence
• squeeze theorem
• limit are linear
• multiplication and division can be extracted

Cauchy sequence

$\forall \epsilon >0,\exists N$ so that, if $n\ge N,m\ge N$, then

$$|a_n-a_m|\le \epsilon$$

• convergent sequence are Cauchy sequence
• axiom of completeness
  Cauchy sequence in $\mathbb{R}$ converge to finite number in $\mathbb{R}$

subsequence

$k\mapsto n(k):\mathbb{N}\to \mathbb{N}$

$\forall k\in \mathbb{N},n(k+1)>n(k)$

subsequence of $\{a_n{\}}_{n=1}^{\infty }$,

$$\{a_{n(k)}{\}}_{k=1}^{\infty }\text{ or }\{a_{n_k}\}$$

limit point

limit point $d$ of $\{a_n\}\iff$

$$\forall \epsilon >0,N\in \mathbb{Z},\exists n\ge N,\text{ s.t. }|a_n-d|\le \epsilon$$

• $d$ is limit point $\iff \exists$ subsequence $\{a_{n_k}\}\to d$

Bolzano-Weierstrass Theorem

$\{a_n\}$ bounded $\Rightarrow \exists \{a_{n_k}\},d$ s.t. $\{a_{n_k}\}\to d$

• proof: successively shrink interval by half and keep the half with infinite element
• $a_n\in [b,c]\Rightarrow \exists \{a_{n_k}\},d\in [b,c]$ s.t. $a_{n_k}\to d$

continuity

$f$ is continuous at $c\in Dom(f)\iff$

$$\begin{gathered} \forall \text{ sequence }\{x_n\},x_n\in Dom(f),x_n\to c, \\ \underset{n\to \infty }{\lim }f(x_n)=f(c) \end{gathered}$$

• $f$ is continuous at $c\in Dom(f)\iff$

  $$\begin{gathered} \forall \epsilon >0,\exists \delta >0,\text{ s.t. }\forall x\in Dom(f), \\ |x-c|\le \delta \Rightarrow |f(x)-f(c)|\le \epsilon \end{gathered}$$

continuous function on closed interval

$f$ is continuous on $S\Rightarrow$ $f$ is bounded:

$$\begin{gathered} \exists B\in \mathbb{R},\text{ s.t. }\forall x\in S, \\ |f(x)|\le B \end{gathered}$$

$f$ continuous on $[a,b]\Rightarrow$

$$\begin{gathered} \exists c,d\in [a,b]\text{ s.t.} \\ f(c)=\underset{[a,b]}{\sup }f,f(d)=\underset{[a,b]}{\inf }f \end{gathered}$$

and

$$Ran(f)=[\underset{[a,b]}{\inf }f,\underset{[a,b]}{\sup }f]$$

and $f$ is uniformly continuous on $[a,b]$

• supremum

  $$\underset{S}{\sup }f:=\sup \{f(x)|x\in S\}$$

• infimum

  $$\underset{S}{\inf }f:=\inf \{f(x)|x\in S\}$$

intermediate value theorem

$f$ continuous on $[a,b]$, $f(a)\ne f(b)$, $y\in \mathbb{R}$ is between $f(a),f(b)\Rightarrow$

$$\exists c\in (a,b)\text{ s.t. }f(c)=y$$

uniform continuity

$f$ is uniformly continuous $\iff$

$$\begin{gathered} \forall \epsilon >0,\exists \delta >0\text{ s.t. }\forall x,c\in Dom(f) \\ |x-c|\le \delta \Rightarrow |f(x)-f(c)|\le \epsilon \end{gathered}$$

Lipschitz continuity

$f$ is Lipschitz continuous on $S\iff$

$$\exists M\text{ s.t. }\forall x,c\in S,x\ne c,\frac{f(x)-f(c)}{x-c}\le M$$

integral

partition

partition $P$ of $[a,b]$ is any finite collection of point

$$x_0<x_1<\cdots <x_N$$

where

$$x_0=a,x_N=b$$

upper sum/ lower sum

for subinterval $[x_{i-1},x_i]$ of partition $P$

$$\begin{gathered} M_i:=\underset{x}{\sup }\{f(x)|x_{i-1}\le x\le x_i\} \\ {m}_i:=\underset{x}{\inf }\{f(x)|x_{i-1}\le x\le x_i\} \end{gathered}$$

lower sum

$$L_P(f):=\sum _{i=1}^N{m}_i(x_i-x_{i-1})$$

upper sum

$$U_P(f):=\sum _{i=1}^N{M}_i(x_i-x_{i-1})$$

• $L_P(f)\le U_P(f)$

• for any partition $Q$

  $$L_P(f)\le U_Q(f)$$

• for partition $Q$ that contain all point of $P$ and additional point

  $$L_P(f)\le L_Q(f),U_Q(f)\le U_P(f)$$

Riemann integrability

$f$ on $[a,b]$ is Riemann integrable $\iff$

$$\underset{P}{\sup }\{L_P(f)\}=\underset{P}{\inf }\{U_P(f)\}$$

• $\forall \epsilon >0,\exists$ partition $P$ s.t.

  $$U_P(f)-L_P(f)\le \epsilon$$

  $\Rightarrow f$ is Riemann integrable

• $f$ continuous on $[a,b]\Rightarrow f$ Riemann integrable

Riemann integral

$f$ Riemann integrable on $[a,b]$

Riemann integral

$${\int }_a^{b}f(x)dx:=\inf \{U_P(f)\}$$

• $\forall x\in [a,b],f(x)\le g(x)\Rightarrow$

  $${\int }_a^{b}f(x)dx\le {\int }_a^{b}g(x)dx$$

• $f\in C[a,b]\Rightarrow$

  $$\begin{gathered} \left|{\int }_a^{b}f(x)dx\right|\le {\int }_a^b|f(x)|dx \\ \left|{\int }_a^{b}f(x)dx\right|\le (b-a)\underset{[a,b]}{\sup }|f(x)| \end{gathered}$$

Riemann sum

$$\sum _{i=1}^{N}f(x_i^{\ast })(x_i-x_{i-1})$$

where $x_i^{\ast }\in [x_{i-1},x_i]$

• $f$ continuous on $[a,b]$, $\{P_k\}$ is sequence of partition s.t. maximum length of subinterval $\to 0$ as $k\to \infty$, $S_k$ is any Riemann sum corresponding to $P_k\Rightarrow$

  $$S_k\to {\int }_a^{b}f(x)dx\text{as}k\to \infty$$

differentiation

continuously differentiable

$f$ differentiable on $[a,b]$ and $f^{\prime }$ continuous on $[a,b]$, or $f\in C^{(1)}[a,b]$

• $f$ continuous on $\mathbb{R}$ is denoted as $f\in C(\mathbb{R})$

Rolle’s theorem

$f\in C[a,b]$ differentiable on $(a,b)$, $f(a)=0=f(b)\Rightarrow$

$$\exists c\in (a,b),\text{ s.t. }{f}^{\prime }(c)=0$$

mean value theorem

$f\in C[a,b]$ differentiable on $(a,b)$, $\Rightarrow$

$$\exists c\in (a,b),\text{ s.t. }{f}^{\prime }(c)=\frac{f(b)-f(a)}{b-a}$$

Taylor’s theorem

$f\in C^{(n)}[a,b]$, $f^{(n+1)}$ exist on $(a,b)$, $x_0\in [a,b]\Rightarrow$

$$\begin{gathered} \forall x\in [a,b],x\ne x_0,\exists \xi \text{ between }x,x_0\text{ s.t.} \\ f(x)=T^{(n)}(x,x_0)+\frac{f^{(n+1)}(\xi )}{(n+1)!}(x-x_0{)}^{n+1} \end{gathered}$$

where $T^{(n)}$ is define in Sequence and Series

• proof

  fix $x$, let $\alpha$ s.t.

  $$\begin{gathered} f(x)=T^{(n)}(x,x_0)+\frac{f^{(n+1)}(\xi )}{(n+1)!}(x-x_0{)}^{n+1}+\alpha (x-x_0{)}^{n+1} \\ g(t)=f(t)-T^{(n)}(t,x_0)-\alpha (t-x_0{)}^{n+1} \end{gathered}$$

  apply Rolle’s theorem $n$ time to show $\exists \xi$ between $x,x_0$ s.t.

  $$g^{(n+1)}(x_{n+1})=0$$

sequence of function

$\{f_n\}$, $f_n$ defined on $E$

sequence of function pointwise convergence

$f_n$ converge pointwise to limiting function $f\iff$

$$\forall x\in E,\underset{n\to \infty }{\lim }{f}_n(x)=f(x)$$

sequence of function uniform convergence

$f_n$ converge uniformly to limiting function $f\iff$

$$\begin{gathered} \forall \epsilon >0,\exists N\text{ s.t. }\forall n\ge N \\ \forall x\in E,|f_n(x)-f(x)|\le \epsilon \end{gathered}$$

• $\forall n,f_n\in C(E)\Rightarrow f\in C(E)$

• $\forall n,f_n\in C[a,b]\Rightarrow$

  $$\underset{n\to \infty }{\lim }{\int }_a^b{f}_n(x)dx={\int }_a^{b}f(x)dx$$

• sequence of function $F_n$, $F_n\in C^{(1)}[a,b]$, $F_n^{\prime }=f_n$, $\exists x_0,\{F_n(x_0)\}$ converge $\Rightarrow F\in C^{(1)}[a,b]$, $F^{\prime }=f$

• $Q:=[a,b]\times [c,d]$, $f\in C(Q)$. $\forall x_0\in [a,b],f(x_0,\cdot )\in C[c,d]$, $f_y\in C(Q)$. $F$ on $[c,d]$,

  $$F(y):={\int }_a^{b}f(x,y)dx$$

  $\Rightarrow F\in C^{(1)}[c,d]$,

  $$F^{\prime }(y)={\int }_a^b{f}_y(x,y)dx$$

supremum norm

$f$ bounded on $E$

supremum norm or sup norm of $f$

$$\Vert f{\Vert }_{\infty }:=\underset{E}{\sup }|f(x)|$$

• sup norm is a metric

function converge in the sup norm

$\{f_n\}$ on $E$ converge in the sup norm to $f\iff$

$$\underset{n\to \infty }{\lim }\Vert f_n-f{\Vert }_{\infty }=0$$

• $\iff f_n$ converge to $f$ uniformly on $E$

Cauchy sequence in the sup norm

$\{f_n\}$ on $E$ is Cauchy sequence in the sup norm $\iff$

$$\begin{gathered} \forall \epsilon >0,\exists N\text{ s.t.} \\ \forall m,n\ge N,\Vert f_m-f_n{\Vert }_{\infty }\le \epsilon \end{gathered}$$

• $\Rightarrow f_n$ converge in the sup norm
  • $C[a,b]$ is complete in the sup norm

integral equation

$f\in C[a,b]$

$Q:=[a,b]\times [a,b]$, $K\in C(Q)$, $\forall x,y\in Q,|K(x,y)|\le M$

$$\psi (x)=f(x)+\lambda {\int }_a^{b}K(x,y)\psi (y)dy$$

$\alpha :=|\lambda |M(b-a)<1\Rightarrow \psi$ has unique continuous solution

proof of existence

1. define any continuous ${\psi }_0$ and

   $${\psi }_n(x)=f(x)+\lambda {\int }_a^{b}K(x,y){\psi }_{n-1}(y)dy$$

2. show that ${\psi }_n\in C[a,b]$ by induction

3. show that $\forall x\in [a,b]$

   $$|{\psi }_{n+1}(x)-{\psi }_n(x)|\le \alpha \Vert {\psi }_n-{\psi }_{n-1}{\Vert }_{\infty }$$

4. show by iteration that ${\psi }_n$ is Cauchy in sup norm

proof of uniqueness by contradiction: subtract equation of two solution function

calculus of variation

functional

function with domain containing function

Euler equation

$f$ is twice continuously differentiable functional of three variables

$$J:={\int }_a^{b}f(x,y(x),y^{\prime }(x))dx$$

$y(x)$ is extremal for $J\Rightarrow y(x)$ satisfy Euler equation

$$f_y-\frac{d}{dx}{f}_{y^{\prime }}=0$$

metric space

$(\mathcal{M},\rho )$

metric

$$\rho :\mathcal{M}\times \mathcal{M}\to [0,\infty ]$$

$\forall x,y,z\in \mathcal{M},$

1. $\rho (x,y)\ge 0.\rho (x,y)=0\iff x=y$
2. $\rho (x,y)=\rho (y,x)$
3. $\rho (x,y)+\rho (y,z)\ge \rho (x,z)$

sequence of point in metric space converge

$\{x_n\}$, $x_n\in (\mathcal{M},\rho )$

$\{x_n\}$ converge to $x\in \mathcal{M}\iff$

$$\underset{n\to \infty }{\lim }\rho (x_n,x)=0$$

also denoted as

$$\underset{n\to \infty }{\lim }{x}_n=x$$

equivalent metric

metric $\rho ,\sigma$ on $\mathcal{M}$ are equivalent $\iff$

$$\begin{gathered} \forall \epsilon >0,x\in \mathcal{M},\exists \delta >0\text{ s.t. }\forall y\in \mathcal{M} \\ \rho (x,y)<\delta \Rightarrow \sigma (x,y)<\epsilon \\ \sigma (x,y)<\delta \Rightarrow \rho (x,y)<\epsilon \end{gathered}$$

• $x_n\to x$ in $\rho \iff x_n\to x$ in $\sigma$

Cauchy sequence of point in metric space

complete metric space

contraction

$$T:\mathcal{M}\to \mathcal{M}$$

is contraction $\iff$

$$\begin{gathered} \exists \alpha \in [0,1)\text{ s.t.} \\ \forall x,y\in \mathcal{M},\rho (T(x),T(y))\le \alpha \rho (x,y) \end{gathered}$$

fixed point of contraction

$$T(\bar{x})=\bar{x}$$

• stable

  $$\exists \delta >0\text{ s.t. }{x}_0\in [\bar{x}-\delta ,\bar{x}+\delta ]\Rightarrow x_n\to \bar{x}$$

contraction mapping principle

$(\mathcal{M},\rho )$ complete $\Rightarrow$

1. $T$ has unique fixed point $x$
2. $x_{n+1}:=T(x_n)\Rightarrow \forall x_0\in \mathcal{M},x_n\to x$

series of function

• $\sum a_j$ converge $\iff \forall \epsilon >0,\exists N$ s.t.

  $$\forall n,m\ge N,\left|\sum _{j=n}^m{a}_j\right|\le \epsilon$$

• $\sum a_j$ converge $\Rightarrow a_j\to 0$

• linearity

limit superior and limit inferior

$\{a_n\}$, $a_n\in \mathbb{R}$

$$\begin{gathered} {\bar{s}}_N=\sup \{a_n|n\ge N\} \\ {\underline{\text{s}}}_N=\inf \{a_n|n\ge N\} \end{gathered}$$

• $\{a_n\}\to a\iff$

  $$\begin{gathered} -\infty <\lim \sup a_n\le \lim \inf a_n<\infty \\ \iff \lim \sup a_n=a=\lim \inf a_n \end{gathered}$$

• $a_n>0\Rightarrow$

  $$\lim \inf \frac{a_{n+1}}{a_n}\le \lim \inf \sqrt[n]{a_n}\le \lim \sup \sqrt[n]{a_n}\le \lim \sup \frac{a_{n+1}}{a_n}$$

limit superior

$$\lim \sup a_n:=\bar{s}:=\underset{N\to \infty }{\lim }{\bar{s}}_N$$

• $-\infty <\bar{s}=s<\infty \iff \forall \epsilon >0,$
  • $\exists N$ s.t. $\forall n\ge N,a_n\le s+\epsilon$
  • $\forall N,\exists n\ge N$ s.t. $a_n\ge s-\epsilon$

limit inferior

$$\lim \inf a_n:=\underline{\text{s}}:=\underset{N\to \infty }{\lim }{\underline{\text{s}}}_N$$

• $-\infty <\underline{\text{s}}=s<\infty \iff \forall \epsilon >0,$
  • $\exists N$ s.t. $\forall n\ge N,a_n\ge s+\epsilon$
  • $\forall N,\exists n\ge N$ s.t. $a_n\le s-\epsilon$

partial sum

partial sum $S_n$ of series ${\sum }_{j=1}^{\infty }{a}_j$

$$S_n:=\sum _{j=1}^n{a}_j$$

• series convergence is the same as its sequence of partial sum $\{S_n\}$

series convergence test

series absolute convergence

$\sum |a_j|$ converge $\Rightarrow \sum a_j$ absolutely converge

• otherwise, conditionally converge or diverge
• $f:\mathbb{N}\to \mathbb{N}$ is bijection $\Rightarrow \sum a_{f(j)}=\sum a_j$ (rearrangement)
• $\sum b_k$ absolutely converge $\Rightarrow \sum a_j{b}_k=\sum a_j\sum b_k$ absolutely converge

comparison test

$\exists J,\forall j\ge J,0\le a_j\le b_j\le c_j\Rightarrow$

1. $\sum c_j$ converge $\Rightarrow \sum b_j$ converge
2. $\sum a_j$ diverge $\Rightarrow \sum b_j$ diverge

root test

$$\alpha :=\lim \sup |a_j{|}^{\frac{1}{j}}$$

1. $\alpha <1\Rightarrow \sum a_j$ converge absolutely
2. $\alpha >1\Rightarrow \sum a_j$ diverge

ratio test

$a_j\ne 0$

1. $\lim \sup \frac{|a_{j+1}|}{|a_j|}<1\Rightarrow \sum a_j$ converge absolutely
2. $\lim \sup \frac{|a_{j+1}|}{|a_j|}>1\Rightarrow \sum a_j$ diverge

series of function converge

$\{f_j(x)\},x\in E$

$$f(x)=\sum _{j=1}^{\infty }{f}_j(x)$$

• $\sum f_j$ converge to $f\iff \forall x,\sum f_j(x)$ converge to $f(x)$

Weierstrass M-test

1. $\forall j,\exists M_j$ s.t. $\forall x\in E,|f_j(x)|\le M_j$
2. $\sum M_j$ converge

$\Rightarrow {\sum }_{j=1}^{\infty }{f}_j(x)$ converge uniformly

• $f_j\in C(E)\Rightarrow f\in C(E)$

integral of uniformly convergent series of function

${\sum }_{j=1}^{\infty }{f}_j(x)$ converge uniformly to $f(x)$ on $[a,b]\Rightarrow$

$\forall x\in [a,b]$,

$${\int }_a^{x}f(t)dt=\sum _{j=1}^{\infty }{\int }_a^x{f}_j(t)dt$$

derivative of uniformly convergent series of function

1. $f_j\in C^{(2)}(E)$
2. $\sum f_j(x)$ converge uniformly to $f(x)$ on $E$
3. $\sum f_j^{\prime }(x)$ converge uniformly on $E$

$\Rightarrow$

1. $f\in C^{(2)}(E)$
2. $$f^{\prime }(x)=\sum _{j=1}^{\infty }{f}_j^{\prime }(x)$$

power series

$$\begin{gathered} f(x):=\sum _{j=0}^{\infty }{a}_j(x-x_0{)}^j \\ \rho :=\mathrm{limsup}|a_j{|}^{\frac{1}{j}} \end{gathered}$$

radius of convergence of $f(x)$

$$R=\{\begin{array}{ll}\infty & \rho =0\\ 0& \rho =\infty \\ \frac{1}{\rho }& \text{otherwise}\end{array}$$

convergence of power series

• $f(x)$ converge on $(x_0-R,x_0+R)$
• $f(x)$ diverge outside $[x_0-R,x_0+R]$
• $\forall 0\le r<R,f(x)$ converge uniformly on $[x_0-r,x_0+r]$

property of $f(x)$ in power series

• $f(x)\in C^{\infty }(x_0-R,x_0+R)$
• $f^{\prime }(x)={\sum }_{j=0}^{\infty }(a_j(x-x_0{)}^j{)}^{\prime }$
• ${\sum }_{j=0}^{\infty }{a}_j(x-x_0{)}^j$ is Taylor series of $f$ about $x_0$