# Orthogonality

### From bio-physics-wiki

If $f(x)$ and $g(x)$ are two real valued functions, continuous on the interval $a \leq x \leq b$, we define the **inner product** as the integral

\begin{align} ( f,g ) = \int_a^b f(x) g(x) dx \end{align}

In analogy to the orthogonality concept of vectors, we call two functions orthogonal if their product is zero $(f,g)=0$. In the article about Fourier Series we introduced the concept of orthogonality in an intuitive way, by calculating resulting integrals explicitly. The linear combination of sine and cosine of
\begin{align}
y(x)=a_{\lambda} \, cos (\lambda x) + b_{\lambda} \, sin(\lambda x)
\end{align}
is the general solution to the ODE
\begin{align}
y''=-\lambda^2y
\end{align}
We use this fact to proof the **orthogonality** of solutions (the **eigenfunctions**) to different **eigenvalues** $\lambda$. Let $Y_1$ and $Y_2$ be two different Eigenfunctions. Thus
\begin{align}
Y_1''&=-\lambda_1^2 Y_1\\
Y_2''&=-\lambda_2^2 Y_2\\
\end{align}
If we multiply by $Y_2$ and $Y_1$ respectively
\begin{align}
Y_1''Y_2&=-\lambda_1^2 Y_1Y_2\\
Y_2''Y_1&=-\lambda_2^2 Y_2Y_1\\
\end{align}
and subtract the equations, we have
\begin{align}
Y_1''Y_2-Y_2''Y_1&=(\lambda_2^2-\lambda_1^2) Y_1Y_2=(\lambda_2'-\lambda_1') Y_1Y_2\\
\end{align}
By partial integration we find
\begin{align}
\int_a^b Y_1''Y_2-Y_2''Y_1 dx= \left[Y_1'Y_2 \right]_a^b-\int_a^b Y_1'Y_2' dx - \left[ Y_2'Y_1 \right]_a^b + {\int_a^b Y_1'Y_2'} dx =\left[Y_1'Y_2 - Y_2'Y_1 \right]_a^b
\end{align}
**Green's second identity**

\begin{align} \int_a^b Y_1''Y_2-Y_2''Y_1 dx=\left[Y_1'Y_2 - Y_2'Y_1 \right]_a^b \tag{1} \end{align}

Using Green's second identity, we can draw the following conclusions

- Under
**Dirichlet boundary conditions**$Y_1(a)=Y_1(b)=Y_2(a)=Y_2(b)=0$ the RHS of (1) is zero. - Under
**Neuman boundary conditions**$Y_1'(a)=Y_1'(b)=0$ again the RHS of (1) is zero. - For
**Periodic boundary conditions**$Y_j'(a)=Y_j'(b)$ and $Y_j(a)=Y_j(b)$ for $j=1,2$ ... (1) does vanish again.

the integral \begin{align} \int_a^b (\lambda_2'-\lambda_1') Y_1Y_2 dx =0\\ \end{align} is zero and as long as $\lambda_1' \not = \lambda_2'$, we are permitted to conclude that the functions $Y_1$ and $Y_2$ are orthogonal. Notice, although in all discussed boundary conditions (1) was zero, this is generally not necessarily the case.

## Orthogonality of Sturm-Liouville Eigenfunctions

The sines and cosines are Eigenfunctions of the harmonic differential equation. In fact, the harmonic ODE is a Sturm-Liouville equation, so there are Sturm-Liouville problems whose Eigenfunctions, are orthogonal. In the following we will show, that all Eigenfunctions that are solutions to a Sturm-Liouville problem are orthogonal.

In another article we have seen, that the Sturm-Liouville problems are of the form \begin{align} \left [ u(x)\frac{dy}{dx} \right]'+\tilde{q}(x)y+ \lambda \tilde{\rho}(x) y=0\\ \end{align}

If we use the Eigenfunction Method with the linear *Sturm-Liouville operator* $\mathcal{L}=-\frac{d}{dx}\left [ u(x)\frac{d}{dx} \right]-\tilde{q}(x)$ we can formulate the *Sturm-Liouville Eigenvalue probelm*
\begin{align}
\mathcal{L}y=\lambda \tilde{\rho}(x) y
\end{align}

### Adjoint and Hermitian operators

We allow $x$ to be complex $x \in \mathbb{C}$ and define the *inner product*
\begin{align}
\langle f\mid g \rangle = \int_a^b f(x)^* g(x) dx
\end{align}
where ${}^*$ denotes the complex conjugate. The product has the property $\langle f \mid g \rangle=\langle g \mid f \rangle^*$

In analogy to linear algebra the **adjoint** of an Operator $\mathcal{L}$ denoted $\mathcal{L}^{\dagger}$ is defined as
\begin{align}
\int_a^b f(x)^* [\mathcal{L} g(x) ]dx= \left( \int_a^b g(x)^* [\mathcal{L}^{\dagger} f(x) ]dx \right) ^*
\end{align}
or in product notation
\begin{align}
\langle f \mid \mathcal{L}g \rangle=\langle g \mid \mathcal{L}^{\dagger} f \rangle^*
\end{align}
An operator is called **self-adjoint** or **Hermitian**, if $\mathcal{L}=\mathcal{L}^{\dagger}$
\begin{align}
\int_a^b f(x)^* [\mathcal{L} g(x) ]dx= \left( \int_a^b g(x)^* [\mathcal{L} f(x) ]dx \right) ^*
\end{align}
or in product notation
\begin{align}
\langle f \mid \mathcal{L}g \rangle=\langle g \mid \mathcal{L} f \rangle^*
\end{align}

We are now ready to start the general argument for orthogonality of Sturm-Liouville problems similar to that allready tackled for Eigenfunctions of the harmonic ODE. Consider an Hermition operator that has at least two Eigenfunctions $y_i(x)$ and $y_j(x)$ with corresponding Eigenvalues $\lambda_i$ and $\lambda_j$, hence
\begin{align}
\mathcal{L}y_i=\lambda_i \rho(x) y_i\\
\mathcal{L}y_j=\lambda_j \rho(x) y_j\\
\end{align}
(we drop the tilde for convinient notation). Multiplying by $y_j^*$ and $y_i^*$ we have
\begin{align}
\int_a^b y_j^* \mathcal{L}y_i \, dx=\lambda_i \int_a^b \rho(x) y_j^*y_i \, dx \tag{2}\\
\int_a^b y_i^* \mathcal{L}y_j \, dx=\lambda_j \int_a^b \rho(x) y_i^*y_j \, dx \tag{3}\\
\end{align}
Taking the adjoint of (2) we have
\begin{align}
\left( \int_a^b y_j^* \mathcal{L}y_i \, dx \right)^* = \left( \lambda_i \int_a^b \rho(x) y_j^*y_i \, dx \right)^* \\
\int_a^b y_i^* \mathcal{L}^{\dagger}y_j \, dx = \lambda_i^* \int_a^b \rho(x)^* y_i^*y_j \, dx
\end{align}
Subtracting this equation from (3) and assuming $\rho(x)$ real, this results in
\begin{align}
\int_a^b y_i^* \mathcal{L}y_j - y_i^* \mathcal{L}^{\dagger}y_j \, dx = \left(\lambda_j- \lambda_i^* \right) \int_a^b \rho(x) y_i^*y_j \, dx \tag{4}
\end{align}
Suppose for now the operator is hermitian and $\mathcal{L}=\mathcal{L}^{\dagger}$, then the LHS of this equation would vanish. Later we show that this is only true, when certain boundary conditions are satisfied. We arrived at the similar, but more general equation as for sine and cosine
\begin{align}
\left(\lambda_j- \lambda_i^* \right) \int_a^b \rho(x) y_i^*y_j \, dx=0
\end{align}
Again the argument is, that for $i \not = j$, $\lambda_j-\lambda_i^* \not = 0$ and thus $\int_a^b \rho(x) y_i^*y_j \, dx$ must vanish. We conclude that the *Eigenfunctions of the Sturm-Liouville equation* $y_i$ and $y_j$ *are orthogonal*. We can normalise the Eigenfunctions so that $\int_a^b \rho(x) |y_i|^2 \, dx=1$, then we have

\begin{align} \int_a^b \rho(x) y_i^*y_j \, dx=\left\{\begin{array}{ll} 0 & \text{for } i \not = j \\ 1 & \text{for } i=j \\ \end{array}\right. \\ \int_a^b \rho(x) y_i^*y_j \, dx=\delta_{ij} \end{align}

On the other hand for $i=j$ the integral $\int_a^b \rho(x) |y_i|^2 \, dx$ is certainly not zero if $\rho(x)$ is even, hence it must hold that $\lambda_i-\lambda_i^* = 0$. This implies that $\lambda_i=\lambda_i^*$, which can only be the case if $\lambda_i$ *is real*.

Let us now turn to our assumption, that $\mathcal{L}$ is hermitian. If we plug in the Sturm-Liouville operator $Ly=(py')'-qy$ into (4) we find
\begin{align}
\int_a^b y_i^* \mathcal{L}y_j - y_i^* \mathcal{L}^{\dagger}y_j \, dx &= \int_a^b y_i^*[(py_j')'-qy_j] - \left( y_j^*[(py_i')'-qy_i] \right)^* \, dx \\
&= \int_a^b y_i^* [(py_j')'-q y_j ] - y_j [(p (y_i ^*) ')' -q y_i ^* ] \, dx \tag{5}
\end{align}
by the property $\langle f \mid g \rangle=\langle g \mid f \rangle^*$ the term $q y_j^*y_i-q y_jy_i ^*$ cancels. Integrating one remaining terms of
\begin{align}
\int_a^b y_i^* \mathcal{L}y_j - y_i^* \mathcal{L}^{\dagger}y_j \, dx =\int_a^b y_i^*(py_j')' - y_j (p (y_i ^*) ')' \, dx= \underbrace{\int_a^b y_i^*(py_j')' \, dx}_{I} - \underbrace{\int_a^b y_j (p (y_i ^*) ')' \, dx}_{II}
\end{align}
two times by parts
\begin{align}
I:\int_a^b y_i^*(py_j')' \, dx= \left[y_i^*py_j' \right]_a^b - \,\int_a^b (y_i^*)'py_j' \, dx= \left[y_i^*py_j' \right]_a^b - \left[(y_i^*)'py_j \right]_a^b\, + \underbrace{\int_a^b ((py_i^*)')'y_j \, dx}_{II}
\end{align}
we see that (5) becomes
\begin{align}
\int_a^b y_i^* \mathcal{L}y_j - y_i^* \mathcal{L}^{\dagger}y_j \, dx = \left[y_i^*py_j' \right]_a^b - \left[(y_i^*)'py_j \right]_a^b + II - II
\end{align}
Now we are in the position to conclude, that the operator $\mathcal{L}$ is hermitian provided
\begin{align}
\left[ y_i^* py_j' \right]_a^b - \left[(y_i^*)' py_j \right]_a^b =[p(b)-p(a)] \left[ y_i ^* y_j ' - (y_i ^*) ' y_j \right]_a^b=0
\end{align}
the boundary conditions on $a \leq x \leq b$ with
\begin{align}
\left[ y_i^*y_j' - (y_i^*) ' y_j \right]_a^b=0
\end{align}
are satisfied. Such boundary conditions are called *symmetric* or *hermitian* and have the form
\begin{align}
\alpha_1y(a)+\beta_1y(b)+\gamma_1 y'(a) + \delta_1 y'(b)=0\\
\alpha_2y(a)+\beta_2y(b)+\gamma_2 y'(a) + \delta_2 y'(b)=0\\
\end{align}
If one or more Eigenvalues are degenerate, then the proof is not so strait forward, but it can still be found an orthonormal basis, which can be constructed via *Gram-Schmidt orthonormalisation*. One can show that the argument we made holds in the different direction, that is:

- If the boundary conditions are symmetric, then the Operator is hermitian
- If the operator is hermitian, then the Eigenfunctions are orthogonal and can be chosen orthonormal
- and the Eigenvalues of an hermitian operator, like the Sturm-Liouville operator, are real

The last point is an important fact in Quantum Mechanics, where Eigenvalues represent the possible measured values of observable quantities such angular momentum and energy. As a measured probability to observe a system in a certain state, the Eigenvalues have to be *real numbers* between zero and one.

## General Coefficients of Orthogonal Polynomials

Any Eigenfunctions of a Sturm-Liouville problem are orthogonal, thus we can express the function $f(x)$ in terms of this orthogonal Eigenfunctions.
\begin{align}
f(x)=\sum_{n=0}^{\infty} c_n y_n(x) \rho (x)
\end{align}
To find the general coefficients $c_n$ of this series, we multiply through by $y_n^*$ and integrate over $a \leq x \leq b$
\begin{align}
\int_a^b y_n(x)^*f(x) \rho (x)\, dx =\sum_{n=0}^{\infty} c_n \int_a^b y_n(x)^*y_n(x) \rho (x)\, dx
\end{align}
and rearrange the terms and arrive at the general formulation for the **Coefficients of the Eigenfunction Expansion**
\begin{align}
c_n = \frac{\int_a^b y(x)^*f(x) \rho(x) \, dx}{\int_a^b y_n^*y_n \rho (x)\, dx} = \frac{\int_a^b y(x)^*f(x) \rho (x)\, dx}{\int_a^b|y_n(x)|^2 \rho (x)\, dx}=\frac{ \langle y_n \mid f \rangle}{ \langle y_n \mid y_n \rangle}
\end{align}