Vector Space

From bio-physics-wiki

Jump to: navigation, search

In an intuitive way, a vector space is the place where all mathematical operations take place. A vector space has to satisfy certain properties ($VS1-VS9$ listed below) in order for the mathematical operations to be possible. The most basic operations are multiplication of vectors by a scalar and addition of vectors. In other words we want to take linear combinations of vectors $c \mathbf{v} + d \mathbf{w}$. If e.g. $\mathbf{v},\mathbf{w} \in \mathbb{R}^2$ and if $\mathbf{v}$ is not a multiple of $\mathbf{w}$ ($c \mathbf{v} + d \mathbf{w} \not =0$ for any $c$ and $d$), then all possible linear combinations, fill out the whole $\mathbb{R}^2$. A vector space is thus a very general concept, it takes the idea of multiplying and adding certain vectors (and more) to a more general and abstract formulation, where we don't care about certain vectors anymore, but any vectors of the space (e.g. $\mathbb{R}^2$) and consider rules that must be valid for any of the vectors in the space.

An important property, we require from an element (e.g. the vector $\mathbf{v}$) of the vector space, is that any operation we're allowed to do on the element does not result in an element that is not part of the same vector space anymore. This is exactly the statement of $VS1$. For example, If we look at $\mathbb{R}^2$, forms the positive quadrant a vector space?


We can certainly add vectors from the positive quadrant and stay in the positive quadrant, but if we multiply a vector of the pos. quadrant by a negative scalar we leave the positive quadrant. So the positive quadrant of $\mathbb{R}^2$ does not form a vector space, we say the positive quadrant is not closed with respect to scalar multiplication.

Also important is, that a vector space always includes the zero vector ($VS3$), Since we must always be able to multiply by the scalar $0$ without leaving the vector space. So if a space does not contain the zero vector, then we can immediately conclude that it can not be a vector space.

To proof that a space is a vector space we have to show that each of the properties $VS1 - VS9$ are satisfied. However, sometimes it is easy to show that, a space is not a vector space by finding one property that is not satisfied.

A nonempty set $V$ with two operations $+$ and $\cdot$ (addition of elements in $V$ and multiplication of elements in $V$ with elements of $\mathbb{K}$) is called a vector space over $\mathbb{K}$ of just $\mathbb{K}$-vector space. If for all $\mathbf{u},\mathbf{v},\mathbf{w} \in V$ and $\lambda,\mu \in \mathbb{K}$ the following vector space axioms are true.

  • (VS1) $\hspace{1cm}$ $\mathbf{v},\mathbf{w} \in V$ and $\lambda \cdot \mathbf{v} \in V$ (Closure)

  • (VS2) $\hspace{1cm}$ $(\mathbf{u}+\mathbf{v})+\mathbf{w}=\mathbf{u}+(\mathbf{v}+\mathbf{w})$ (Associativity)

  • (VS3) $\hspace{1cm}$ It exists a neutral element $\mathbf{0} \in V$ so that $\mathbf{v}+\mathbf{0}=\mathbf{v}$ (Neutral Element)

  • (VS4) $\hspace{1cm}$ It exists a $\mathbf{v}' \in V$ so that $\mathbf{v}+\mathbf{v}'=\mathbf{0}$ (Inverse Element)

  • (VS5) $\hspace{1cm}$ $\mathbf{v}+\mathbf{w}=\mathbf{w}+\mathbf{v}$ (Kommutativity)

  • (VS6) $\hspace{1cm}$ $\lambda \cdot (\mathbf{v}+\mathbf{w})= \lambda \mathbf{w}+ \lambda \mathbf{v}$

  • (VS7) $\hspace{1cm}$ $(\lambda + \mu) \cdot \mathbf{v}= \lambda \mathbf{v}+ \mu \mathbf{v}$

  • (VS8) $\hspace{1cm}$ $(\lambda \mu) \cdot \mathbf{v}= \lambda (\mu \cdot \mathbf{v})$

  • (VS9) $\hspace{1cm}$ $1 \cdot \mathbf{v}= \mathbf{v}$

For $\mathbb{K}=\mathbb{R}$ the vector space is called a real vector space and for $\mathbb{K}=\mathbb{C}$ a complex vector space.


A subspace is a lower dimensional space inside a higher dimensional space. A subspace is also a vector space and hence has to satisfy $VS1-VS9$. A line through the origin (green) is an example of a subspace in $\mathbb{R}^2$. We can add a vector on the line the another vector on the line and are still on the same line. We can multiply a vector on the line by a scalar and are still on the line. The zero vector is also included, so we are allowed to multiply any vector on the line by zero as well. In fact, all properties are satisfied and the line in $\mathbb{R}^2$ through the origin is a subspace of $\mathbb{R}^2$.


Would the line not go through the origin, it would not be a vector space; the zero vector must always be included. The subspaces of $\mathbb{R}^2$ are:

  • $\mathbb{R}^2$ is a subspace of itself
  • every line through the origin is a subspace of $\mathbb{R}^2$
  • the zero vector is a subspace of $\mathbb{R}^2$

The subspaces of $\mathbb{R}^3$ are:

  • $\mathbb{R}^3$ is a subspace of itself
  • any plane through the origin is a subspace of $\mathbb{R}^3$
  • every line through the origin is a subspace of $\mathbb{R}^3$
  • the zero vector is a subspace of $\mathbb{R}^3$

Column Space

If we now look at the columns of a $m \times n$-matrix $\mathbf{A}$, than every column contains a vector that itself forms a one dimensional subspace in $m$-dimensional space. If we take linear combinations of two columns of the matrix, they form a plane, we say the columns span a two dimensional subspace of $\mathbf{R}^m$. If we take linear combinations of all columns of the matrix, we call the spanned space the column space of $\mathbf{A}$ and write it short as $C(\mathbf{A})$. A ($3\times 2$)-matrix for example has a column space which is two-dimensional


What is the columns space of the matrix \begin{align} \mathbf{A} = \begin{bmatrix} 1 & 1 & 2 \\ 2 & 1 &3 \\ 3 & 1 & 4 \\ 4 & 1 & 5 \end{bmatrix} \end{align} The columns of $\mathbf{A}$ are vectors in $\mathbb{R}^4$. It consist of the linear combination of all three vectors of the $\mathbf{A}$. Certainly, the column space will be only a subspace of $\mathbb{R}^4$, because three vectors can not span the whole four dimensional space. Has the equation \begin{align} \mathbf{Ax} &=\mathbf{b}\\ \begin{bmatrix} 1 & 1 & 2 \\ 2 & 1 &3 \\ 3 & 1 & 4 \\ 4 & 1 & 5 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}&= \begin{bmatrix} b_1 \\ b_2 \\ b_3 \\ b_4 \end{bmatrix} \end{align} a solution for every RHS $\mathbf{b}$? There are four equations and only three unknowns so the general answer is no. Which RHSs do have a solution?

If $\mathbf{b}$ is in the column space of $\mathbf{A}$, then the equation $\mathbf{Ax} =\mathbf{b}$ has a solution.

The column space of $\mathbf{A}$ is only 2-dimensional, since the third column is just the sum of column one and two. Thus the third column lies in the plane spanned by column one and two.

Further Reading:

  • Tilo Arens et al. - Mathematik
  • Riley, Hobson, Bence - Mathematical Methods for Physics and Engineering

For an intuitive introduction to Linear Algebra and linear vector spaces watch the

MIT video lectures:

  • Gilbert Strang - Introduction to Linear Algebra Lec. 5
  • Gilbert Strang - Introduction to Linear Algebra Lec. 6
  • Herbert Gross - Part III: Linear Algebra, Lec 1: Vector Spaces