# Inner Product, Orthogonality and Length of Vectors

   

The inner product , the condition of orthogonality and the length of vectors are presented through examples including their detailed solutions.

## Definition of the Inner Product of two Vectors

Let vectors $\textbf {x}$ and $\textbf y$ be two column vectors (or an $n$ by $1$ matrix) defined by
$\textbf x = \begin{bmatrix} x_1 \\ x_2 \\ . \\ . \\ . \\ x_n \end{bmatrix}$ and $\textbf y = \begin{bmatrix} y_1 \\ y_2 \\ . \\ . \\ . \\ y_n \end{bmatrix}$
The inner product of $\textbf x$ and $\textbf y$ is a scalar quantity written as $\textbf x \cdot \textbf y$ defined by
$\textbf x. \textbf y = \textbf {x}^T \textbf y = \begin{bmatrix} & x_1 & x_2 & . . . & x_n\\ \end{bmatrix} \begin{bmatrix} y_1 \\ y_2 \\ . \\ . \\ . \\ y_n \end{bmatrix} = x_1 y_1 + x_2 y_2 + ... + x_n y_n$ where $\textbf {x}^T$ is the transpose of vector $\textbf {x}$

## Properties of the Inner Product

If $\textbf x$ , $\textbf y$ and $\textbf z$ are vectors in $R^n$ and $k_1$ and $k_2$ are scalars, then

1. $\textbf x \cdot \textbf y = \textbf y \cdot \textbf x$
2. $(\textbf x + \textbf y) \cdot \textbf z = \textbf x \cdot \textbf z + \textbf y \cdot \textbf z$
3. $(k_1 \textbf x) \cdot (k_2 \textbf y) = k_1 k_2 (\textbf x \cdot \textbf y)$

## Orthogonal Vectors

Vectors $\textbf x$ and $\textbf y$ are called orthogonal if
$\textbf x \cdot \textbf y = 0$

## Definition of the Length (or Norm) of a Vector and Unit Vector

The length (or norm ) of vector $\textbf x = \begin{bmatrix} x_1 \\ x_2 \\ . \\ . \\ . \\ x_n \end{bmatrix}$ written as $|| \textbf x ||$ is given by
$|| \textbf x || = \sqrt {x_1^2 + x_2^2 + .... + x_n^2} = \sqrt {\textbf x \cdot \textbf x}$
From the above definition, we can easily conclude that
$|| \textbf x || \ge 0$ and $|| \textbf x ||^2 = \textbf x \cdot \textbf x$
A unit vector is a vector whose length (or norm) is equal to 1.

## Pythagorean Theorem

Vectors $\textbf x$ and $\textbf y$ are orthogonal if and only if
$||x+y||^2 = ||x||^2 + ||y||^2$

## Distance Between two Vectors

The distance between vectors $\textbf x$ and $\textbf y$ is defined as
$dist(\textbf x,\textbf y) = || \textbf x - \textbf y ||$

## Examples with Solutions

Example 1
Given the vectors $\textbf x = \begin{bmatrix} -2 \\ 3 \\ 0 \\ -1 \end{bmatrix}$ , $\textbf y = \begin{bmatrix} 3 \\ -1 \\ 4 \\ 0 \end{bmatrix}$ , $\textbf z = \begin{bmatrix} 2 \\ -1 \\ 4 \\ -7 \end{bmatrix}$, find
a) $\textbf x \cdot \textbf y$ and $\textbf y \cdot \textbf x$
b) $\textbf x \cdot \textbf y + \textbf x \cdot \textbf z$ and $\textbf x \cdot (\textbf y + \textbf z)$
c) $(3 \textbf x ) \cdot (-2\textbf z)$

Solution to Example 1
Use the definition given above
a)
$\textbf x \cdot \textbf y = (-2)(3) + 3(-1) + 0(4) + (-1)0 = - 9$
Using property 1 of the inner product above
$\textbf y \cdot \textbf x = \textbf x \cdot \textbf y = - 9$

b)
$\textbf x \cdot \textbf y + \textbf x \cdot \textbf z = - 9 + 23 = 14$
According to property 2 of the inner product
$\textbf x \cdot (\textbf y + \textbf z) = \textbf x \cdot \textbf y + \textbf x \cdot \textbf z = 14$

c)
According to property 3 of the inner product
$(3 \textbf x ) \cdot (-2\textbf z) = (3)(-2) \textbf x \cdot \textbf y = -6 (-9) = 54$

Example 2
a) Show that vectors $\textbf x = \begin{bmatrix} -2 \\ 3 \\ 0 \end{bmatrix}$ and $\textbf y = \begin{bmatrix} 3 \\ 2 \\ 4 \end{bmatrix}$ are orthogonal.
b) Find the constant $a$ and $b$ so that the vector $\textbf z = \begin{bmatrix} a \\ b \\ 4 \end{bmatrix}$ is orthogonal to both vectors $\textbf x$ and $\textbf y$

Solution to Example 2
Calculate the inner product of vectors $\textbf x$ and $\textbf y$
a)
$\textbf x \cdot \textbf y = (-2)(3) + 3(2) + 0(4) = 0$
Since the inner product of vectors $\textbf x$ and $\textbf y$ is equal to zero, the two vectors are orthogonal.

b)
We first calculate the following inner product
$\textbf x \cdot \textbf z = -2(a) + 3(b) + 0(4) = -2a + 3b$
$\textbf y \cdot \textbf z = 3(a) + 2(b) + 4(4) = 3a + 2b + 16$
For vector $\textbf z$ to be orthogonal to both $\textbf x$ and $\textbf y$, both inner product calculated above must be equal to zero. Hence the system of equations to solve
$-2a + 3b =0 \\ 3a + 2b + 16 = 0$
Solve the above system to to obtain
$a = -\dfrac{48}{13} , b = - \dfrac{32}{13}$

Example 3
Let $\textbf x = \begin{bmatrix} \sqrt 2 \\ 0 \\ 1 \end{bmatrix}$ and $\textbf y = \begin{bmatrix} 0 \\ \sqrt 5 \\ 0 \end{bmatrix}$.
Find $|| \textbf x ||$, $|| \textbf y ||$ and $|| \textbf x + \textbf y||$ and compare $|| \textbf x ||^2 + || \textbf y ||^2$ and $|| \textbf x + \textbf y||^2$

Solution to Example 3
Use formula for the definition of the length of a vector
$|| \textbf x || = \sqrt { (\sqrt 2)^2 + 0^2 + 1^2 } = \sqrt 3$
$|| \textbf y || = \sqrt { 0^2 + (\sqrt 5)^2 + 0^2 } = \sqrt 5$
$|| \textbf x + \textbf y|| = \sqrt { (\sqrt 2)^2 + (\sqrt 5)^2 + 1} = \sqrt 8$
$|| \textbf x ||^2 + || \textbf y ||^2 = 3 + 5 = 8$
$|| \textbf x + \textbf y||^2 = 8$
We notice that $|| \textbf x + \textbf y||^2 = || \textbf x ||^2 + || \textbf y ||^2$ and that is becaues vectors $\textbf x$ and $\textbf y$ are orthogonal (the inner product of two vectors is equal to 0) which verify the Pythagorean theorem above.