Inner Product, Orthogonality and Length of Vectors
The inner product , the condition of orthogonality and the length of vectors are presented through examples including their detailed solutions.
Definition of the Inner Product of two Vectors
Let vectors \( \textbf {x} \) and \( \textbf y \) be two column vectors (or an \( n \) by \( 1 \) matrix) defined by
\( \textbf x =
\begin{bmatrix}
x_1 \\
x_2 \\
. \\
. \\
. \\
x_n
\end{bmatrix}
\) and
\( \textbf y =
\begin{bmatrix}
y_1 \\
y_2 \\
. \\
. \\
. \\
y_n
\end{bmatrix}
\)
The inner product of \( \textbf x \) and \(\textbf y \) is a scalar quantity written as \( \textbf x \cdot \textbf y \) defined by
\[ \textbf x. \textbf y = \textbf {x}^T \textbf y = \begin{bmatrix}
& x_1 & x_2 & . . . & x_n\\
\end{bmatrix}
\begin{bmatrix}
y_1 \\
y_2 \\
. \\
. \\
. \\
y_n
\end{bmatrix}
= x_1 y_1 + x_2 y_2 + ... + x_n y_n\]
where \( \textbf {x}^T \) is the transpose of vector \( \textbf {x} \)
Properties of the Inner Product
If \( \textbf x \) , \( \textbf y \) and \( \textbf z \) are vectors in \( R^n \) and \( k_1 \) and \( k_2 \) are scalars, then
- \( \textbf x \cdot \textbf y = \textbf y \cdot \textbf x \)
- \( (\textbf x + \textbf y) \cdot \textbf z = \textbf x \cdot \textbf z + \textbf y \cdot \textbf z \)
- \( (k_1 \textbf x) \cdot (k_2 \textbf y) = k_1 k_2 (\textbf x \cdot \textbf y) \)
Orthogonal Vectors
Vectors \( \textbf x \) and \( \textbf y \) are called orthogonal if
\[ \textbf x \cdot \textbf y = 0 \]
Definition of the Length (or Norm) of a Vector and Unit Vector
The length (or norm ) of vector \( \textbf x =
\begin{bmatrix}
x_1 \\
x_2 \\
. \\
. \\
. \\
x_n
\end{bmatrix}
\)
written as \( || \textbf x || \) is given by
\[ || \textbf x || = \sqrt {x_1^2 + x_2^2 + .... + x_n^2} = \sqrt {\textbf x \cdot \textbf x} \]
From the above definition, we can easily conclude that
\( || \textbf x || \ge 0 \) and \( || \textbf x ||^2 = \textbf x \cdot \textbf x \)
A unit vector is a vector whose length (or norm) is equal to 1.
Pythagorean Theorem
Vectors \( \textbf x \) and \( \textbf y \) are orthogonal if and only if
\[ ||x+y||^2 = ||x||^2 + ||y||^2 \]
Distance Between two Vectors
The distance between vectors \( \textbf x \) and \( \textbf y \) is defined as
\[ dist(\textbf x,\textbf y) = || \textbf x - \textbf y || \]
Examples with Solutions
Example 1
Given the vectors \( \textbf x =
\begin{bmatrix}
-2 \\
3 \\
0 \\
-1
\end{bmatrix}
\) ,
\( \textbf y =
\begin{bmatrix}
3 \\
-1 \\
4 \\
0
\end{bmatrix}
\)
,
\( \textbf z =
\begin{bmatrix}
2 \\
-1 \\
4 \\
-7
\end{bmatrix}
\), find
a)
\( \textbf x \cdot \textbf y \) and \( \textbf y \cdot \textbf x \)
b)
\( \textbf x \cdot \textbf y + \textbf x \cdot \textbf z \) and \( \textbf x \cdot (\textbf y + \textbf z) \)
c) \( (3 \textbf x ) \cdot (-2\textbf z) \)
Solution to Example 1
Use the definition given above
a)
\( \textbf x \cdot \textbf y = (-2)(3) + 3(-1) + 0(4) + (-1)0 = - 9 \)
Using property 1 of the inner product above
\( \textbf y \cdot \textbf x = \textbf x \cdot \textbf y = - 9 \)
b)
\( \textbf x \cdot \textbf y + \textbf x \cdot \textbf z = - 9 + 23 = 14 \)
According to property 2 of the inner product
\( \textbf x \cdot (\textbf y + \textbf z) = \textbf x \cdot \textbf y + \textbf x \cdot \textbf z = 14 \)
c)
According to property 3 of the inner product
\( (3 \textbf x ) \cdot (-2\textbf z) = (3)(-2) \textbf x \cdot \textbf y = -6 (-9) = 54 \)
Example 2
a) Show that vectors \( \textbf x =
\begin{bmatrix}
-2 \\
3 \\
0
\end{bmatrix}
\) and
\( \textbf y =
\begin{bmatrix}
3 \\
2 \\
4
\end{bmatrix}
\)
are orthogonal.
b)
Find the constant \( a \) and \( b \) so that the vector
\( \textbf z =
\begin{bmatrix}
a \\
b \\
4
\end{bmatrix}
\) is orthogonal to both vectors \( \textbf x \) and \( \textbf y \)
Solution to Example 2
Calculate the inner product of vectors \( \textbf x \) and \( \textbf y \)
a)
\( \textbf x \cdot \textbf y = (-2)(3) + 3(2) + 0(4) = 0 \)
Since the inner product of vectors \( \textbf x \) and \( \textbf y \) is equal to zero, the two vectors are orthogonal.
b)
We first calculate the following inner product
\( \textbf x \cdot \textbf z = -2(a) + 3(b) + 0(4) = -2a + 3b \)
\( \textbf y \cdot \textbf z = 3(a) + 2(b) + 4(4) = 3a + 2b + 16 \)
For vector \( \textbf z \) to be orthogonal to both \( \textbf x \) and \( \textbf y \), both inner product calculated above must be equal to zero. Hence the system of equations to solve
\( -2a + 3b =0 \\ 3a + 2b + 16 = 0 \)
Solve the above system to to obtain
\( a = -\dfrac{48}{13} , b = - \dfrac{32}{13} \)
Example 3
Let \( \textbf x =
\begin{bmatrix}
\sqrt 2 \\
0 \\
1
\end{bmatrix}
\) and
\( \textbf y =
\begin{bmatrix}
0 \\
\sqrt 5 \\
0
\end{bmatrix}
\).
Find \( || \textbf x || \), \( || \textbf y || \) and \( || \textbf x + \textbf y|| \) and compare \( || \textbf x ||^2 + || \textbf y ||^2 \) and \( || \textbf x + \textbf y||^2 \)
Solution to Example 3
Use formula for the definition of the length of a vector
\( || \textbf x || = \sqrt { (\sqrt 2)^2 + 0^2 + 1^2 } = \sqrt 3 \)
\( || \textbf y || = \sqrt { 0^2 + (\sqrt 5)^2 + 0^2 } = \sqrt 5 \)
\( || \textbf x + \textbf y|| = \sqrt { (\sqrt 2)^2 + (\sqrt 5)^2 + 1} = \sqrt 8\)
\( || \textbf x ||^2 + || \textbf y ||^2 = 3 + 5 = 8 \)
\( || \textbf x + \textbf y||^2 = 8 \)
We notice that \( || \textbf x + \textbf y||^2 = || \textbf x ||^2 + || \textbf y ||^2 \) and that is becaues vectors \( \textbf x \) and \( \textbf y \) are orthogonal (the inner product of two vectors is equal to 0) which verify the Pythagorean theorem above.
More References and links
- Vector Spaces - Questions with Solutions
- Linear Algebra and its Applications - 5 th Edition - David C. Lay , Steven R. Lay , Judi J. McDonald
- Elementary Linear Algebra - 7 th Edition - Howard Anton and Chris Rorres
