Inner Product, Orthogonality, and Vector Length in Linear Algebra

The concepts of the inner product, the condition of orthogonality, and the length of vectors form the foundation of geometric interpretations in linear algebra. These are presented below along with detailed, step-by-step examples.

Table of Contents

Definition of the Inner Product of Two Vectors

Let vectors \( \textbf{x} \) and \( \textbf{y} \) be two column vectors (or \( n \times 1 \) matrices) defined by:

\[ \textbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}, \quad \textbf{y} = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix} \]

The inner product (or dot product) of \( \textbf{x} \) and \( \textbf{y} \) is a scalar quantity written as \( \textbf{x} \cdot \textbf{y} \), defined by:

\[ \textbf{x} \cdot \textbf{y} = x_1 y_1 + x_2 y_2 + \dots + x_n y_n \]

Properties of the Inner Product

If \( \textbf{x}, \textbf{y} \), and \( \textbf{z} \) are vectors in \( \mathbb{R}^n \) and \( k_1, k_2 \) are scalars, then the following properties hold:

  1. Commutativity: \( \textbf{x} \cdot \textbf{y} = \textbf{y} \cdot \textbf{x} \)
  2. Distributivity: \( (\textbf{x} + \textbf{y}) \cdot \textbf{z} = \textbf{x} \cdot \textbf{z} + \textbf{y} \cdot \textbf{z} \)
  3. Scalar Multiplication: \( (k_1 \textbf{x}) \cdot (k_2 \textbf{y}) = (k_1 k_2) (\textbf{x} \cdot \textbf{y}) \)

Orthogonal Vectors

Vectors \( \textbf{x} \) and \( \textbf{y} \) are called orthogonal (perpendicular) if their inner product is equal to zero:

\[ \textbf{x} \cdot \textbf{y} = 0 \]

Definition of the Length (or Norm) and Unit Vector

The length (or norm) of a vector \( \textbf{x} \), written as \( \|\textbf{x}\| \), is given by the square root of the inner product of the vector with itself:

\[ \|\textbf{x}\| = \sqrt{x_1^2 + x_2^2 + \dots + x_n^2} = \sqrt{\textbf{x} \cdot \textbf{x}} \]

From this definition, it follows that \( \|\textbf{x}\| \ge 0 \) and \( \|\textbf{x}\|^2 = \textbf{x} \cdot \textbf{x} \).

A unit vector is any vector whose length (or norm) is equal to 1. To normalize a vector, divide it by its norm:

\[ \hat{\textbf{x}} = \frac{\textbf{x}}{\|\textbf{x}\|} \]

Cauchy-Schwarz and Triangle Inequalities

Two fundamental inequalities follow directly from the inner product and norm:

Cauchy-Schwarz Inequality: For any vectors \( \textbf{x} \) and \( \textbf{y} \) in \( \mathbb{R}^n \):

\[ |\textbf{x} \cdot \textbf{y}| \leq \|\textbf{x}\| \, \|\textbf{y}\| \]

Triangle Inequality: For any vectors \( \textbf{x} \) and \( \textbf{y} \) in \( \mathbb{R}^n \):

\[ \|\textbf{x} + \textbf{y}\| \leq \|\textbf{x}\| + \|\textbf{y}\| \]

Pythagorean Theorem & Distance Between Vectors

Pythagorean Theorem: Vectors \( \textbf{x} \) and \( \textbf{y} \) are orthogonal if and only if:

\[ \|\textbf{x} + \textbf{y}\|^2 = \|\textbf{x}\|^2 + \|\textbf{y}\|^2 \]

Distance: The distance between two vectors \( \textbf{x} \) and \( \textbf{y} \) is defined as the length of their difference:

\[ \text{dist}(\textbf{x}, \textbf{y}) = \|\textbf{x} - \textbf{y}\| \]

Examples with Solutions

Example 1: Calculating Inner Products

Show Solution

Given the vectors:

\[ \textbf{x} = \begin{bmatrix} -2 \\ 3 \\ 0 \\ -1 \end{bmatrix}, \quad \textbf{y} = \begin{bmatrix} 3 \\ -1 \\ 4 \\ 0 \end{bmatrix}, \quad \textbf{z} = \begin{bmatrix} 2 \\ -1 \\ 4 \\ -7 \end{bmatrix} \]

Find:

a) \( \textbf{x} \cdot \textbf{y} \) and \( \textbf{y} \cdot \textbf{x} \)
b) \( \textbf{x} \cdot \textbf{y} + \textbf{x} \cdot \textbf{z} \) and \( \textbf{x} \cdot (\textbf{y} + \textbf{z}) \)
c) \( (3\textbf{x}) \cdot (-2\textbf{z}) \)

Solution

a) Calculate \( \textbf{x} \cdot \textbf{y} \) using the inner product definition:

\[ \textbf{x} \cdot \textbf{y} = (-2)(3) + (3)(-1) + (0)(4) + (-1)(0) = -6 - 3 = -9 \]

By the commutativity property (Property 1), \( \textbf{y} \cdot \textbf{x} = \textbf{x} \cdot \textbf{y} = -9 \).

b) First, find \( \textbf{x} \cdot \textbf{z} \):

\[ \textbf{x} \cdot \textbf{z} = (-2)(2) + (3)(-1) + (0)(4) + (-1)(-7) = -4 - 3 + 0 + 7 = 0 \]

Then, \( \textbf{x} \cdot \textbf{y} + \textbf{x} \cdot \textbf{z} = -9 + 0 = -9 \).

According to the distributivity property (Property 2), \( \textbf{x} \cdot (\textbf{y} + \textbf{z}) \) must yield the same result. Let's verify by calculating \( \textbf{y} + \textbf{z} \) first:

\[ \textbf{y} + \textbf{z} = \begin{bmatrix} 5 \\ -2 \\ 8 \\ -7 \end{bmatrix} \implies \textbf{x} \cdot (\textbf{y} + \textbf{z}) = (-2)(5) + (3)(-2) + (0)(8) + (-1)(-7) = -10 - 6 + 0 + 7 = -9 \]

c) Using the scalar multiplication property (Property 3) and the result from part b) that \( \textbf{x} \cdot \textbf{z} = 0 \):

\[ (3\textbf{x}) \cdot (-2\textbf{z}) = (3)(-2)(\textbf{x} \cdot \textbf{z}) = -6(0) = 0 \]

Example 2: Orthogonal Vectors and Unknown Constants

Show Solution

a) Show that vectors \( \textbf{x} = \begin{bmatrix} -2 \\ 3 \\ 0 \end{bmatrix} \) and \( \textbf{y} = \begin{bmatrix} 3 \\ 2 \\ 4 \end{bmatrix} \) are orthogonal.

b) Find the constants \( a \) and \( b \) so that the vector \( \textbf{z} = \begin{bmatrix} a \\ b \\ 4 \end{bmatrix} \) is orthogonal to both vectors \( \textbf{x} \) and \( \textbf{y} \).

Solution

a) Calculate the inner product of \( \textbf{x} \) and \( \textbf{y} \):

\[ \textbf{x} \cdot \textbf{y} = (-2)(3) + (3)(2) + (0)(4) = -6 + 6 + 0 = 0 \]

Since the inner product is zero, the two vectors are orthogonal.

b) Calculate the inner product of \( \textbf{z} \) with both \( \textbf{x} \) and \( \textbf{y} \):

\[ \textbf{x} \cdot \textbf{z} = -2(a) + 3(b) + 0(4) = -2a + 3b \] \[ \textbf{y} \cdot \textbf{z} = 3(a) + 2(b) + 4(4) = 3a + 2b + 16 \]

For vector \( \textbf{z} \) to be orthogonal to both \( \textbf{x} \) and \( \textbf{y} \), both inner products must equal zero. This sets up a system of linear equations:

\[ \begin{cases} -2a + 3b = 0 \\ 3a + 2b + 16 = 0 \end{cases} \]

From the first equation, \( 2a = 3b \implies a = 1.5b \). Substitute this into the second equation:

\[ 3(1.5b) + 2b + 16 = 0 \implies 4.5b + 2b = -16 \implies 6.5b = -16 \implies b = -\frac{32}{13} \]

Now find \( a \):

\[ a = 1.5 \left(-\frac{32}{13}\right) = -\frac{48}{13} \]

The constants are \( a = -\frac{48}{13} \) and \( b = -\frac{32}{13} \).

Example 3: Vector Lengths and the Pythagorean Theorem

Show Solution

Let \( \textbf{x} = \begin{bmatrix} \sqrt{2} \\ 0 \\ 1 \end{bmatrix} \) and \( \textbf{y} = \begin{bmatrix} 0 \\ \sqrt{5} \\ 0 \end{bmatrix} \).

Find \( \|\textbf{x}\| \), \( \|\textbf{y}\| \), and \( \|\textbf{x} + \textbf{y}\| \). Compare \( \|\textbf{x}\|^2 + \|\textbf{y}\|^2 \) with \( \|\textbf{x} + \textbf{y}\|^2 \).

Solution

Use the formula for the length of a vector:

\[ \|\textbf{x}\| = \sqrt{(\sqrt{2})^2 + 0^2 + 1^2} = \sqrt{2 + 0 + 1} = \sqrt{3} \] \[ \|\textbf{y}\| = \sqrt{0^2 + (\sqrt{5})^2 + 0^2} = \sqrt{0 + 5 + 0} = \sqrt{5} \]

Now, calculate the sum vector \( \textbf{x} + \textbf{y} \):

\[ \textbf{x} + \textbf{y} = \begin{bmatrix} \sqrt{2} \\ \sqrt{5} \\ 1 \end{bmatrix} \]

Find its length:

\[ \|\textbf{x} + \textbf{y}\| = \sqrt{(\sqrt{2})^2 + (\sqrt{5})^2 + 1^2} = \sqrt{2 + 5 + 1} = \sqrt{8} \]

Compare the squared magnitudes:

\[ \|\textbf{x}\|^2 + \|\textbf{y}\|^2 = 3 + 5 = 8 \] \[ \|\textbf{x} + \textbf{y}\|^2 = 8 \]

Conclusion: We notice that \( \|\textbf{x} + \textbf{y}\|^2 = \|\textbf{x}\|^2 + \|\textbf{y}\|^2 \). This occurs because vectors \( \textbf{x} \) and \( \textbf{y} \) are orthogonal (\( \textbf{x} \cdot \textbf{y} = 0 \)), perfectly verifying the Pythagorean theorem for vector spaces.

Frequently Asked Questions

What is the inner product of two vectors?

The inner product (dot product) of vectors \( \textbf{x} \) and \( \textbf{y} \) is the sum of the products of their corresponding components: \( \textbf{x} \cdot \textbf{y} = x_1 y_1 + x_2 y_2 + \dots + x_n y_n \).

When are two vectors orthogonal?

Two vectors are orthogonal (perpendicular) if and only if their inner product equals zero: \( \textbf{x} \cdot \textbf{y} = 0 \).

How do you calculate the length of a vector?

The length (norm) of a vector \( \textbf{x} \) is the square root of the sum of the squares of its components: \( \|\textbf{x}\| = \sqrt{x_1^2 + x_2^2 + \dots + x_n^2} \).

What is a unit vector?

A unit vector is any vector whose length (norm) is equal to 1. To normalize a vector, divide it by its norm: \( \hat{\textbf{x}} = \frac{\textbf{x}}{\|\textbf{x}\|} \).

What is the Pythagorean theorem for vectors?

Vectors \( \textbf{x} \) and \( \textbf{y} \) are orthogonal if and only if \( \|\textbf{x} + \textbf{y}\|^2 = \|\textbf{x}\|^2 + \|\textbf{y}\|^2 \).

More References and Links