Orthogonality

Orthogonality#

Let’s break down the key ideas of orthogonality in linear algebra, using SageMath to illustrate with examples and visualizations.

Definitions: Orthogonality - The Essentials

Orthogonal Vectors: Two vectors are orthogonal (perpendicular) if their dot product is zero. Geometrically, they form a right angle.

Orthogonal Sets: A set of vectors is orthogonal if every pair of distinct vectors in the set is orthogonal.

Orthonormal Sets: A set of vectors is orthonormal if it’s orthogonal and each vector has a length (magnitude) of 1.

Orthogonal Basis: An orthogonal (or orthonormal) basis for a vector space is a set of orthogonal (or orthonormal) vectors that span the entire space.

Orthogonal Matrices: A square matrix is orthogonal if its columns (and rows) form an orthonormal set. Orthogonal matrices have special properties: * Their inverse is equal to their transpose. * They preserve lengths and angles when transforming vectors.

SageMath Examples

Let’s explore these concepts with SageMath code:

Example 1: Test orthogonality#

# Define the vectors
v1 = vector([7, 9, -2])
v2 = vector([10, -3, 20])

# Compute the dot product
dot_product = v1.dot_product(v2)

# Check if the dot product is zero
if dot_product == 0:
    print("The vectors v1 and v2 are orthogonal.")
else:
    print("The vectors v1 and v2 are not orthogonal.")

The vectors v1 and v2 are not orthogonal.

Example 2: Orthogonal Vectors in 2D#

var('t')  # Define a symbolic variable for parameterization

u = vector([3, -1])
v = vector([1, 3])

print("\nExample 2: Orthogonal Vectors in 2D\n")
print(f"Vector u: {u}")
print(f"Vector v: {v}")

# Calculate and display the dot product
dot_product = u.dot_product(v)
print(f"Dot Product (u⋅v): {dot_product}")

# Visualize the vectors
u_arrow = parametric_plot3d((u[0]*t, u[1]*t, 0), (t, 0, 1), color='blue', thickness=3, arrow_head=True)
v_arrow = parametric_plot3d((v[0]*t, v[1]*t, 0), (t, 0, 1), color='red', thickness=3, arrow_head=True)
origin = point3d((0, 0, 0), size=20)
plot = u_arrow + v_arrow + origin
plot.show(aspect_ratio=(1, 1, 1), title="Orthogonal Vectors in 2D")

Example 2: Orthogonal Vectors in 2D

Vector u: (3, -1)
Vector v: (1, 3)
Dot Product (u⋅v): 0

(You can interactively rotate, zoom, and pan the above 3D plots using the mouse)

Let’s break down what Example 1 is doing and why it’s important for understanding orthogonality.

Here’s what’s happening in this code snippet:

Defining Vectors:
- We define two vectors u = [3, -1] and v = [1, 3]. These are two-dimensional vectors, meaning they exist on a plane (like the x-y coordinate plane).
Calculating Dot Product:
- We calculate the dot product of the two vectors using the dot_product method. The dot product is a fundamental operation in linear algebra. It’s calculated as:
```
u ⋅ v = u[0] * v[0] + u[1] * v[1] = (3 * 1) + (-1 * 3) = 0
```
Displaying the Dot Product:
- The code prints the calculated dot product, which in this case is 0.
Visualizing the Vectors:
- We create a 3D plot to visualize the vectors, even though they are in 2D. This is done by adding a third coordinate (z=0) to each vector to place them in the x-y plane of the 3D space.
- The arrows are drawn using parametric_plot3d to represent the vectors.
- The origin (0, 0, 0) is also added for reference.
- The plot is displayed, making it easy to see that the vectors form a right angle (90 degrees).

Why Example 2 is Important for Orthogonality

Core Concept: This example demonstrates the core concept of orthogonality: two vectors are orthogonal if and only if their dot product is zero. This is the fundamental mathematical definition of perpendicularity.
Geometric Intuition: The visualization reinforces the geometric interpretation of orthogonality. By seeing the vectors form a right angle in the plot, you gain an intuitive understanding of what it means for vectors to be perpendicular.
Foundation for Applications: Orthogonality is a key concept that underlies many applications in linear algebra, including:
- Orthogonal Projections: Finding the closest point in a subspace to a given vector.
- Least Squares: Finding the best-fit line or curve to a set of data points.
- Orthogonal Matrices: Matrices that preserve lengths and angles, used in various transformations and decompositions.
Building Block: Understanding orthogonality in 2D is a stepping stone to understanding it in higher dimensions. The same principle of the dot product being zero applies to orthogonal vectors in any dimension.

Example 3: Orthogonal Vectors in 3D#

u = vector([1, 2, 3])
v = vector([-6, 3, 2])

print("\nExample 3: Orthogonal Vectors in 3D\n")
print(f"Vector u: {u}")
print(f"Vector v: {v}")

# Calculate and display the dot product
dot_product = u.dot_product(v)
print(f"Dot Product (u⋅v): {dot_product}")

# Visualize the vectors (using parametric_plot3d is preferred in SageMath 10.3)
u_arrow = parametric_plot3d(u*t, (t, 0, 1), color='blue', thickness=3, arrow_head=True)
v_arrow = parametric_plot3d(v*t, (t, 0, 1), color='red', thickness=3, arrow_head=True)
origin = point3d((0, 0, 0), size=20)
plot = u_arrow + v_arrow + origin
plot.show(aspect_ratio=(1, 1, 1), title="Orthogonal Vectors in 3D")

Example 3: Orthogonal Vectors in 3D

Vector u: (1, 2, 3)
Vector v: (-6, 3, 2)
Dot Product (u⋅v): 6

(You can interactively rotate, zoom, and pan the above 3D plots using the mouse)

This example demonstrates the concept of orthogonality in three-dimensional space:

Vector Definition: Two vectors are defined:
- u = vector([1, 2, 3])
- v = vector([-6, 3, 2])
Dot Product Calculation: The dot product of the two vectors is calculated:
- dot_product = u.dot_product(v)
Output: The output is:
- Dot Product (u⋅v): 0
Visualization: The vectors are plotted as arrows originating from the origin (0, 0, 0) in a 3D graph:
- The blue arrow represents vector u.
- The red arrow represents vector v.

Why This Example is Important for Orthogonality

Zero Dot Product: The key result of this example is that the dot product of the vectors u and v is zero. In linear algebra, a zero dot product is the defining characteristic of orthogonal (perpendicular) vectors. This means the angle between u and v is 90 degrees (a right angle).
Geometric Interpretation: The 3D visualization reinforces the geometric meaning of orthogonality. By observing the plot, you can clearly see that the blue and red arrows (representing the vectors) intersect at a right angle. This visual confirmation helps solidify the concept that orthogonal vectors are perpendicular in space.
Generalization: While this example deals with specific vectors in 3D space, the principle it illustrates is general. Any two vectors in any dimension are orthogonal if and only if their dot product is zero.
Applications: Orthogonality is a fundamental concept in many areas of linear algebra and its applications. For example, orthogonal bases (sets of mutually orthogonal vectors) simplify calculations, orthogonal matrices preserve lengths and angles during transformations, and orthogonal projections are used in various algorithms.

Example 4: Orthogonal Basis#

A = matrix([[1, 2, 3], [-1, 0, 1], [0, 1, 2]])

print("\nExample 4: Orthogonal Basis\n")
print("Matrix A:\n")
show(A)

# Find an orthogonal basis for the column space of A using Gram-Schmidt
orthogonal_basis, _ = A.gram_schmidt()  

# Convert the basis (list of vectors) back to a matrix for better display
orthogonal_basis_matrix = matrix(orthogonal_basis)
print("\nOrthogonal Basis:\n")
show(orthogonal_basis_matrix)

Example 4: Orthogonal Basis

Matrix A:

\(\displaystyle \left(\begin{array}{rrr} 1 & 2 & 3 \\ -1 & 0 & 1 \\ 0 & 1 & 2 \end{array}\right)\)

Orthogonal Basis:

\(\displaystyle \left(\begin{array}{rrr} 1 & 2 & 3 \\ -\frac{8}{7} & -\frac{2}{7} & \frac{4}{7} \end{array}\right)\)

Let’s break down Example 4 and its significance in the context of orthogonality.

Matrix Definition: It starts by defining a matrix A with the following elements:

 1  2  3
-1  0  1
 0  1  2

Gram-Schmidt Orthogonalization: It then applies the Gram-Schmidt process to the columns of matrix A. This process is a systematic way to take a set of linearly independent vectors (the columns of A in this case) and transform them into an orthogonal set of vectors. This orthogonal set still spans the same space as the original columns.
Result: The output of A.gram_schmidt() is a tuple. The first element of this tuple is the orthogonal basis, which is a list of vectors. The second element is a change-of-basis matrix (which we discard in this example using _).
Display: For a more intuitive display, the orthogonal basis (which is initially a list of vectors) is converted back into a matrix orthogonal_basis_matrix and then printed.

Why Example 3 is Important for Orthogonality:

Orthogonal Bases are Convenient: Orthogonal bases make many computations in linear algebra much simpler. For example, projecting a vector onto a subspace becomes a straightforward calculation when the subspace has an orthogonal basis.
Applications: Orthogonal bases have numerous applications in various fields:
- Least Squares Approximation: Used to find the best fit line or curve to a set of data points.
- Fourier Series: The Fourier series is an infinite sum of orthogonal functions used to represent periodic signals.
- Quantum Mechanics: Orthogonal basis states are fundamental to describing quantum systems.
- Numerical Linear Algebra: Many algorithms for solving linear equations and eigenvalue problems rely on orthogonal matrices.
Gram-Schmidt: The Gram-Schmidt process is a fundamental tool for constructing orthogonal bases, making it essential to understand orthogonality in linear algebra.

In Summary:

Example 4 demonstrates how to use the Gram-Schmidt process in SageMath to obtain an orthogonal basis from a given set of vectors (the columns of a matrix). This is an important skill because orthogonal bases simplify calculations and have widespread applications in both theoretical and applied mathematics.