Visualizing linear algebra: Dot product

Figure 1: Column vector as an arrow

This is Part 5 in a series on linear algebra [1].

Figure 1 shows a vector that is represented geometrically as a yellow arrow and numerically as a column. It scales the unit vectors i-hat and j-hat by 2 and 4 respectively. That is:

v = 2î + 4ĵ

Now suppose the column is transposed to a row, as follows:

[2]^T = [2 4]
[4]

This row is a 2x1 matrix that represents the transformation of a 2D vector space to a 1D vector space, that is, a number line. The number line is the span of the yellow arrow (i.e., it extends along the yellow arrow in both directions).

Figure 2: Row vector as a transformation matrix

The effect of this transformation on the unit vectors is shown in Figure 2 where i-hat is scaled by 2 and j-hat is scaled by 4. The length of the yellow arrow, per Pythagoras' Theorem, is √(2² + 4²) =  √20, or about 4.47.

For any given input vector, the position on the number line where the input vector ends up will be the vector's X coordinate scaled by 2 plus the vector's Y coordinate scaled by 4.

Figure 3: Calculating the dot product

For example, the purple arrow in Figure 3 is transformed to a 1D vector, or scalar, on the number line as follows:

[2 4][2] = 2[2] + 1[4] = 8
     [1]

Geometrically, this is equivalent to projecting the purple arrow onto the span of the yellow arrow (at position 8/√20, or 1.79) and scaling by the length of the yellow arrow (√20, or 4.47). The small right-angled triangle in Figure 3 shows the projection.

In the symmetrical example where the column for the purple arrow is transposed to a row (representing a transformation to the number line that is the span of the purple arrow), the yellow arrow is transformed as follows:

[2 1][2] = 2[2] + 4[1] = 8
     [4]

The scalar result is the same as for the first example - the vector order doesn't matter. However geometrically, it is equivalent to projecting the yellow arrow onto the span of the purple arrow (at position 8/√5, or 3.58) and scaling by the the length of the purple arrow (√(2² + 1²) =  √5, or 2.24). The large right-angled triangle in Figure 3 shows the projection.

More generally, this operation is the dot product (or scalar product) of two vectors and for 2D vectors is defined [2] as:

a.b = a_xb_x + a_yb_y

When the arrows are pointing in the same general direction the dot product is positive, when pointing in the opposite direction the dot product is negative, and when the arrows are orthogonal the dot product is zero.

Figure 4: Dot product = transform

Numerically the dot product multiplies two vectors, taking direction into account [3].

Geometrically, one vector encodes a linear transformation that projects space onto a number line and scales the projection by the length of the vector.

Next up: the cross product

--

[1] The figures and examples of the posts in this series are based on the Essence of Linear Algebra series by 3Blue1Brown.

[2] The dot product is also defined as

a.b = ∥a∥∥b∥cos(θ)

where ∥v∥ denotes the magnitude of vector v. This can also be interpreted as projecting one of the vectors onto the other vector (via the cosine) and scaling by the other vector.

[3] The numerical summary comes from Better Explained which also offers further analogies.