Assignment 1
(Winograd's Variant of Strassen's Algorithm)

Due Tue. July 5 by the start of class.

Overview

In 1969, Volker Strassen discovered an algorithm to multiply two n X n matrices in time O(n^(2.81)) instead of O(n^3) as is required by the standard algorithm. The algorithm is based on a clever way to multiply 2 X 2 matrices using 7 multiplications and 18 additions/subtractions. On the face of it this is worse than the standard approach which uses 8 multiplications and 4 additions; however, when applied to 2 X 2 matrices whose elements are m X m matrices this observation leads to a factor of 7/8's improvement as m gets large since matrix addition is of order n^2 and matrix multiplication is of order n^3 when the standard algorithm is used. The improvement is even greater when Strassen's trick is used recursively for the 7 products of size m. The computing time satisfies the recurrence T(n) = 7*T(n/2) + 18*(n/2)^2, which can be shown to be O(n^(2.81)).

Shortly after Strassen's result was announced, S. Winograd found a way to multiply 2 X 2 matrices using 7 multiplications and 15 additions/subtractions. This observation improves the constant but does not change the asymptotic computing time. Winograd subsequently showed that 7 multiplications and 15 additions/subtractions is optimial for 2 X 2 matrix multiplication. His approach is based on the following equations.

             [ A11  A12 ]                [ B11  B12 ]               [ C11  C12 ]
Let A = [                 ]  and  B = [                 ]  and C = [                 ] = A*B.
             [ A21  A22]                 [ B21  B22 ]               [ C21  C22 ]

S1 = A21 + A22,  T1 = B12 - B11
S2 = S1 - A11,     T2 = B22 - T1
S3 = A11 - A21,  T3 = B22 - B12
S4 = A12 - S2,    T4 = B21 - T2

P1 = A11*B11
P2 = A12*B21
P3 = S1*T1
P4 = S2*T2
P5 = S3*T3
P6= S4*B22
P7 = A22*T4

U1 = P1 + P2
U2 = P1 + P4
U3 = U2 + P5
U4 = U3 + P7
U5 = U3 + P3
U6 = U2 + P3
U7 = U6 + P6

Then C11 = U1, C12 = U7, C21 = U4, and C22 = U5.
 

Problems

1. Verify that Winograd's variant of Strassen's algorithm is correct. I.E. symbolically evaluate the equations to verify that the compute the product of two 2 X 2 matrices.
2. Assume n = 2^k. Write down and solve the recurrence relation for the number of operations used by Strassen's algorithm and Winograd's variant. Plot the number of operations of the different algorithms. Compute the ratio of the number of operations used by Winograd's variant compared to Strassen's algorithm as k goes to infinity.
3. Compute the ratio of the number of operations used by Winograd's variant of Strassen's algorithm to the standard algorithm. Plot the ratio as k becomes sufficient large so that the asymptotic behavior is apparent. Plot the ratio. When does Strassen's algorithm become faster?
4. The cross over point observed in the previous question convinced many people that Strassen's algorithm (Winograd's variant) was impractical; however, if a hybrid algorithm is used, where the regular algorithm is used when it is faster than applying Strassen's algorithm, then the cross over point is much smaller. Find the optimal hybrid algorithm and determine the point at which it becomes faster than the regular algorithm.
5. As stated Strassen's algorithm requires the input size to be a power of two. However, this limitation can be removed by padding with zeros. Verify that if A and B are n X n matrices and are embedded in the lower left hand corner of (n+m) X (n+m) matrices A' and B' whose last m columns and rows are zero than the lower left hand n X n block of A'*B' = A*B. This observation provides several strategies for applying Strassen's algorithm for arbitrarily sized matrices. For each approach below write a function that determines the number of arithmetic operations used to multiply n X n matrices. By computing ratios and showing plots compare the different approaches. Obtain an exact count and comparison for n = 2^k + 1 and n = 2^k - 1.
1. Embed A and B in 2^k X 2^k matrices A' and B' where k is the smallest integer such that n <= 2^k.
2. At each recursive step of Strassen's algorithm, if n is odd add an extra row and column of zeros so that A and B are embedded in (n+1) X (n+1) matrices A' and B'.  If n is even no embedding is necessary.
3. At each recursive step in Strassen's algorithm, if n is odd view A and B as 2 X 2 matrices whose upper left hand block A11 and B11 are (n-1) X (n-1) matrices.  Perform the multiplication of A11*B11 using a recursive call to Strassen's algorithm.  The remaining multiplications are performed using ordinary matrix multiplication.

What to Hand in