Lecture 1: Three Divide and Conquer Algorithms

Background Material

• Chapters 1-6 of Cormen, Lieserson, and Rivest
• Master theorem: the solution to the recurrence T(n) = aT(n/b) + Theta(n) is Theta(n) when a < b, Theta(nlogn) when a=b, Theta(n^log_b(a)) when a > b.
• Recurrence Relations
• Geometric series: sum_{i=0}^{n-1} r^i = (r^n-1)/(r-1) when r &ne 1
• Summation Notation and Manipulation
• Growth Rates and Asymptotic Analysis

• Cooley, J. M. and J. W. Tukey, "An algorithm for the machine calculation of complex Fourier series", Math. Comp., 19, 297-301, 1965.
• A. Karatsuba and Y. Ofman, "Multiplication of multidigit numbers on automata," Dokl. Akad. Nauk SSSR 145, 293-294, 1962.
• V. Strassen, "Gaussian elimination is not optimal", Numerische Mathematik, 13, 354-356, 1969.
• S. Huss-Lederman and E.M. Jacobson and A. Tsao and T. Turnbull and J.R. Johnson, Implementation of {S}trassen's algorithm for matrix multiplication, Proceedings of the 1996 ACM/IEEE conference on Supercomputing.

Topics

• Overview of the course.
• Objectives
• Topics
• Three Problems/Three Algorithms
• Matrix Multiplication and Strassen's algorithm
• Discrete Fourier transform (DFT) and the fast Fourier transform (FFT)
• Integer multiplication and Karatsuba's algorithm

Lecture Notes

• lec1p0.pdf (Part zero - overview - of lecture notes)
• lec1p1.pdf (Part one - Strassen - of lecture notes)
• lec1p2.pdf (Part two - FFT - of lecture notes)
• lec1p3.pdf (Part three - Karatsuba - of lecture notes)

Assignments

• Prove that the product of an m digit integer with an n digit integer can have either m+n or m+n-1 digits.
• Prove the master theorem. To simplify the argument, assume that n = b^k. Hint - use repeated substitution to unwind the recurrence and then use the formula for geometric series.
• Practice Assignment on Strassen's Algorithm (not to be submitted)
• For additional discussion on implementing Winograd's variant of Strassen's algorithm see: P.D'Alberto and A.Nicolau, Adaptive Winograd's Matrix Multiplications, ACM Transaction on Mathematical Software 2008.
Created: Sept. 27, 2006 (revised Sept. 25, 2009) by jjohnson AT cs DOT drexel DOT edu