Lecture 6: WHT Package - A self-adapting package to compute the Walsh-Hadamard Transform (WHT).

Background Material

• Matrices and Matrix Multiplication
• C/C++/Ruby Programming

Reading

• Jeremy Johnson and Markus Pueschel (2000), "In Search of the Optimal Walsh-Hadamard Transform", Proc. ICASSP 2000, pp. 3347-3350 (wht.ps,wht.pdf)

Topics

• Walsh-Hadamard Transform (WHT) (see JP00, WHT package)
1. Definition of the Walsh-Hadamard Transform (WHT)
2. Tensor Product and a factorization of the WHT
3. Space of algorithms for computing the WHT
4. Number of flops/instructions
5. WHT package
6. Performance distribution of WHT algorithms
7. Searching for the optimal WHT algorithm with dynamic programming
• Matrix operations and the tensor product (See JJRT90, HJ91, TAL89, vL92)
1. Partitioned matrices and block matrix multiplication
2. Matrix transposition: (AB)^T = B^T A^T
3. Permutation matrices: exactly one 1 in each row and column.
   P_pq = P_q P_p, P^(-1) = P^T, (PAP^T)_ij = A_p(i)p(j)
4. Stride Permutations: L^(mn)_n X = X_n = (x_0,x_n,...,x_(m-1)n,...,x_(n-1),x_(n-1 + n),...,x_(n-1 +(m-1)n)). (L^(mn)_n)^T = L^(mn)_m.
5. Tensor (Kronecker) Product: ij block of A tensor B = a_ijB
6. Properties of tensor product
   1. (aA) tensor B = a(A tensor B) = A tensor (aB)
   2. (A1 + A2) tensor B = (A1 tensor B) + (A2 tensor B)
   3. A tensor (B1 + B2) = (A tensor B1) + (A tensor B2)
   4. 1 tensor A = A = A tensor 1
   5. (A tensor (B tensor C)) = ((A tensor B) tensor C)
   6. (A tensor B)^T = (A^T tensor B^T)
   7. (A tensor B)(C tensor D) = AC tensor BD
   8. (A tensor B)(x tensor y) = Ax tensor By
   9. (A tensor B) = (A tensor I)(I tensor B) = (I tensor B)(A tensor I)
   10. (A1 tensor ... tensor At)(B1 tensor ... tensor Bt) = A1B1 tensor ... tensor AtBt
   11. (A1 ... At) tensor (B1 ... Bt) = (A1 tensor B1) ... (At tensor Bt)
   12. (A tensor B)^(-1) = A^(-1) tensor B^(-1)
   13. I_m tensor I_n = I_mn
   14. tensor product of [permutation, diagonal, monomial] matrices are [permutaiton, diagonal, monomial].
   15. A tensor B = L^(mn)_m(B tensor A) L^(mn)_n
7. Programming tensor products (see JJRT90, JJ92, Pit97)

Lecture notes and slides

• WHT Lecture (wht_slides.ppt), wht_slides.pdf)
• Tensor Product notes (tensor_notes.pdf)

Programs and References

• WHT package
• spiral_wht-2.0-pre-1.tar.gz
(developers WHT package)
• Jeremy Johnson and Markus Püschel (2000), "In Search of the Optimal Walsh-Hadamard Transform," Proc. ICASSP 2000, pp. 3347-3350. (wht.ps,wht.pdf)
• J. Johnson, R.W. Johnson, D. Rodriguez, and R. Tolimieri (1990), "A Methodology for Designing, Modifying, and Implementing Fourier Transform Algorithms on Various Architectures," Circuits, Systems, and Signal Processing 9, 449-500. (fft.ps,wht.pdf)
• J. Johnson and R.W. Johnson (1992), "Programming Schemata for Tensor Product," Technical report DU-MCS-92-01, Dept. of Mathematics and Computer Science, Drexel University. (schema.ps, schema.pdf)
• Roger A. Horn and Charles R. Johnson (1991). Topics in Matrix Analysis. Cambridge University Press, Cambridge.
• N. P. Pitsianis (1997), The Kronecker Product in Approximation and Fast Transform Generation, PhD Thesis, Department of Computer Science, Cornell University, Jan. 1997, Advisor Charles F. Van Loan.
• R. Tolimieri, M. An, and C. Lu (1989). Algorithms for the Discrete Fourier Transform and Convolution. Springer-Verlag, New York. Second edition, 1997.
• Charles Van Loan (1992). Computational Frameworks for the Fast Fourier Transform. Frontiers in Applied Mathematics, Society for Industrial and Applied Mathematics, Philadelphia, Vol. 10.

Tasks

• Prove by induction that I tensor A1...At = (I tensor A1)...(I tensor At), or more generally A1...At tensor B1...Bt = (A1 tensor B1)...(At tensor Bt).
• Install and test WHT package
• Implement recursive and iterative WHT algorithms (in place)
• Implement radix 4 recursive and iterative WHT algorithms (in place)
• Perform timings for recursive and iterative WHT with different sizes of small. Also measure small and compare to iterative.
• Time random WHTs and produce histogram
• Run dynamic programming and compare to recursive, iterative and random.