Lecture 2: Implementation and Optimization of Matrix Multiplication

Background Material

• C programming language.
• Assembly language - IA32 (Chapter 3 of Bryant and O'Hallaron)

Reading

• PAPI (Performance Application Programming Interface)
• Gnu Assembly Reference Manual
• Numerical Recipes - code for matrix multiplication.
• Memory Hierarchy (Chapter 6 of Bryant and O'Hallaron) - material on matrix multiplication loop variants - handout.
• See Lecture 1 notes on Strassen's algorithm, which discusses block matrix multiplication.

Topics

• Performance of Matrix Multiplication
• Numerical Recipes
  • matrix.c
  • matrix.h
  • block.c
  • timemm.c
  • timeblockmm.c
• Optimization flags
• Machine parameters (/proc/cpuinfo, mem_info, cpuid, manuals)
• Variants of Matrix multiplication (rearranging loops, block matrix multiplication)
• Performance counters (PAPI)
• Analysis of Assembly code
  • Counting instructions (assembly code - performance counters)
  • loop.c
  • Makefile
  • testloop.c
  • timeloop.c
  • countloop.c
  • loop_unopt.s
  • loop_opt.s

Tasks

• Time and count instructions for loop.c (optimized and unoptimized)
• Determine the number of cycles per iteration for loop.c (optimized and unoptimized)
• Time and instrument matrix multiplication code.
• Experiment with variants of matrix multiplication.

Lecture Notes

• None.

Resources

• BLAS (Basic Linear Algebra Subprograms
• Intel Math Kernel Library (MKL)
• ATLAS (Automatically Tuned Linear Algebra Software
• PAPI (Performance Application Programming Interface)
• LMbench (Tools for Performance Analysis)
• Numerical Recipes
• Gnu Assembly Reference Manual
• Computer Systems: A Programmer's Perspective

Assignments

• Assignment 1