CS 730 Parallel Processing

Course Description

Covers a variety of paradigms and languages for programming parallel computers. Several tools for debugging and measuring the performance of parallel programs will be introduced. Issues related to writing correct and efficient parallel programs will be emphasized. Students will have ample opportunity to write and experiment with parallel programs using a variety of parallel programming environments.

Course Objective

To be able to design, implement, and analyze correct and efficient parallel programs. To become familiar with existing parallel computers and programming environments for parallel computation. All students will complete an extensive programming project which will involve parallelizing an application of interest.

Prerequisites

(CS557 Data Structures and Algorithms I), (CS680 Unix Programming Environment) or equivalent courses.

Instructor

Jeremy Johnson

Office: 206C Korman Center
Phone: (215) 895-2893
E-mail: jjohnson@mcs.drexel.edu
Office hours:

Meeting Time

W 6:00-9:00 in Korman 241

Textbook

1. Peter Pacheco, Parallel Programming with MPI, Morgan Kaufman Publishers, 1996.
2. In addition to the main text, the course will rely on references and lecture notes provided by the instructor and available from the course web page. The first three references books listed in the coures resource section of this web page would, when combined, would be an appropriate text. Rather than require all three texts, students should purchase the main text and can refer to the other reference texts.   The first of these books is out of print (selected material from this book will be distributed), and the second and third are available for purchase from the links provided or through other online sources.

---

Grading

1. Programming Assignments (three) 30% (3 at 10%)
2. Midterm Exam 30%
3. Project 40%

Final grades will be determined by your total points weighted according to this distribution. Grades will be curved based on relative student performance. Students who successfully complete all of the homework and do reasonably well on the exam and project should receive a B. Students with high exam and project scores and who do well on the assignments will receive an A.

All assignments must be completed alone unless otherwise stated. No Late assignments will be accepted without prior approval.

---

Resources

Reference Books

1. Nicholas Carriero and David Gelernter, How to Write Parallel Programs: A First Course, MIT Press, 1990.
2. Rohit Chandra, Ramesh Menon, Leo Dagum, David Kohr, Dror Maydan, Jeff McDonald, Parallel Programming in OpenMP, Morgan Kaufman Publishers, 2000.
3. Peter Pacheco, Parallel Programming with MPI, Morgan Kaufman Publishers, 1996.
4. Marc Snir, Steve Otto, Steven Huss-Lederman, David Walker, and Jack Dongarra, MPI: The Complete Reference (Vol. 1 - The MPI Core), 2nd Ed., 1998. The first edition is available online at MPI: The Complete Reference.
5. Marc Snir, Steve Otto, Steven Huss-Lederman, David Walker, and Jack Dongarra, MPI: The Complete Reference (Vol. 2 - The MPI-2 Extensions), 1998.
6. F. Thomas Leighton, Introduction to Parallel Algorithms and Architectures: Arrays, Trees & Hypercubes, Morgan Kaufman Publishers, 1992.

Web Pages

• Linda
1. Linda Group
2. Scientific Computing Associates
3. Linda FAQ
4. Introduction to using Linda on MCS machines
• MPI
1. Message Passing Interface (MPI) standard
2. Message Passing Interface References
3. MPI faq
4. MPI: The Complete Reference.
5. IBM Parallel Environment for AIX - includes description for using MPI in the IBM Parallel Environment.
• OpenMP
1. OpenMP webpage
2. Introduction to using OpenMP in the IBM Parallel Environment
• Threads
• Parallel Computers
1. IBM S80 - A shared memory SMP with upto 24 processors.
2. IBM Parallel Environment for AIX
3. Introduction to using MPI in the IBM Parallel Environment
• Benchmarks
• Parallel Languages
• More to be added.

---

Look Here for Important Announcements

Announcements (Dec. 6 @ 8:00 pm)

---

Lectures

This list is tentative and may be modified at the instructor's discretion.

1. Sept. 25, 2002 (Introduction to Parallel Programming with Linda)
2. Oct. 2, 2002 (Designing and Implementing Parallel Programs with Linda)
3. Oct. 9, 2002 (Performance Analysis and Optimization of Linda Programs)
4. Oct. 16, 2002 (Quiz 1 and Shared Memory Programming with Threads)
5. Oct. 23 2002 (Shared Memory Programming with OpenMP)
6. Oct. 30, 2002 (Message Passing using MPI)
7. Nov. 6, 2002 (Grouping Data and Communicators in MPI)
8. Nov. 13, 2002 (Designing and Optimizing Parallel Programs in MPI)
(MPI Performance Analysis and Modeling)
9. Nov. 20, 2002 (Quiz 2 and Advanced Communication Patterns)
10. Nov. 27, 2002 (Thanksgiving Holiday - No Class)
11. Dec. 4, 2002 (Parallel Algorithms)
12. Dec. 11, 2001 (Project Presentations/Final Exam)

---

Programs

• Linda
1. Parallel Hello World ( hello_p.cl).
2. Distributed Lists ( list.cl and list.h).
3. Unbounded buffer using streams (stream.cl).
4. Bounded buffer using a counter (count.cl).
5. Bounded buffer using semaphores (sem.cl).
6. Result parallel primes program (primes_result.cl).
7. Agenda parallel primes program (primes_agenda.cl).
8. Specialist parallel primes program (primes_sieve.cl).
• OpenMP programs
1. hello.c
2. hellosplit.c
3. hello.f
4. badcount.c
5. goodcount.c
6. bettercount.c
7. gorycount.c
• Solaris Threads programs
1. thr.c - sample program with fork/join
2. badcount.c - shared memory program with race condition.
3. goodcount.c - shared memory program with mutex.
4. bettercount.c - improved shared memory program with mutex.
5. pcount.c - Parallel divide and conquer program to add the elements of an array.
6. rcount.c - sequential divide and conquer program to add the elements of an array.
7. icount.c - sequential iterative program to add the elements of an array. array.
• MPI programs
1. hello.c
2. helloa.c
3. hellooof.c
4. hellocs.c
5. hellocr.c
6. hellocsr.c
7. trap.c
8. ptrap.c
9. ptrapr.c
10. ptrapg.c
11. ptrapb.c
• FFT programs using Pthreads
1. cfft.c - recursive sequential FFT with temporary storage.
2. cffts.c - recursive sequential FFT without temporary storage.
3. complex.c and complex.h - complex arithmetic needed for cfft.c.
4. wtime.c - timing function.
5. timecfft.c - program to time cfft.
6. pcfft.c - parallel recursive FFT based on cfft.
7. pcffts.c - parallel recursive FFT based on cffts.
8. timepcfft.c - program to time pcfft.
• FFT programs using OpenMP
1. cfft.f - iterative FFT.
2. timecfft.f - program to time cfft and pcfft.
3. pfft.f - iterative FFT (Pease algorithm).
4. pcfft.f - parallel iterative FFT.
5. ppfft.f - parallel iterative FFT (Pease algorithm).
6. timepfft.f - program to time pfft and ppfft.
• FFT programs using MPI
1. To be added.

---

Assignments

• Practice Assignment 1 (NOT to be handed in)
• Assignment 1 (due in the beginning of class on Oct. 16)
• Assignment 2 (due in the beginning of class on Nov. 6)
• Assignment 3 (due in the beginning of class on Nov. 18)
• Final Project (due in class on Dec. 14)

Exam Studyguide

• Studyguide for quiz 1.
• Studyguidefor quiz 2.

Solutions

• To be added when appropriate

---

Created: 9/24/02 by jjohnson@mcs.drexel.edu