The Parallel Computing lecture covers software and hardware-related topics of parallel systems, algorithms, and application design. In addition to learning theoretical foundations, students will get hands-on experience with selected parallel computing topics.

Key information

ContactRamin Yahyapour
Presence time56h
Independent study124h

Learning Objectives

Successfully completing the lecture, students are able to

  • define and describe the benefit of parallel computing and identify the role of software and hardware in parallel computing
  • specify the Flynn classification of parallel computers (SISD, SIMD, MIMD)
  • analytically evaluate the performance of parallel computing approaches (Scaling/Performance models)
  • understand the different architecture of parallel hardware and performance improvement approaches (e.g., caching and cache coherence issues, pipeline, etc.)
  • know and define the interconnects and networks and their role in parallel computing
  • architecture of parallel computing (MPP, Vector, Shared memory, GPU, Many-Core, Clusters, Grid, Cloud)
  • design and develop parallel software using a systematic approach
  • generate parallel computing algorithms and development environments (i.e. shared memory and distributed memory parallel programming)
  • write parallel algorithms/programs using different paradigms and environments (e.g., POSIX Multi-threaded programming, OpenMP, MPI, OpenCL/CUDA, MapReduce, etc.)
  • understand and develop sample parallel programs using different paradigms and development environments (e.g., shared memory and distributed models) expose to some applications of Parallel Computing



  • Computer architecture
  • Basic knowledge of computer networks and topologies
  • Data structures and algorithms
  • Programming in C(/C++)


Type: Written exam (90 minutes) or oral examination, 20 minutes, with grade

Expectation on the examinee

Profund knowledge of:

  • Parallel programming
  • Shared Memory Parallelism
  • Distributed Memory Parallelism
  • Single Instruction Multiple Data (SIMD)
  • Multiple Instruction Multiple Data (MIMD)
  • Hypercube
  • Parallel interconnects and networks
  • Pipelining
  • Cache Coherence
  • Parallel Architectures
  • Parallel Algorithms
  • OpenMP
  • MPI
  • Multi-Threading (pthreads)
  • Heterogeneous Parallelism (GPGPU, OpenCL/CUDA).
  • An Introduction to Parallel Programming, Peter S. Pacheco, Morgan Kaufmann(MK), 2011, ISBN: 978-0-12-374260-5.
  • Designing and Building Parallel Programs, Ian Foster, Addison-Waesley, 1995, ISBN 0-201-57594-9 (Available online).
  • Advanced Computer Architecture: Parallelism, Scalability, Programmability, Kai Hwang, Int. Edition, McGraw Hill, 1993, ISBN: 0-07-113342-9.
  • In addition to the mentioned text book, tutorial and survey papers will be distributed in some lectures as extra reading material.


Portrait von Ramin Yahyapour Prof. Dr.Ramin Yahyapour


Portrait von Sven Bingert Dr.Sven Bingert