Parallel and Distributed Data Processing Technologies

Entry requirements: Basic C/C++ programming skills

Credits: 5

Course: Core

Language of the course: English


Katerina Bolgova


Learning outcomes:

  • To get familiar with the main development trends of high performance computers and key classification of multiprocessor systems
  • To be able to use basic methods and models of parallel computing
  • To learn to use parallel programming tools for distributed and shared memory systems
  • To learn to design parallel algorithms for computational problems
  • To implement parallel programs for distributed and shared memory systems
  • To be able to analyze acceleration and efficiency of parallel algorithms


  • Introduction to the parallel computing theory: key definitions and tasks of parallel programming, classification of multiprocessor systems. Formal models of computation for parallel programs.
  • Modern means and technologies of parallel programming: Languages of parallel programming, libraries of parallel programming. Principles of developing parallel algorithms and programs.
  • Methods of parallelizing sequential programs: Parallelism in computational mathematics. Data dependency. Decomposition method. Methods of parallelizing sequential algorithms. Estimation of efficiency and communication input of parallel algorithms.
  • Development technologies of parallel programs for shared memory systems: Concurrent programming in C#. OpenMP parallel programming technology for shared memory systems. parallel algorithms of numerical integration of functions.
  • Development technologies of parallel programs for distributed memory systems: MPI technology. Key concepts and definitions. Parallelizing principles. Message types and modes. MPI technology. Diverse data forwarding. Managing groups and communicators. Virtual topologies, parallel algorithms of data marshalling.
  • Development technologies of parallel programs for graphics processing units: CUDA technology. CUDA programming model. CUDA host API basics. CUDA memory types and memory hierarchy. Parallel numerical algorithms of linear algebra.


Practical sessions


Attendance is mandatory. Students should complete all the assignments and present a report. The final grade is based on the student performance throughout the course.