Efficient and distributed training with TensorFlow on Piz Daint

Vacancies available
CHF 160.00

The Swiss National Supercomputing Centre is pleased to announce that the "Efficient and distributed training with TensorFlow on Piz Daint" workshop will be held from March 14-15, 2019 at CSCS in Lugano, Switzerland.


The Piz Daint supercomputer at CSCS provides an ideal platform for supporting intensive deep learning workloads as it comprises thousands of Tesla GPU compute nodes communicating  through a high-speed interconnect. In this two-day course, we will look at how to run distributed deep learning workloads with TensorFlow on Piz Daint. We will use simple examples to demonstrate best practices for building efficient input pipelines to maximize the throughput of deep learning models with TensorFlow. TensorFlow is one of the most popular numerical libraries for deep learning and contains an extensive collection of algorithms optimized to exploit hardware as efficiently as possible.

The course will include the following topics:
- Running TensorFlow on Piz Daint.
- Creating efficient input pipelines with TensorFlow's Dataset API for optimizing the throughput on Piz Daint.
- Reading and writing data as TFRecords files.
- Understanding the stochastic gradient descent and distributed synchronous stochastic gradient descent algorithms.
- Performing distributed training with TensorFlow and the ring allreduce algorithm implemented in Horovod (Keras and Tensorflow's Estimator API).
- Understanding Horovod and TensorFlow's operations timeline.

Target Audience

This course is addressed to scientists who are planning or are already engaged in intensive machine learning workloads and wish to start using TensorFlow on Piz Daint.

Tentative Agenda

Day 1

10:00-12:00 Lecture
12:00-13:00 Lunch
13:00-15:00 Lecture
15:00-15:20 Coffee Break
15:20-16:00 Lecture

Day 2

09:00-10:10 Lecture
10:10-10:30 Coffee Break
10:30-11:30 Lecture
11:30-12:00 Machine room visit
12:00-13:00 Lunch
13:00-15:00 Lecture
15:00-15:20 Coffee Break
15:20-16:00 Lecture


All participants must register for the course. The registration fee includes coffee breaks and lunches throughout the two day course.

Course Fee: 160 CHF

Deadline for registration: Tuesday, March 5, 2019

Kindly note that the course can take place only if there are sufficient confirmed registrations received by the deadline. The minimum number of participants is 8.

Please contact Rafael Sarmiento (rafael.sarmiento@cscs.ch) and Guilherme Peretti-Pezzi  (peretti@cscs.ch) for questions related to the course content and info@cscs.ch for questions related to the event logistics. 

Kindly note that no parking space is available at the Swiss National Supercomputing Centre. The closest bus stop to the centre is Lugano, Stadio. From Lugano railway station, you should take bus number 4 or 6.

Suggestions regarding travels and accommodation are available here.

You are encouraged to travel by public transportation or to use the Park & Ride Resega parking lot, within five minutes walk from CSCS.

We look forward to welcoming you at CSCS!