Welcome to the website of the 2nd AI summer school of the Austrian Academy of Sciences, held on September 13-17, 2021 in Vienna. Below, you can find more information about the event. This website will be updated with new information when it is available. If you registered for the summer school, please check this website regularly for updates, in particular in the weeks leading up to the event.
- Website created [June 28, 2021]
The ÖAW AI summer school 2021 will be a 5-day event, offering
- An introduction and refresher on differentiable programming in PyTorch on the first day.
- Three 1-day courses on different aspects of deep learning, differentiable programming and interpretable machine learning. Each course will be split into lecture and practical (programming) sessions, see below for more information on the courses.
- An outlook towards advanced and more specialized methods in deep learning on the last day. (Voluntary programming exercises on the topics covered on this day will be provided.)
To follow the courses and to complete the practical sessions, participants should be familiar with the fundamental concepts of deep learning and programming in Python. For prospective participants that have no experience with either or both topics, introductory material will be linked from this website at a later time and/or provided directly to the participants some weeks before the event.
The summer school primarily serves the education of young scientists from all fields and hence mainly targets PhD students and young Postdocs that have little to no hands-on experience with deep learning, but any student and researcher affiliated with the Austrian Academy of Sciences may register.
Registrations are closed.
If you have registered for the summer school, you should meanwhile have received an e-mail informing you about the status of your registration.
If you did not receive this information, or if any other questions regarding your registration arise, please contact [nicki.holighaus(at)oeaw.ac.at].
No participation fee:
We are happy to announce that due to generous support by the Austrian Academy of Sciences, the Institute of High Energy Physics [https://www.oeaw.ac.at/en/hephy] and the Acoustics Research Institute [https://www.oeaw.ac.at/en/ari], we can offer the summer school free of charge.
During coffee breaks (twice per day), catering will be available. Participants are responsible for covering their own travel, accommodation and regular meals, which are not offered at the summer school venue. Participants are encouraged to inquire at their institute or research project for funding these positions.
Covid-19 pandemic related information:
The summer school will be held as an in-person event. To maximize the safety of all our participants, lecturers and staff, the ÖAW AI summer school employs a set of Covid-19 prevention rules that are slightly stricter than what we are legally required to. The details can be found below:
- Participants, lecturers and staff will be required to provide a document proving that they are fully vaccinated against Covid-19 or negatively PCR-tested within the last 48 hours. Non-vaccinated participants and staff are required to provide a valid negative test result each day of the summer school. Additionally, when joining the summer school for the first time, i.e., in most cases on Monday morning, everyone will be required to provide a valid negative PCR test result, including those that are fully vaccinated. The registration staff will scan the QR code on your documents to verify them. If there is a plausible reason why you cannot bring a negative PCR test result on Monday morning, please contact us until Friday, 10.09.2021. We will evaluate your request and try to find an adequate solution.
- To ease the organizational burden of providing PCR test results beyond Monday, we will provide free ‘Alles gurgelt’-PCR-Testskits and instructions on site in sufficient quantity. We encourage everyone to test themselves responsibly and more often than required.
- Indoors, everyone will be required to wear an FFP2-compliant face mask, except during consumption of food and drink, in particular during the coffee breaks. During the lectures, lecturers and tutors can opt to not wear a mask at their own discretion.
- We will to provide a sufficient supply of FFP2-compliant face masks on site and encourage everyone to exchange their mask for a fresh one regularly.
- Please follow the usual hygiene guidelines, e.g., wash your hands regularly and properly, keep a distance of at least 1m whenever possible, etc. Additionally, liquid hand-disinfectant and surface-disinfectant in the form of wipes will be available on site.
- The first time that you join the event each day, you will be asked to fill in a short contact form and roughly specify your time of arrival to enable contact tracing if necessary.
- If you feel sick or unwell, we ask you to stay at home, call 1450, get tested and inform us in case of a positive test result. Contact information for that purpose will be provided when you first join us.
Place: Sonnenfelsgasse 19, 1010 Wien
Contact: Nicki Holighaus (nicki.holighaus(at)oeaw.ac.at)
Programme and Schedule:
Welcome (10m) + Course 1 – I
Course 2 – I
Course 3 – I
Course 4 – I
Outlook – I
Course 1 – II (Wagner/Haider)
Course 2 – II
Course 3 – II
Course 4 – II (Grosse-Wentrup)
Outlook – II
(Perraudin) + Closing (10m)
Course 1 – III (Wagner/Haider)
Course 2 – III (Harár)
Course 3 – III
Course 4 – III
Afternoon sessions include an ‘open’ coffee break, available from 15.00. The structure of the afternoon session may differ at the discretion of the responsible lecturer.
Course 1: Introduction to deep learning and differentiable programming in Python - This first session of the summer school provides a general introduction to machine learning with neural networks: A historical retrospective, a sketch of modern applications and the basics of their architecture and training procedure. The Python skills that are necessary to set up a simple machine learning project are reviewed. In the exercise session, participants are familiarized with the workflow and will design their first model for classification in PyTorch. Starting from an introductory level, this course levels the fundaments of all participants.
Course 2: Planning, execution and dissemination of a machine learning project - During the life cycle of our machine learning research projects, we interact with source code, data, and models almost daily. Therefore, it is wise to make use of various tools and tricks of the community that save a lot of time during the development (versioning), can be decisive during the peer review (reproducibility), and are invaluable for making an impact (presentation). Moreover, the community is constantly coming up with tools useful even before the beginning (to find ideas) or after a seeming end of the project (publicity). Together, in this hands-on session, we will use those tools while developing a specific ML project. After the session, you should be able to use the tools in projects of your own.
Course 3: Optimization in Deep Neural Networks: Basics, Methods and Challenges - Training a deep neural network corresponds to solve a minimization optimization problem. In this lecture, we will review the building blocks used to construct algorithms for this task (such as stochastic gradient descent, momentum and adaptive learning rate). Doing so, we can understand the challenges of deep network optimization. Finally, this course will also include specific techniques to overcome these challenges (such as batch norm, regularization, dropout, early stopping, data augmentation, architecture invariance and parameter sharing). This lesson will equip students with the intuition and tools to face the most common optimization challenges and design choices in Deep Learning.
Course 4: A Causal Perspective on Interpretable Machine Learning and explainable AI - Machine learning and AI models are increasingly deployed in high-stakes environments, e.g., in medical decision support or loan approval, where ethical and legal considerations require models to be interpretable. Prof. Grosse-Wentrup will review current methods for interpretable machine learning and explainable AI, discuss challenges in designing such methods, and then argue that we can resolve some of these issues by adopting a causal perspective on the problem. In particular, he outlines how to reason about causal relations in general and then show how a causal perspective enables us to distinguish between interpreting the model and using the model to interpret the data-generating process.
Univ.-Prof. Dipl.-Ing. Dr. -Ing. Moritz Grosse-Wentrup is head of the research group Neuroinformatics at the Faculty of Computer Science of the University of Vienna and board member of the university’s research network Data Science @ Uni Vienna. [https://ni.cs.univie.ac.at/team/person/107192/]
Ing. Pavol Harár, Ph.D. is a machine learning researcher at the research network Data Science @ Uni Vienna at the University of Vienna and Senior Researcher at the Brain Diseases Analysis Laboratory, a international, multidisciplinary research group with a focus on brain diseases. [https://pavol.harar.eu/]
Dr. Nathanaël Perraudin is a Senior Data Scientist at the Swiss Data Science Center in Zurich, a Cooperation of ETH Zurich and the École polytechnique fédérale de Lausanne. [https://perraudin.info/]
Dipl.-Ing. Felix Wagner is a PhD Student at the Institute of High Energy Physics (HEPHY) of the Austrian Academy of Sciences (OEAW) and member of the CRESST and COSINUS dark matter search collaborations.
Daniel Haider MSc is a PhD student in Mathematics at the Acoustics Research Institute (ARI) of the Austrian Academy of Sciences (OEAW) and member of the groups “Mathematics and Signal Processing in Acoustics” and “Machine Learning in Acoustics”.
Preparation for Participants
The summer school participants will be expected to possess a very basic understanding of the Python programming language and the numpy package for scientific computing in Python. Participants will not be required to set up a development environment (such as PyCharm), nor need they be familiar with such an environment. If the following statements apply to you, you should be fine:
- I understand how basic data types, variables and loops are defined and used in Python (e.g., strings. numbers, Dictionaries, for-loops and lists).
- I know about Classes and Objects and how they are used in Python.
- I know about numpy and its purpose.
- I know how a matrix array is created using numpy, what it looks like, and how I can apply some basic manipulations (e.g., reshaping) and mathematical operations (e.g., addition, multiplication) to them.
Furthermore, participants should be familiar with the fundamental ideas behind machine learning and deep learning and familiar with the concept of a (deep) neural network. If you can answer the following questions with ‘yes’, you should be fine:
- Do I know what machine learning is and could I explain its general idea to a colleague?
- Do I know what deep learning is and could I explain its general idea to a colleague?
- Do I understand what a neural network is and can I describe it, as well as explain the difference between a shallow and a deep neural network, to a colleague?
More familiarity with Python and/or machine learning might make some aspects of the summer school easier to follow, but should not be strictly necessary. You can find a wealth of resources about all these topics on the internet and some below. If you are unsure about your own prior knowledge, you will have a significantly better experience at the summer school if you invest some time (1-2 working days) into preparing yourself.
We provide a few exemplary links for preparatory material. These are separated into 3 categories: Mandatory prior knowledge, helpful prior knowledge and optional in-depth or advanced material. Obviously, you are invited to do your own research instead, if you prefer.
The list below may be updated at a later time. Last update 13.08.2021
Mandatory prior knowledge:
- https://python.land (Chapters Introduction & Classes and Objects)
- https://numpy.org/doc/stable/user/absolute_beginners.html#welcome-to-numpy (Absolute basics for beginners)
- https://www.youtube.com/watch?v=nKW8Ndu7Mjw (Basic understanding of machine learning)
- https://www.youtube.com/watch?v=aircAruvnKk (What is a Neural Network?)
Helpful prior knowledge (read/watch as much or as little as you like):
- https://pytorch.org/tutorials/ (PyTorch Tutorial)
- https://scikit-learn.org/stable/tutorial/index.html (Scikit-Learn Tutorial)
- https://pandas.pydata.org/pandas-docs/stable/user_guide/dsintro.html (Pandas Introduction – for data structures and data analysis tools)
- https://www.datacamp.com/community/tutorials/seaborn-python-tutorial (Seaborn Tutorial – for Statistical Data Visualization)
In-depth and advanced material:
Deep Learning (extensive)