Unfortunately, this event was cancelled on short notice. If you planned to attend the seminar, we apologize for any inconvenience caused by the cancellation. We are in active discussion with the speaker to find an alternative date in the next months. Original information below.

For the next MLA2S special seminar, we are happy to announce a talk by Günther Klambauer (LIT AI Lab & Institute for Machine Learning, Johannes Kepler University). Prof. Klambauer is among  the key players in Machine Learning research in Austria, with central contributions to novel methods in deep learning, their theoretical foundation, and various applications. In this talk, he will present one of the latest success stories in the long history of the Institute for Machine Learning at Johannes Kepler University, the family of Extended Long Short-Term Memory (xLSTM) models. 

In the established style of MLA2S events, the talk is followed by an open get-together session, where participants will have the chance to engage in discussion with the speaker and members of his team, and to explore potential connections and collaboration opportunities.  

You can also find the event at https://indico.cern.ch/event/1482937/. To plan for the catering during the get-together, we kindly ask participants to to fill in the non-obligatory registration here.

Title: xLSTM and the age of recurrent neural networks

Speaker: Günther Klambauer   

Abstract. Since their inception in the early 1990, long short-term memory recurrent neural networks (LSTMs) revolutionized AI with their ability to manage long-term dependencies, playing a key role in early language models. However, neural networks based on Transformers later outperformed LSTMs by leveraging parallel processing and self-attention. This work revisits LSTMs and a potential new age of recurrent neural networks, asking: how well can they scale with billions of parameters and modern techniques? We introduce new methods to enhance LSTMs, including exponential gating and memory structure updates. These lead to sLSTM (simplified memory) and mLSTM (parallelizable memory). Combined into xLSTM architectures, these innovations make LSTMs competitive with state-of-the-art Transformers in both performance and scalability. We demonstrate applications of xLSTM beyond natural language processing, such as robotics, molecular biology, genetics, and chemistry.

Program:
15:00-16:20 (Talk and Discussion)
16:20-17:30 (Get-together with snacks)