Lifelong Learning Machines: The Quest for Sustainable Adaptation in a World of Unexpected Change

Beschreibung

Deep machine learning continues to advance the frontiers of innovative applications, recently through the rise of generative AI and foundation models. However, as much as their success is astonishing, their highly opaque nature continues to entail an increasing amount of scientific and socio-economic concerns. Many of these considerations seem to derive from a predominantly static workflow and are related to a respective culture of benchmarking. In fact, the machine learning textbooks teach us that an appropriate recipe is to acquire training data, select a model, and then tune its parameters until a desired performance criterion is reached on a held-out test set. The real world is significantly more ruthless than these crafted benchmarks: data distributions tend to drift, unexpected changes frequently arise, and evaluation is seldom as straightforward as pre-defined test sets make it out to be. Put into practice, the textbook machine learning approach thus entails unreasonable amounts of re-training, the produced model outputs tend to be erratic and unreliable, and the data-hungry and compute-heavy development cycle shifts power to the privileged.

Inspired by the human’s remarkable ability to adapt efficiently to a variety of contexts without amassing tremendous amounts of resources, lifelong learning promises to surmount many of the aforementioned predicaments. In this talk, I will introduce the elements necessary to shift the prevalent static design towards lifelong machine learning systems. These systems transcend stationary datasets and continually learn in a world full of unknowns, highlighting pivotal aspects to render AI systems more adaptive, robust and sustainable. Focusing on our group’s own key contributions, I will present remedies to enable deep neural networks to successfully handle new situations, to overcome the need to store large datasets, and to efficiently incorporate new knowledge over time. Importantly, I will highlight our proposed mechanisms to achieve these aspects inherently and synergistically, counteracting the present machine learning trend to alleviate discovered caveats through post-hoc modifications. Finally, I will provide some examples of our pursued applications in computer vision to highlight why lifelong learning is key to sustainable adaptation in a world of unexpected changes.

Referent: Prof. Martin Mundt  *** ANTRITTSVORLESUNG ***

Martin Mundt is the newly appointed Professor for Lifelong Machine Learning at the University of Bremen. He previously led an independent research group at the Hessian Center for Artificial Intelligence (hessian.AI) and the Technical Unviersity of Darmstadt, where he was also an interim professor (Vertretungsprofessor). He holds a PhD degree in Computer Science from Goethe University Frankfurt, and an M.Sc. in Physics. He is also a board member of directors at the non-profit organization ContinualAI and part of the core-organizer team at Queer in AI. In his research, the central goal is to develop AI systems that continue learning throughout their lifecycles. This capability for lifelong machine learning – a key ability of the human brain – is pivotal to render AI systems more adaptive, robust and inclusive. As such, Martin believes that lifelong learning is an essential missing element to make AI both technologically and socially more sustainable. These goals are reflected in Martin’s commitment to the research community, for instance, as diversity & inclusion chair of AAAI-24 and CoLLAs-25, review-process chair of CoLLAs-24, general chair of the CLAI-23 Unconference, and associate editor-in-chief at Pattern Recognition. His respective research has received best paper distinctions at ACM FAccT 2023, outstanding student paper at AISTATS 2024, alongside awards for his PhD thesis and lectures.