Highlights

Celebrating Our Achievements in 2024: A Year of Innovation, Collaboration, and Growth

As 2024 comes to an end, we’re taking a moment to reflect on the remarkable journey our research team has undertaken over the past twelve months. The excitement, curiosity, and commitment to pushing boundaries have defined our work this year. Here’s a look at some of the highlights that made 2024 extraordinary.

With language generation models being more powerful than ever, we successfully started exploring numerous cross-disciplinary areas:

LLMs and Human Behavior:

With LLMs producing seemingly natural conversations, we wonder how exactly their skills differ from their human counterparts, and where they can complement and learn from each other. How do LLMs impact collaboration? How well can they reason about their choices? And what does it mean for LLMs to exhibit social skills or to master the creative art of storytelling? Our multi-institutional paper on collective intelligence was published in Nature Human Behavior and was covered by numerous media such as Forbes. Our ACL Workshop on Human-Centered LLMs had over 200 in person attendees in Bangkok. At ACL we have shown how LLMs can improve in taking the perspective of others by learning to generate responses to conflict situations. In our awarded poster at EMNLP WiNLP in Miami, we explained that to-date LLMs are by far not yet robust in reasoning about the behavior of others. At NeurIPS Workshop on System 2 Reasoning we have demonstrated how telling better stories in explanations can improve LLM accuracy in scientific question answering. Narratives also play a key role in our Bonn-Melbourne Research Excellence Grant on understanding indoctrination mechanisms at scale.

LLMs for Mental Health:

What is the potential of LLMs for mental health screening? And for treatment? With our psychology colleagues from Stony Brook University, Stanford and Lund, we found that GPT-4’s assessment and explanation of depression severity had high overall convergent validity (r = 0.81 with experts) and internal model consistency that largely aligned with literature and item-level self-assessment via well-established questionnaires. Such pre-screening may allow clinicians to invest more time into carefully tailored treatment plans. In the InVirtuo collaboration, we explore the development of empathetic, socially competent personalized LLMs to empower virtual reality avatars in behavioral therapy.

LLMs and Physics:

LLMs are merely powerful pretrained models for finding complex patterns in long time series. What if we apply these to particle physics and astrophysics? Whether searching for high-frequency gravitational waves (TRA Matter / GravNet), analyzing AGN light curves (Lamarr Physics), investigating the adversarial robustness of particle physics models (ErUM-Data AI Safety), or searching for new galaxy clusters with AstroLlama and multimodal representation learning (DynaVerse), we are excited what our new projects may reveal!

LLMs and Biomedicine:

Can LLMs accelerate research by harmonizing scientific literature? With our new seed funding from TRA Modelling, we use LLMs to detect patterns in gene regulation dysfunctions associated with cancer, reconstructing gene regulatory networks from published experimental findings.

Within Lamarr NLP, we have established a strong collaboration on LLM research with the Fraunhofer IAIS teams on Foundation Models and Trustworthy AI (joint paper on multilingual alignment), as well as with TU Dortmund on Sustainable AI (joint paper on LLM pruning). With the newly released Teuken-7B model, we are excited about more joint open-source multilingual LLM research to come.

With 2024 being the USA-NRW year, we celebrated our partnerships by organizing a Lamarr AI Event at the German Consulate General in NYC, marking a start of numerous PhD visits. Apart from our traditionally strong collaborations with Stony Brook, NYU, CMU, UPenn and UMich, we are happy about our reinforced ties to Stanford University, where Prof. Flek recently presented our perspectivism work. In 2024 we organized a joint LLM workshop (Diyi Yang), authored a joint mental health paper (Johannes Eichstadt, Betsy Stade), founded the IMPULSE House for Intellectual Innovation and Creativity in Bonn (Sepp Gumbrecht), and Oussama Khatib from the b-it advisory board has welcomed our NRW delegation in his Stanford robotics lab.

In numbers, our team has hired 7 new FTEs (including a new junior research group on LLM agents), published 17 peer-reviewed papers, organized 4 LLM workshops, gave many talks across three continents, acquired four new grants, joined 10 PhD committees in 4 countries, and hosted 4 visiting research fellows and 10 invited speakers from all over the world (UPenn, EPFL, RMIT, Monash, UMich, NYU, SBU, Sheffield, Edinburgh, UPV). One team member accepted a professorship offer and two completed their PhD, all continuing their work in the field (Canada, US, Germany). Also with junior researchers we are off to a great start. We supervised over 20 master theses, many of which were published in international workshops; two students received the Best Poster Award in Miami, one work was awarded the BDD Research Award. Last but not least, the first two caisarians are now mothers of beautiful babies.

While 2024 was a great year, it’s only the beginning of our journey. As we enter 2025, we carry forward the momentum generated by our research projects and collaborations.

To our dedicated researchers, collaborators, and supporters: thank you for your unwavering commitment to pushing the boundaries of human knowledge. Here’s to another year of curiosity, innovation, and transformative research!

Previous post
Florian Mai joins CAISA group