Poziv na predavanje: "From Facts to Conversations: Exploring Reliability and Interaction in LLMs"

Saad Obaid ul Islam (Center for AI and Data Science, Julius-Maximilians-Universität Würzburg)

Saad is a second-year PhD student at the Center for AI and Data Science at the University of
Würzburg, where he is affiliated with the Equitably Fair and Trustworthy Language Technology
(EQUIFAIR) project. Before starting his PhD, he worked as a research engineer on the
NORFACE-funded EUINACTION project. Prior to this, he studied computer science and
computational linguistics as an Erasmus Mundus scholar at Charles University and Saarland
University.

Dimensions of Factual Reliability in LLMs: Multilinguality and the Short–Long Form Gap

As Large Language Models (LLMs) become globally deployed for information-seeking tasks,
ensuring their factual reliability across diverse contexts has become paramount. Yet most
research on LLM hallucinations remains narrowly focused—either English-centric or limited to
controlled tasks like summarization and translation. This talk presents a comprehensive
investigation into factual alignment across two critical dimensions: languages and task formats.
First, we will talk about a large-scale study evaluating hallucination across 30 languages in
open-domain question answering, revealing surprising patterns in how factual accuracy varies
across linguistic and scaling contexts. Second, I will explore the factual alignment gap between
short and long-form responses, demonstrating how the same model can exhibit vastly different
reliability depending on response format. Together, these works provide a unified perspective on
LLM factuality "in the wild," offering diagnostic tools and insights for practitioners deploying
LLMs internationally and researchers working to build more trustworthy AI systems.

Nicolò Penzo (University of Trento and Fondazione Bruno Kessler, Italy)

Nicolò is a third-year PhD Student in Information Engineering and Computer Science at the University of Trento, holding a scholarship funded by the Digital Humanities group at Fondazione Bruno Kessler. He is also part of the ELLIS PhD programme and he recently spent six months at the Center for Artificial Intelligence and Data Science, University of Würzburg. He holds a Master’s degree in Computer Engineering from University of Padua. His PhD project deals with the design, analysis and evaluation of models to perform downstream tasks on Multi-Party Conversations (MPCs). The thesis's main focus is the generation of synthetic MPCs, by combining NLP solutions (i.e., transformer-based models) with techniques from network analysis. The goal is to diagnose the models' behavior in different interactional scenarios and favour privacy-preserving solutions. Other research interests involve detection and contrast of toxic language on social media, and interactions among LLMs.

Interacting in Natural Language in the era of LLMs: NLP for Online Discussions and Multi-Party Conversations

With the rise of online social media platforms, written conversations among multiple participants have become an essential part of daily communication, enabling users to exchange ideas, share opinions, and engage in discussions. Beyond the textual content of individual messages, the dynamics of participant interactions influence the meaning and the pragmatic aspects of the conversations. In this talk, we will explore the challenges faced by models in processing Multi-Party Conversations (MPCs), the key downstream tasks in this domain, and the importance of incorporating interaction dynamics. Specifically, we will discuss the impact of training data size, highlight critical evaluation challenges, and propose a potential solution to address these issues based on the generation of synthetic MPCs. We will mainly focus on the latter, exploring different models and strategies for generating MPCs under precise constraints. We also introduce an evaluation framework to assess both constraint compliance and generation quality, in order to identify which model-strategy combinations are more effective.