Results & Publications

Deliverables

Publications

Click on a publication to show its abstract.

2026

Münker, S., Schwager, N., Rettinger, A. (2026): Don't Trust Generative Agents to Mimic Communication on Social Networks Unless You Benchmarked their Empirical Realism.
Abstract. The ability of Large Language Models (LLMs) to mimic human behavior triggered a plethora of computational social science research, assuming that empirical studies of humans can be conducted with AI agents instead. Since there have been conflicting research findings on whether and when this hypothesis holds, there is a need to better understand the differences in their experimental designs. We focus on replicating the behavior of social network users with the use of LLMs for the analysis of communication on social networks. First, we provide a formal framework for the simulation of social networks, before focusing on the sub-task of imitating user communication. We empirically test different approaches to imitate user behavior on in English and German. Our findings suggest that social simulations should be validated by their empirical realism measured in the setting in which the simulation components were fitted. With this paper, we argue for more rigor when applying generative-agent-based modeling for social simulation.
Read the paper
Münker, S., Sartori, F. (2026): Guardrail Vulnerabilities in Open-Source Language Models: Implications for Democratic Discourse and Marginalized Communities.
Abstract. The proliferation of open-source Large Language Models (LLMs) presents a complex technological phenomenon with significant societal implications. While these models democratize access to advanced Natural Language Processing (NLP) capabilities, they simultaneously amplify risks for marginalized communities who often bear the disproportionate burden of technological misuse. Our research examines systematic vulnerabilities in guardrail mechanisms across seven prominent open-source LLMs, revealing patterns of harmful content generation that threaten democratic discourse and social cohesion. Through empirical analysis using advanced NLP classification methods, we demonstrate that popular open-source models consistently generate content classified as hateful or offensive when subjected to adversarial prompting techniques. These findings directly contradict the safety assurances provided by model developers, particularly Meta AI’s stated commitment that their systems should present balanced perspectives on debated policy issues rather than singular viewpoints.
Read the paper
Schwager, N., Münker, S., Plum, A., Rettinger, A. (2026): Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction.
Abstract. The transition of Large Language Models (LLMs) from exploratory tools to active “silicon subjects” in social science lacks extensive validation of operational validity. This study introduces Conditioned Comment Prediction (CCP), a task in which a model predicts how a user would comment on a given stimulus by comparing generated outputs with authentic digital traces. This framework enables a rigorous evaluation of current LLM capabilities with respect to the simulation of social media user behavior. We evaluated open-weight 8B models (Llama-3.1, Qwen3, Ministral) in English, German, and Luxembourgish language scenarios. By systematically comparing prompting strategies (explicit vs. implicit) and the impact of Supervised Fine-Tuning (SFT), we identify a critical form vs. content decoupling in low-resource settings: while SFT aligns the surface structure of the text output (length and syntax), it degrades semantic grounding. Furthermore, we demonstrate that explicit conditioning (generated biographies) becomes redundant under fine-tuning, as models successfully perform latent inference directly from behavioral histories. Our findings challenge current “naive prompting” paradigms and offer operational guidelines prioritizing authentic behavioral traces over descriptive personas for high-fidelity simulation.
Read the paper
Sittar, A., Češnovar, M., Guček, A., Grobelnik, M. (2026): Constructing a Dataset to Support Agent-Based Modeling of Online Interactions: Users, Topics, and Interaction Networks.
Abstract. Agent-based modelling provides a powerful framework for exploring how individual behaviours and interactions produce collective social dynamics. This paper introduces a large-scale, empirically grounded Reddit dataset designed to support the development and evaluation of agent-based social simulations. The dataset includes technology-focused, climate-focused, and COVID-related agent groups, covering around one million posts and comments. By modelling agent categories, interaction patterns, and directed weighted networks, the dataset enables researchers to calibrate and benchmark simulations of user behaviour, network structure, and information diffusion.
Read the paper
Sittar, A., Smiljanić, M., Guček, A., Grobelnik, M. (2026): Fake News Detection Through LLM-Driven Text Augmentation Across Media and Languages.
Abstract. The spread of fake news across social media, headlines, and news articles creates major challenges for automated detection, particularly in multilingual and cross-media settings affected by data imbalance. This paper proposes a fake news detection framework based on LLM-driven, feature-guided text augmentation. The method generates realistic synthetic samples across languages, media types, and text granularities while preserving meaning and stylistic coherence. Experiments using classical and transformer-based models show consistent performance improvements across social media, headline, and multilingual news datasets.
Read the paper
Sittar, A., Guček, A., Grobelnik, M., Rettinger, A., Smiljanić, M. (2026): Simulating Multi-Agent Social Media Ecosystem with LLMs: Modeling, Evaluation, and Structural Effects of Ranking Mechanisms.
Abstract. We present a data-driven multi-agent framework for simulating social media activity using large language models (LLMs). The system combines persona-based agents, a temporal fusion transformer (TFT) for predicting posting times, retrieval-based context grounding, and configurable ranking mechanisms. These components allow agents to post, comment, and interact within a dynamic network. We evaluate the simulation across three domains: Technology, Climate Change, and COVID-19. We compare the generated data with real Reddit data using temporal, linguistic, and network metrics. When agents follow real-world event timing predictions, the simulation produces realistic patterns, participation distributions, and cascade structures. However, qualitative analysis shows that while structural diffusion patterns are present, semantic coherence often weakens in longer interaction chains. Two main interaction types appear: opinion–argument threads and question–answer threads. Homophily is mainly visible at the stylistic and emotional level, while strong topical alignment is less consistent, especially in large-scale simulations. We also analyze how ranking strategies affect network structure. Chronological ranking preserves diversity. Engagement-based ranking increases influence inequality. Hybrid ranking increases clustering and socially close exposure. These structural patterns are often associated with echo chambers. Overall, the framework provides a controlled environment for studying temporal activity, ranking effects, and structural interaction dynamics in simulated social networks.
Read the paper
Sittar, A., Heseltine, M., t’Serstevens, F., Viteznik, N., Oschatz, C., Smiljanić, M., Guček, A., Trilling, D., Grobelnik, M. (2026): TWON social media: a scalable MERN-Stack platform for experimental research in online social networks.
Abstract. Online social networks play a central role in shaping public discourse, yet controlled experimental research on these platforms remains difficult because of limited access, opaque ranking algorithms, and restricted intervention capabilities. This paper presents TWON, a scalable and modular social media platform designed to support reproducible experiments on user behaviour, information diffusion, and algorithmic interventions. The platform incorporates LLM capabilities for content generation, moderation, and agent-based simulation, enabling hybrid experimental designs that combine human participants with automated agents.
Read the paper

2025

Bojić et al. (2025): Does GPT-4 surpass human performance in linguistic pragmatics?
Abstract. As Large Language Models (LLMs) become increasingly integrated into everyday life as general-purpose multimodal AI systems, their capabilities to simulate human understanding are under examination. This study investigates LLMs’ ability to interpret linguistic pragmatics, which involves context and implied meanings. Using Grice’s communication principles, we evaluated both LLMs (GPT-2, GPT-3, GPT-3.5, GPT-4, and Bard) and human subjects ( N = 147) on dialogue-based tasks. Human participants included 71 primarily Serbian students and 76 native English speakers from the United States. Findings revealed that LLMs, particularly GPT-4, outperformed humans. GPT-4 achieved the highest score of 4.80, surpassing the best human score of 4.55. Other LLMs performed well: GPT-3.5 scored 4.10, Bard 3.75, and GPT-3 3.25; GPT-2 had the lowest score of 1.05. The average LLM score was 3.39, exceeding the human cohorts’ averages of 2.80 (Serbian students) and 2.34 (U.S. participants). In the ranking of all 155 subjects (including LLMs and humans), GPT-4 secured the top position, while the best human ranked second. These results highlight significant progress in LLMs’ ability to simulate understanding of linguistic pragmatics. Future studies should confirm these findings with more dialogue-based tasks and diverse participants. This research has important implications for advancing general-purpose AI models in various communication-centered tasks, including potential application in humanoid robots in the future.
Read the paper
Bojić et al. (2025): An Agent‑Based Simulation of Politicized Topics Using Large Language Models: Algorithmic Personalization and Polarization on Social Media.
Abstract. Digital platforms now act as the primary environments for public discourse, where recommender systems shape visibility, emotion, and interpretation. This study introduces the Recommender Systems LLMs Playground (RecSysLLMsP), a simulation framework designed to examine how algorithmic personalization interacts with language generation to influence engagement and polarization. The research provides a reproducible and transparent environment for testing algorithmic effects on collective reasoning, which is an issue central to democratic communication. The study employs a one‑hundred‑agent simulation grounded in psychometric and demographic data from Serbian social media users. Agents interact through five stages of progressively personalized content feeds mediated by LLM‑generated posts. Quantitative metrics such as engagement intensity, network modularity, sentiment variance and qualitative linguistic validation are used to assess behavioral and structural change. Results reveal that moderate personalization maximizes engagement, while full personalization reduces diversity and amplifies both structural and affective polarization (Q = 0.22 → 0.68). LLM‑based agents successfully reproduce realistic patterns of emotional contagion and ideological clustering. The implications extend to computational social science and policy. Simulation‑based experimentation can inform ethical recommender design and algorithmic governance. Limitations concern the absence of genuine human cognition. Thus, findings indicate systemic tendencies rather than behavioral prediction. Future research should integrate real‑world datasets, multilingual testing, and policy‑driven intervention modeling to further calibrate this digital “laboratory” for exploring AI‑mediated communication.
Read the paper
Bojić et al. (2025): The dual impact of virtual reality: examining the addictive potential and therapeutic applications of immersive media in the metaverse.
Abstract. The emergence of the metaverse – envisioned as a hyperreal virtual universe enabling boundless human interaction – has the potential to revolutionize our conception of media. This transformation could alter society as we know it. This paper identifies addictive features of social media, including immersion, interactivity, real-time access, and personalization. These features are examined within the context of virtual reality through a literature review and content analysis, aimed at exploring the potential consequences of metaverse development. From an initial pool of 193,218 documents, a refined selection of N = 44 relevant papers formed the basis of our qualitative analysis. About half of the analyzed papers indicate that these features contribute to VR addiction. Interestingly, the same features that contribute to addictive behaviors can also be harnessed for positive therapeutic interventions of VR, particularly in treating addictions and managing mental health conditions. This duality, observed in the other half of the papers, emphasizes the complex role of VR technologies, suggesting that they can serve as a substitute for other addictions. This phenomenon is placed into the historical context of evolving media technologies that increasingly mimic reality. The complex interplay of factors contributing to addiction necessitates the development of algorithmic solutions that actively curate diverse offerings, rather than promoting a closed loop of like-minded views. Traditional models of addiction should be adapted to address these unique challenges. Finally, the discussion turned to the implications of these findings for a society where the metaverse is widely accepted as a mainstream technology.
Read the paper
Münker, S. (2025): Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnaires.
Abstract. Are AI systems truly representing human values, or merely averaging across them? Our study suggests a concerning reality: Large Language Models (LLMs) fail to represent diverse cultural moral frameworks despite their linguistic capabilities. We expose significant gaps between AI-generated and human moral intuitions by applying the Moral Foundations Questionnaire across 19 cultural contexts. Comparing multiple state-of-the-art LLMs’ origins against human baseline data, we find these models systematically homogenize moral diversity. Surprisingly, increased model size doesn’t consistently improve cultural representation fidelity. Our findings challenge the growing use of LLMs as synthetic populations in social science research and highlight a fundamental limitation in current AI alignment approaches. Without data-driven alignment beyond prompting, these systems cannot capture the nuanced, culturally-specific moral intuitions. Our results call for more grounded alignment objectives and evaluation metrics to ensure AI systems represent diverse human values rather than flattening the moral landscape.
Proceedings of 0th Symposium on Moral and Legal AI Alignment of the IACAP/AISB Conference
Read the paper
Münker, S. (2025): Fingerprinting LLMs through Survey Item Factor Correlation: A Case Study on Humor Style Questionnaire.
Abstract. LLMs increasingly engage with psychological instruments, yet how they represent constructs internally remains poorly understood. We introduce a novel approach to “fingerprinting” LLMs through their factor correlation patterns on standardized psychological assessments to deepen the understanding of LLMs constructs representation. Using the Humor Style Questionnaire as a case study, we analyze how six LLMs represent and correlate humor-related constructs to survey participants. Our results show that they exhibit little similarity to human response patterns. In contrast, participants’ subsamples demonstrate remarkably high internal consistency. Exploratory graph analysis further confirms that no LLM successfully recovers the four constructs of the Humor Style Questionnaire. These findings suggest that despite advances in natural language capabilities, current LLMs represent psychological constructs in fundamentally different ways than humans, questioning the validity of application as human simulacra.
Read the paper
Münker, S. (2025): Political Bias in LLMs: Unaligned Moral Values in Agent-centric Simulations.
Abstract. Contemporary research in social sciences increasingly utilizes state-of-the-art generative language models to annotate or generate content. While these models achieve benchmarkleading performance on common language tasks, their application to novel out-of domain tasks remains insufficiently explored. To address this gap, we investigate how personalized language models align with human responses on the Moral Foundation Theory Questionnaire. We adapt open-source generative language models to different political personas and repeatedly survey these models to generate synthetic data sets where model-persona combinations define our sub-populations. Our analysis reveals that models produce inconsistent results across multiple repetitions, yielding high response variance. Furthermore, the alignment between synthetic data and corresponding human data from psychological studies shows a weak correlation, with conservative persona-prompted models particularly failing to align with actual conservative populations. These results suggest that language models struggle to coherently represent ideologies through in-context prompting due to their alignment process. Thus, using language models to simulate social interactions requires measurable improvements in in-context optimization or parameter manipulation to align with psychological and sociological stereotypes properly.
Read the paper
Münker, S., Kugler, K., Rettinger, A. (2025): Zero-shot prompt-based classification: topic labeling in times of foundation models in German Tweets.
Abstract. Filtering and annotating textual data are routine tasks in many areas, like social media or news analytics. Automating these tasks allows to scale the analyses wrt. speed and breadth of content covered and decreases the manual effort required. Due to technical advancements in Natural Language Processing, specifically the success of large foundation models, a new tool for automating such annotation processes by using a text-to-text interface given written guidelines without providing training samples has become available. In this work, we assess these advancements in-the-wild by empirically testing them in an annotation task on German Twitter data about social and political European crises. We compare the prompt-based results with our human annotation and preceding classification approaches, including Naive Bayes and a BERT-based fine-tuning/domain adaptation pipeline. Our results show that the prompt-based approach – despite being limited by local computation resources during the model selection – is comparable with the fine-tuned BERT but without any annotated training data. Our findings emphasize the ongoing paradigm shift in the NLP landscape, i.e., the unification of downstream tasks and elimination of the need for pre-labeled training data.
Read the paper
Sittar, A., Mladenić, D., Guček, A., Grobelnik, M. (2025): BAR-analytics: A web-based platform for analyzing information spreading barriers in news: Comparative analysis across multiple barriers and events.
Abstract. This paper presents BAR-Analytics, a web-based, open-source platform for analysing how news spreads across geographical, economic, political, and cultural boundaries. Using the Russian–Ukrainian and Israeli–Palestinian conflicts as case studies, the platform combines propagation analysis, trend analysis, sentiment analysis, and temporal topic modelling. The work demonstrates how integrated analytical tools can support comparative research on information-spreading barriers across major events and regions.
Read the paper
Sittar, A., Guček, A., Grobelnik, M. (2025): Agent-Based Simulations of Online Political Discussions: A Case Study on Elections in Germany.
Abstract. User engagement on social media platforms is shaped by conversation history, time constraints, and reward-driven interactions. This paper presents an agent-based simulation approach for modelling online political discussions using German Twitter data. The study fine-tunes AI models to generate posts and replies and evaluates them using sentiment analysis, irony detection, and offensiveness classification. The simulation applies a myopic best-response model to represent agent decision-making, showing how historical context and resource constraints influence engagement dynamics.
Read the paper
Stolwijk, S. et al. (2025): Can we use automated approaches to measure the quality of online political discussion? How to (not) measure interactivity, diversity, rationality, and incivility in online comments to the news.
Abstract. This article explores the (in)ability of automated tools to measure the deliberative quality of online user comments along the standards set out by Habermas: interactivity, diversity, rationality, and (in)civility. Utilizing a stratified sample of manually coded comments ( n = 3,862) responding to news videos on YouTube and Twitter , we examined the performance of rule-based measures (i.e. dictionaries), machine-learning classifiers (conventional and transformer-based) and measurements by generative AI (Llama 3.1, GPT-4o, GPT-4T). We present results for over 50 metrics side-by-side to judge the opportunity costs of choosing one method over another. The results revealed strong variation across different groups of models. Overall, our expectation that more modern methods (transformers and generative AI) outperform the older, simpler ones was confirmed. However, the absolute differences between these model groups strongly depended on the measured concept, and we observed strong variance in performance among models of the same group. We provide recommendations for future research that balance ease of use with the performance of automated measurements, along with important cautions to consider.
Read the paper
Stolwijk, S. (2025): Twin of Online Social Networks – Definition of Metrics.
Abstract. This report proposes a set of metrics to determine the deliberative quality of discussions on social media in general and TWON in particular. Chapter 2 lists the key indicators of: (1) Exposure to political content, (2) Engagement with political content, (3) Contributing political content, (4) Diversity of exposure and (5) Quality of exposure. Chapter 3 explains how and why this set of indicators differs from the typical list of deliberative indicators and proposes to view deliberation from a summative rather than an additive perspective. In this view, social media do not need to aim at perfect deliberation within one platform; rather, the goal is to contribute to deliberation at a societal scale via the platform. We propose that social media can contribute especially by offering an avenue for users (citizens, journalists and politicians alike) to be exposed to political debate, but also to engage and participate in that debate. In addition, social media can connect otherwise unconnected users and expose them to ideas they might otherwise have missed. Ideally, these ideas are substantiated with arguments and evidence. Chapter 4 evaluates a large set of automatic classifiers to determine the degree to which social media comments meet several deliberative criteria, specifically whether comments are rational, interactive, diverse and civil. Results show how more modern techniques like fine-tuned transformers and generative large language models have improved our ability to reproduce manual codings automatically, but also that results vary considerably between models. Chapter 5 integrates the aims of Chapter 3 with the results of Chapter 4 and translates them to the case of TWON to arrive at the metrics proposed in Chapter 2. It adds tests of the performance of different classifiers to determine whether a comment is political or not. Chapter 6 takes a look into the future, beyond what is currently feasible for TWON, to explore whether new techniques can help determine the deliberative quality of online social media debates to the more fine-grained level of specific claims and shows some promising first results.
Read the paper
Žust, M., Grobelnik, M., Grobelnik, A., Sittar, A. (2025): Towards AI-Powered Real-Time Negotiation Agent.
Abstract. Negotiation is a fundamental human skill critical to personal, business, and societal interactions, yet many lack the expertise to negotiate effectively, leading to significant economic losses. Research shows unskilled negotiators in industries lose large amounts of value due to inadequate real-time guidance, while existing AI tools focus on automation rather than dynamic support. This introductory study aims to help unskilled negotiators with accessible, real-time assistance to improve outcomes across diverse contexts. Here we introduce a web-based negotiation agent that combines AI’s analytical power with human intuition, providing live recommendations during simulated conversations. Unlike prior tools limited to post-event analysis or structured tasks, our agent transcribes dialogue, builds a dynamic world model, and offers tailored advice beyond what standalone AI or humans typically achieve. This hybrid approach reveals that integrating AI into live negotiations can bridge the efficacy gap, making proficiency more attainable than previously thought with static or isolated solutions. Broadly, it suggests a shift toward collaborative AI-human systems to address possibly complex interpersonal challenges
Read the paper

2024

Bojić (2024): AI alignment: Assessing the global impact of recommender systems.
Abstract. The recent growing concerns surrounding the pervasive adoption of generative AI can be traced back to the long-standing influence of AI algorithms that have predominantly served as content curators on large online platforms. These algorithms are used by online services and platforms to decide what content to show and in what order, and they can have a negative impact, including the spread of misinformation, social polarization, and echo chambers around important topics. Frances Haugen, a former Facebook employee turned whistleblower, has drawn significant public attention to this issue by revealing the company’s alleged knowledge about the negative impacts of their own algorithms. Additionally, a recent initiative to ban TikTok as a threat to US national security indicates the influence of recommender systems. The objective of this study is threefold. The first goal is to provide an exhaustive evaluation of the profound worldwide influence exerted by algorithm-based recommendations. The second goal is to determine the degree of priority accorded by the scientific community to pivotal subjects in recommender systems discussions, such as misinformation, polarization, addiction, emotional contagion, privacy, and bias. Finally, the third goal is to assess whether the level of scientific research and discourse is commensurate with the significant impact these recommendation systems have globally. The research concludes the impact of recommender systems on society has been largely neglected by the scientific community, despite the fact that more than half of the world’s population interacts with them on a daily basis. This becomes especially apparent when considering that algorithms exert influence not just on major societal issues but on every aspect of a user’s online experience. The potential consequences for humanity are discussed, such as addiction to technology, weakening relations between humans, and the homogenizing effects on human minds. One possible direction to address the challenges posed by these algorithms is the application of algorithmic regulation to promote content diversity and facilitate democratic engagement, such as the tripartite solution which is elaborated upon in the conclusion. Therefore, future research should not only be centered around further evaluating influence of this technology, but also the analysis of how such systems can be regulated. A broader conversation among all stakeholders should be evoked on these potential approaches, aiming to align AI with societal values and enhance human well-being.
Keywords: AI alignment, Digital humanism, Recommender systems, Global impact, Social polarization
Ljubisa Bojić. 2024. “AI alignment: Assessing the global impact of recommender systems”. In: Futures 160 (2024).
Read the paper
Bojić et al. (2024): CERN for AI: a theoretical framework for autonomous simulation-based artificial intelligence testing and alignment.
Abstract. This paper explores the potential of a multidisciplinary approach to testing and aligning artificial intelligence (AI), specifically focusing on large language models (LLMs). Due to the rapid development and wide application of LLMs, challenges such as ethical alignment, controllability, and predictability of these models emerged as global risks. This study investigates an innovative simulation-based multi-agent system within a virtual reality framework that replicates the real-world environment. The framework is populated by automated ‘digital citizens,’ simulating complex social structures and interactions to examine and optimize AI. Application of various theories from the fields of sociology, social psychology, computer science, physics, biology, and economics demonstrates the possibility of a more human-aligned and socially responsible AI. The purpose of such a digital environment is to provide a dynamic platform where advanced AI agents can interact and make independent decisions, thereby mimicking realistic scenarios. The actors in this digital city, operated by the LLMs, serve as the primary agents, exhibiting high degrees of autonomy. While this approach shows immense potential, there are notable challenges and limitations, most significantly the unpredictable nature of real-world social dynamics. This research endeavors to contribute to the development and refinement of AI, emphasizing the integration of social, ethical, and theoretical dimensions for future research.
Read the paper
Bojić et al. (2024): Navigating the Digital Age: An In-Depth Exploration into the Intersection of Modern Technologies and Societal Transformation.
Abstract. This anthology explores the impacts of new digital technologies on society. In four sections international scholars investigate issues such as the transformation of communication technologies and journalism, the digital transformation and algorithmic bureaucracy, the relationship of democracy, security and technology, and new media, artificial intelligence and the arts. It offers critical perspectives engaging with the ethical, moral, and cultural issues developing in relkationship with the digital transformation.
Read the paper
Golob, L., Sittar, A. (2024): Fact Manipulation in News: LLM-Driven Synthesis and Evaluation of Fake News Annotation.
Abstract. Advancements in artificial intelligence and increased internet accessibility have made it simpler to create and disseminate fake news with customized content. However, they also improved the ability to analyze and identify such misinformation. To effectively train high-performance models, we require high-quality, up-to date training datasets. This article delves into the potential for generating fake news through factual modifications of articles. This is facilitated by prompt-based content generated by large language models (LLMs), which can identify and manipulate facts. We intend to outline our methodology, highlighting both the capabilities and limitations of this approach. Additionally, this effort has resulted in new quality synthetic data that can be incorporated into the standard FAK-ES dataset.
Read the paper
Ludwig-Walz et al. (2024): Integrating European contexts and needs into WHO guiding principles on online mental health content for young people: five recommendations.
Abstract. The World Health Organization (WHO) has published guiding principles on online mental health content for young people, recognizing the transformative role of online technologies in providing mental health support. This article aims to integrate the key principles of the WHO report into European contexts and needs, to enhance the efficacy and relevance of online mental health services for young people.
Read the paper
Sittar, A., Mladenić, D., Grobelnik, M. (2024): News dissemination: a semantic approach to barrier classification.
Abstract. News media play an important role in spreading information globally, but some events receive only limited coverage in specific geographic regions because of information-spreading barriers. These barriers may be political, geographical, economic, cultural, or linguistic. This paper proposes a semantic approach to classifying such barriers by extracting Wikipedia-based concepts from news articles. The study shows how semantic representations can help identify the factors that shape whether and how news travels across regions and communities.
Read the paper
Sittar, A., Guček, A., Mladenić, D. (2024): Perception of AI in Slovenia.
Abstract. This paper introduces the AI News Monitor system developed for real-time monitoring and analysis of artificial intelligence (AI) perception in global and local news media. Leveraging data from the Event Registry platform, the AI News Monitor tracks AI-related news articles across multiple dimensions, providing insights through three key views: a global historical overview, current global trends, and local trends specific to Slovenian media. The system facilitates both passive observation of AI discourse and active exploration of specific AI-related events. Our illustrative analysis reveals significant global trends, including heightened media focus on deep learning, generative AI, and robotics, and examines the implications of these trends on public trust in AI. Additionally, the paper discusses the practical applications of the AI News Monitor for stakeholders such as policymakers, journalists, business leaders, and researchers. We conclude with a discussion on the impact of media coverage on public perception of AI and propose possible future enhancements of the system, including broader language and source coverage.
Read the paper
Weinhardt et al. (2024): Digital Democracy: A Wake-Up Call – How IS Research Can Contribute to Strengthening the Resilience of Modern Democracies.
Christof Weinhardt, Jonas Fegert, Oliver Hinz, Wil M. P. van der Aalst. 2024. “Digital Democracy: A Wake-Up Call. How IS Research Can Contribute to Strengthening the Resilience of Modern Democracies”. In: Business & Information Systems Engineering 66(2), p. 127–134.
Read the paper

2023

Bezzaoui, I., Nikolajevic, N., Fegert, J. (2023): Demokratiegefährdende Plattform-Mechanismen – Erkennen, Verstehen, Bekämpfen.
Abstract. Das Internet hat die Welt näher zusammengebracht und einen digitalen Raum geschaffen, der internationale Kommunikation und Vernetzung ermöglicht. Dieser Raum hat jedoch nicht nur die deliberative Kraft entwickelt, die ihm ursprünglich zugeschrieben wurde. Für populistische Bewegungen, insbesondere die extreme Rechte, wurden die sozialen Netzwerke zu mächtigen Instrumenten der Selbstorganisation, wie der Angriff auf den Deutschen Bundestag (2020), der Sturm auf das US-Kapitol (2021) und der Angriff auf den brasilianischen Regierungssitz (2023) zeigen. Phänomene wie Filterblasen und die Verbreitung von Desinformation haben reale Auswirkungen auf Gesellschaften. Eine kritische Auseinandersetzung mit den Plattform-Mechanismen, die die gesellschaftliche Polarisierung im digitalen Raum vorantreiben, sowie eine Erforschung, wie sich diese Polarisierungstendenzen auf gesellschaftliche Realitäten auswirken, ist daher notwendiger denn je. Desinformationskampagnen sind zu einer Bedrohung für die Demokratie und den sozialen Zusammenhalt geworden. Daher bedarf es einerseits eines umfassenden Verständnisses ihrer Mechanismen und ihrer Ausbreitung und andererseits, darauf aufbauend, Methoden zu ihrer systematischen Bekämpfung. Für das Verständnis und die Entwicklung von Strategien und Instrumenten zur Analyse sowie zur Bekämpfung von gesellschaftlicher Spaltung sind interdisziplinäre Ansätze unerlässlich. Bei den beschriebenen Herausforderungen handelt es sich nicht nur um solche, die die Kommunikationswissenschaft tangieren, sondern um Vorgänge, die auch auf technologischer Ebene analysiert und interpretiert werden sollten. Dieser Beitrag stellt Ansätze und Ergebnisse aus zwei interdisziplinären Forschungsprojekten vor, die demokratiegefährdende Tendenzen im digitalen Raum erkennen, verstehen und bekämpfen sollen. Die weltweit beobachtete Zunahme der politischen Polarisierung wird oft auf den Trend der algorithmischen Filterung von Inhalten in sozialen Medien, auf Nachrichtenplattformen oder Suchmaschinen zurückgeführt. Es wird angenommen, dass die weitverbreitete Nutzung von Nachrichtenempfehlungssystemen (NRS) Nutzer*innen in homogenen Informationsumgebungen einschließt und dadurch die affektive, ideologische und wahrgenommene Polarisierung verstärkt (Sunstein, 2001). Sowohl die affektive als auch die ideologische Polarisierung sind durch eine Trennung von Individuen verschiedener politischer Lager, typischerweise von der ideologischen Linken und Rechten, über politische Differenzen gekennzeichnet (Webster & Abramowitz, 2017). Im Falle der affektiven Polarisierung äußert sich dies in einer starken Sympathie gegenüber der eigenen Partei, begleitet von einer gleichzeitigen Abneigung der gegnerischen Partei (Iyengar et al., 2012). Die ideologische Polarisierung basiert auf der Distanz zwischen Ablehnung und Unterstützung von Positionen oder Einstellungen zu politischen Themen (DiMaggio et al., 1996). Die wahrgenommene Polarisierung wiederum gibt an, wie sehr eine Person das Meinungsklima in der Gesellschaft entlang von Parteilinien oder Ideologien als polarisiert wahrnimmt (z. B. Yang et al., 2015). In einer interdisziplinären Studie werden drei Online-Experimente mit laufenden Algorithmen durchgeführt, die verschiedene NRS vergleichen. Untersucht wird, welchen Einfluss verschiedene NRS-Arten auf die Nutzer*innen haben, sowie das Sentiment der Nachrichtentexte und die Dauer, für die Nutzer*innen den NRS ausgesetzt sind. Ziel ist es, die Wirkung realer NRS nicht nur zu verstehen, sondern auch alternative Konzepte zu entwickeln, die die positiven Eigenschaften bisheriger Systeme aufgreifen, darüber hinaus aber nicht demokratiegefährdend sind. Die Tatsache, dass heutzutage fast jede*r Inhalte im Internet veröffentlichen kann, erhöht nicht nur die Möglichkeiten der sozialen Teilhabe, sondern schafft auch neue Möglichkeiten für die Verbreitung von Desinformation und Propaganda (Shu et al., 2017). Die COVID-19-Pandemie hat bereits eine Flut von Falschmeldungen hervorgebracht und gezeigt, wie wichtig es ist, verlässliche von irreführenden Informationen unterscheiden zu können (Sharma et al., 2021; Delcker et al., 2020). Darüber hinaus erfordert der Krieg gegen die Ukraine eine besondere Konfrontation mit Desinformation, die von staatlichen Stellen verbreitet wird. Online Desinformation wird daher als eine der größten Herausforderungen für die Demokratie, den Journalismus und die freie Meinungsäußerung angesehen, was den Bedarf an Forschung zur Erkennung von betrügerischen Inhalten erhöht (Shu et al., 2017). Derzeit ist die Forschung zur Erkennung von “Fake News” mithilfe von Systemen, die auf maschinellem Lernen basieren, ein schnell wachsendes Gebiet, das zahlreiche Disziplinen umfasst, darunter Informatik, Medien- und Kommunikationswissenschaften, Sozialwissenschaften und Psychologie (vgl. Yu & Lo, 2020; Verma et al., 2021; Kapantai et al., 2021; Mahyoob et al., 2020). Mit präventiven Maßnahmen und Mechanismen setzt ein weiteres interdisziplinäres Forschungsprojekt an. Gemeinsam mit Organisationen der Zivilgesellschaft wird darin versucht, Nutzer*innen über verschiedene Plattformen hinweg zu befähigen, Nachrichten und Social-Media-Inhalte kritisch zu hinterfragen. Das Projekt verfolgt einerseits ein umfassendes Verständnis der Mechanismen von Desinformation und ihrer Ausbreitung. Andererseits sollen auf dieser Wissensgrundlage Methoden entwickelt werden, um die Verbreitung von Desinformation einzudämmen. Zu diesem Zweck wird das Projekt eine erklärbare KI (XAI) für eine Beteiligungsplattform entwickeln, die darauf abzielt, online Desinformation zu bekämpfen, indem sie diese den Nutzer*innen sichtbar macht und somit aktiv vor deren Auftreten warnt. Die XAI soll dabei unterstützen, kritische Medienkompetenz unter Bürger*innen zu fördern, um den schädlichen Folgen von Desinformationskampagnen wirksam und nachhaltig entgegenzutreten. In diesem Sinne soll ein Beitrag zur Förderung der demokratischen Teilhabe geleistet werden. Vertrauen ist dabei eine der wichtigsten Komponenten für die Förderung aktiver, engagierter und informierter Bürger*innen (Dahlgren, 2009). Dementsprechend soll die Einreichung dazu beitragen, aufzuzeigen, welche Perspektive kritische Technologieforschung einbringen kann, um Systeme der Desinformation und Algorithmic Biases zu enttarnen. Denn um die demokratische Resilienz sowie das Vertrauen von Bürger*innen nachhaltig zu stärken, bedarf es interdisziplinärer Forschungsansätze zur umfassenden Untersuchung und Bekämpfung demokratiegefährdender Phänomene im digitalen Raum.
Isabel Bezzaoui, Nevena Nikolajevic, Jonas Fegert. 2023. “Demokratiegefährdende Plattform-Mechanismen – Erkennen, Verstehen, Bekämpfen”. In: KI – Konflikte – Konventionen, Polkomm 2023.
Read the paper
Buza (2023): Time Series Forecasting with Distortion-Aware Convolutional Neural Networks.
Abstract. Thanks to its prominent applications in science, medicine, industry and finance, time series forecasting has been a key area of research. Several recent solutions for time series forecasting are based on convolutional neural networks. While convolutional layers act as local pattern detectors, we point out that they match local patterns in a rigid manner in the sense that they do not account for local shifts and elongations. In this paper, we address this issue and propose distortion-aware convolution. We discuss how to train neural networks with distortion-aware convolution in a multi-phase training process. Our experimental results on publicly available real-world datasets from various domains show that replacing conventional convolution by distortion-aware convolutions leads to more accurate models for time series forecasting both in terms of mean squared error and mean absolute error. Furthermore, distortion-aware convolution may serve as an essential building block of future neural networks. In order to support reproduction and follow-up works, we made our prototypical implementation publicly available at https://github.com/kr7/dcnn-forecast.
Keywords: time series forecasting, convolutional neural networks, distortion aware convolution
Krisztian Buza. 2023. “Time Series Forecasting with Distortion-Aware Convolutional Neural Networks”. In: MILETS 2023 (August 06–10, 2023, Long Beach, CA).
Read the paper
Buza (2023): Classification of Sparse and Irregularly Sampled Time Series with Convolutional Neural Networks.
Abstract. Analysis of sparse and irregularly sampled time series is an important task with prominent applications in various domains such as medicine, manufacturing, social networks and environmental sciences. In this paper, we focus on convolutional neural networks for the classification of the aforementioned type of time series. We point out that dynamic time warping (DTW) is suitable in case of such time series and present a new convolutional block, dynamic time warping convolution with sparse input (DConv). We propose to use DConv for the classification of sparse and irregularly sampled time series. We also consider networks with sparsity-invariant convolution, an operation which is known in the domain of 3D reconstruction, but new in the domain of time series classification, and use neural networks with sparsity-invariant convolution as strong baselines. We perform experiments with fully-convolutional neural networks (FCNs) and residual networks (ResNet) on publicly available time series datasets. The results show that neural architectures with the proposed convolutional block outperform their counterparts, in most cases statistically significantly. We made our implementation publicly available at https://github.com/kr7/dconv.
Keywords: sparse and irregularly sampled time series, classification, convolutional neural networks, dynamic time warping
Krisztian Antal Buza. 2023. “Classification of Sparse and Irregularly Sampled Time Series with Convolutional Neural Networks”. In: MILETS 2023.
Read the paper
Grobelnik et al. (2023): Emergent Behaviors from LLM-Agent Simulations.
Abstract. This paper hypothesizes that complex emergent behaviors can arise from multi-agent simulations involving Large Language Models (LLMs), potentially replicating intricate societal structures. We tested this hypothesis through three progressively complex simulations, where we evaluated the LLM-agents’ understanding, task execution, and their capacity for strategic interactions such as deception. Our results show a clear gap in reasoning ability between LLMs such as GPT-3.5-Turbo and GPT-4, especially in simpler simulations. We demonstrate emergent behaviors can arise from LLM-agent simulations ranging from simple games to geopolitics.
Keywords: large language models, multi-agent simulations, emergent behaviors, societal structures, gpt, simulation environments, agent-based modelling, agent architecture
Adrian Mladenic Grobelnik, Faizon Zaman, Jofre Espigule-Pons, Marko Grobelnik. 2023. “Emergent Behaviors from LLM-Agent Simulations”. In: SiKDD, October 2023, Ljubljana, Slovenia.
Read the paper
Sittar, A., Mladenić, D., Grobelnik, M. (2023): Profiling the barriers to the spreading of news using news headlines.
Abstract. News headlines can be a good data source for detecting the barriers to the spreading of news in news media, which can be useful in many real world applications. In this study, we utilize semantic knowledge through the inference-based model COMET and the sentiments of news headlines for barrier classification. We consider five barriers, including cultural, economic, political, linguistic, and geographical and different types of news headlines, including health, sports, science, recreation, games, homes, society, shopping, computers, and business. To that end, we collect and label the news headlines automatically for the barriers using the metadata of news publishers. Then, we utilize the extracted common-sense inferences and sentiments as features to detect the barriers to the spreading of news. We compare our approach to the classical text classification methods, deep learning, and transformer-based methods.The results show that (1) the inference-based semantic knowledge provides distinguishable inferences across the 10 categories that can increase the effectiveness and enhance the speed of the classification model; (2) the news of positive sentiments cross the political barrier, whereas the news of negative sentiments cross the cultural, economic, linguistic, and geographical barriers; (3) the proposed approach using inferences-based semantic knowledge and sentiment improves performance compared with using only headlines in barrier classification. The average F1-score for 4 out of 5 barriers has significantly improved as follows: for cultural barriers from 0.41 to 0.47, for economic barriers from 0.39 to 0.55, for political barriers from 0.59 to 0.70 and for geographical barriers from 0.59 to 0.76.
Keywords: news spreading barriers, profiling news spreading barriers, common-sense inferences, sentiment analysis, economic barrier, political barrier, cultural barrier, linguistic barrier
Abdul Sittar, Dunja Mladenić, Marko Grobelnik. 2023. “Profiling the barriers to the spreading of news using news headlines”. In: Frontiers in Artificial Intelligence 6 (2023).
Read the paper
Sittar, Mladenić (2023): An approach to creating a time-series dataset for news propagation: Ukraine-war case study.
Abstract. An efficient technique to comprehend news spreading can be achieved through the automation of machine learning algorithms. These algorithms perform the prediction and forecasting of news dissemination across geographical barriers. Despite the fact that news regarding any events is generally recorded as a time-series due to its time stamps, it cannot be seen whether or not the news time-series is propagating across geographical barriers. In this article, we explore an approach for generating time-series datasets for news dissemination that relies on Chat-GPT and sentence-transformers. The lack of comprehensive, publicly accessible event-centric news databases for use in time-series forecasting and prediction is another limitation. To get over this bottleneck, we collected a news dataset consisting of 1 year and 3 months related to the Ukraine war using Event Registry. We also conduct a statistical analysis of different time-series (propagating, unsure, and not-propagating) of different lengths (2, 3, 4, 5, and 10) to document the prevalence of geographical barriers. The dataset is publicly available on Zenodo.
Keywords: news propagation, time-series dataset, geographical barriers, Ukraine-war
Abdul Sittar, Dunja Mladenić. 2023. “An approach to creating a time-series dataset for news propagation: Ukraine-war case study”. In: Data Mining and Data Warehouses – SiKDD : Information Society – IS 2023.
Read the paper
Stolwijk et al. (2023): Refining Deliberative Standards for Online Political Communication: Introducing a Summative Approach to Designing Deliberative Recommender Systems.
Abstract. Measuring deliberative debate quality is an emerging topic in computational work, since it allows applying deliberative democratic ideas in an online domain. When doing so, many studies follow Habermas in defining norms of online deliberative debate quality. They then proceed to propose and test new ways to measure indicators like equality, diversity, interactiveness, rationality, and civility. Consequently, implementing them within recommender systems is the necessary next step to realize those values in online communication. Although this is important work, we argue that recent advances in political science suggest that constructing a system which produces such a deliberative debate is unlikely to, by itself, contribute in an optimal way to deliberative democracy at a societal scale. Instead, we propose a complementary, summative approach to designing deliberative recommender systems. It treats online platforms as complementary to other communication channels, and argues for optimizing how to best facilitate (summative) deliberation at a societal scale rather than perfecting (micro) discussions between citizens. We illustrate this with an example of how a news recommender system based on a summative approach would have to be designed vis-a-vis a more traditional, additive approach.
Keywords: deliberative democracy, normative standards, online debate quality, computational text analysis, moral recommender systems
Sjoerd B. Stolwijk, Corinna Oschatz, Michael Heseltine, Damian Trilling. 2023. “Refining Deliberative Standards for Online Political Communication: Introducing a Summative Approach to Designing Deliberative Recommender Systems”. NORMalize 2023: The First Workshop on the Normative Design and Evaluation of Recommender Systems, September 19, 2023, co-located with the ACM Conference on Recommender Systems 2023 (RecSys 2023), Singapore.
Read the paper

Results & Publications

Deliverables

Publications

2026

2025

2024

2023

Source code