• News

New Publication from ComAI P6 Project

We are happy to announce the publication of a new paper by our colleagues from the project “Political Discourse: ComAI and Deliberative Quality” in the ComAI research unit, Patrick Frey and Cornelius Puschmann (ZeMKI, University of Bremen), and Smita Bhattacharya and Gregor Wiedemann (Leibniz Institute for Media Research | Hans-Bredow-Institut), titled „Can AI Gauge Deliberative Quality? Evaluating Deliberation on Social Media With Large Language Models.“ 

Although social media has become a central arena for deliberative discourse, debates are frequently characterized by incivility and other norm violations. Due to the volume of online discourse, it is challenging to assess when and where interventions, such as community‐based counterspeech, are required. In this study, Frey and colleagues investigate how large language models (LLMs) can be used to automatically assess deliberative quality, considering both classical dimensions such as rationality and civility (Type I) and more inclusive forms of communication, including emotionality, storytelling, and humor (Type II). 

The researchers evaluated multiple open-weight LLMs and applied the model offering the best balance between predictive performance and computational efficiency to 64,223 comments from German-language climate change discussions on X. Their findings show that LLMs can reliably capture complex deliberative dimensions, including Type II dimensions that are often difficult to detect due to subtle linguistic cues and class imbalances. The analysis also reveals distinct communicative patterns: While public speakers predominantly employ civil, rational, constructive, and common-good-oriented communication, other debating users show higher levels of reciprocal engagement, humor, and negative emotions, alongside more frequent incivility.  

Overall, the study demonstrates that AI-driven measurement can serve as a foundational tool for identifying norm violations and supporting targeted counterspeech and other discourse interventions to promote inclusive digital public spheres. Moreover, such approaches can help assess the long-term effects of moderation and interventions on deliberative quality. 

The article is part of the fully open-access special issue “The Role of AI for Counter Speech: Detection, Intervention, and Risks,” edited by Lena Frischlich, Cathy Buerger, and Magdalena Obermaier. 

Read the full paper here: 

Frey, P., Bhattacharya, S., Wiedemann, G., & Puschmann, C. (2026). Can AI Gauge Deliberative Quality? Evaluating Deliberation on Social Media With Large Language Models. Media and Communication, 14, 12494. https://doi.org/10.17645/mac.12494