Sermons in the Windy City: Analyzing Political Messaging in Religious Discourse Across Chicago Congregations

Religious congregations represent vital components of American civil society, serving as key sites for the transmission of theological, social, and political ideas. Among the most significant channels for this transmission are sermons, which reach millions of Americans weekly. Research demonstrates that political messages conveyed within congregations can influence parishioners' civic behaviors, including voting patterns, contacting elected officials, and participating in social movements (e.g., Beyerlein, Sikkink, and Hernandez 2019; Brown and Brown 2003).

This study examines how sermons address politically salient local conditions and events across Chicago's diverse religious landscape. We analyze sermon content using a novel dataset drawn from an approximate census of Chicago congregations, addressing three primary research questions:

1. To what degree do sermons discuss socio-political issues like policing and community violence?
2. How responsive are sermons to changing local conditions or politically significant events?
3. How do patterns of political messaging vary across neighborhood contexts and religious traditions?

Our research contributes to existing scholarship in several ways. First, while previous studies on political content in sermons have often relied on self-reported surveys (Scheitle and Cornell 2015; Smidt 2016) or drawn from small, non-representative samples (Boussalis, Coan, and Holman 2021; Krull and Gilliland 2023), we analyze sermons from across Chicago's religious ecosystem. Second, we employ computational methods to measure political content at scale, focusing on discussions of policing and community violence. Third, by situating congregations within their neighborhood contexts using GIS, we reveal relationships between sermon content and local community characteristics.

Data and Methods

Our corpus of sermons builds upon the Chicago Congregations Project (CCP), which has assembled a novel dataset capturing over 3,000 congregations across Chicago's 77 Community Areas. We utilize information on congregations' social media accounts and websites to access publicly available digital recordings of sermons between 2022-2024. We estimate that approximately 35-50% of congregations post sermons online at least once a month. Although sermons are not available for every congregation, we leverage the CCP's approximate population data to construct weights for post-stratification, enabling more representative estimates of sermon content throughout the city.

To generate transcripts of digital recordings of sermons posted on Facebook and YouTube, we use speech-to-text models, primarily OpenAI’s Whisper and Deepgram Enhanced. Whisper provides an open-source transcription framework capable of handling varied audio conditions, while Deepgram Enhanced offers additional robustness for challenging cases, such as sermons with background noise, strong accents, or lower-resourced languages. By combining these models, we aim to ensure high-quality transcripts across diverse congregations and sermon styles. This approach allows us to produce transcripts of over 170,000 sermons from across Chicago. 

Analyzing political messaging in sermons presents several methodological challenges. Sermons are linguistically complex, often featuring historical references, metaphor, allegory, and scriptural citations. They tend to be lengthy (median length in our pilot data: 78 minutes and 6,550 words), with non-linear narrative structures. Political messages may be brief or indirect, making conventional text analysis approaches insufficient.

To address these challenges, we employ large language models (LLMs) to detect explicit and implicit references to our focal topics: policing and community violence. Our measurement approach includes:

1. Using LLMs to classify text segments as discussing policing or community violence
2. Creating a manually coded validation set to assess model performance
3. Testing models' ability to identify political content both in isolation and within longer sermon transcripts

In our pilot analysis, we found that GPT-4 displayed particularly strong performance in detecting political content, with precision ranges of 0.867-0.944 and recall ranges of 0.85-1.0 across various test conditions. 

Preliminary Findings and Broader Implications

Our pilot analysis on a subset of 42 congregations revealed several notable patterns. First, discussions of policing and community violence in sermons are strongly correlated - congregations that discuss one topic tend to address the other as well. Second, congregations located in communities with greater proportions of Black residents tend to discuss both policing and violence more frequently in sermons compared to congregations in areas with higher proportions of Hispanic/Latino or white residents.

Examining trends over time, we found evidence of seasonality in discussions of violence. Sermons addressing violence increased in late spring/early summer of both 2022 and 2023 as violent crime rose in the city. However, these discussions often tapered off by mid-to-late summer, even as local violence peaked, suggesting sermons may anticipate rather than directly respond to community violence levels. 

In the complete study, we will examine how sermons anticipate and respond to salient local or national events, such as incidents of police brutality, elections, or policy changes. Our analysis will explore how congregations' political messaging is structured by factors like geographic proximity to events, social distance (demographic similarity to affected communities), and religious tradition. We will also investigate the semantic similarity of sermons across congregations.

This research offers insight into the messages that religious congregations disseminate across an urban context. By applying computational methods to examine political messaging in sermons, we contribute to understanding the role congregations play in framing social issues and potentially influencing civic engagement.

References

Beyerlein, K, D. Sikkink, and E. Hernandez. 2019. “Citizenship, Religion, and Protest: Explaining Latinos’ Differential Participation in the 2006 Immigrant Rights Marches.” Social Problems, 66: 163–193.

Boussalis, C., T. G. Coan, and M. R. Holman. 2021. “Political Speech in Religious Sermons.” Politics and Religion, 14: 241–268.

Brown, R. K., and R. E. Brown. 2003. “Faith and Works: Church-Based Social Capital Resources and African American Political Activism.” Social Forces, 82: 617–641.

Krull, L. M., and C. C. Gilliland. 2023. “‘You Know, the Church Has Never Agreed on Everything’: Analyzing the Prophetic and Pragmatic Voice in Clergy Sermons.” Sociology of Religion, 84: 324–348. 

Scheitle, C. P., and N. Cornell. 2015. “Hearing Clergy Speak About Social and Political Issues: Examining the Effects of Religious Tradition and Personal Interest*.” Social Science Quarterly, 96: 148–160.

Smidt, C. E. 2016. Pastors and Public Life: The Changing Face of American Protestant Clergy. Oxford, New York: Oxford University Press.