Relevant Research
Review of Algorithms and Journalism: Exploring (Re)Configurations
Media and Communication
By Cindy Royal, PhD
Professor and Director, Media Innovation Lab
August 17, 2020
Media and Communication is an open-access journal with international contributions that deals with broad-ranging media research topics, having a particular emphasis on social and cultural relevance. I was intrigued by a recent special issue on Algorithms and Journalism: Exploring (Re)Configurations, edited by Rodrigo Zamith of the University of Massachusetts at Amherst and Mario Haim of University of Leipzig, Germany. Algorithms influence journalism in numerous ways, in how they use products for creating journalism and how platforms disseminate their work, engage their audiences and provide data. This special issue highlights how an advanced understanding of algorithms and their effects is necessary for educators and scholars, but also identifies important aspects for professionals making decisions about their use and application.
Algorithms: The Future is Now
In the introductory essay “Algorithmic Actants in Practice, Theory, and Method,” the editors identify that journalists have mixed feelings about the increasing sophistication of technology as it affects their workflows. “Journalists have sought to normalize the technology by negotiating them against existing values and practices, and perhaps even reified some normative ideological constructs by finding unique value in what they offer as humans" (Zamith & Haim, 2020, p. 1). In most cases represented in the research projects, media workers focused on the value and improvements brought by technology, in order to free up the human aspect for more creative applications. Most interesting to me, however, is the way that the articles in the issue address changes to roles, norms and routines and that these changes “require scholars to revisit existing theories and methods for understanding emerging assemblages, practices, and norms” (Zamith & Haim, 2020, p. 2).
In an email interview, Zamith described the genesis of the special issue. “I was instantly drawn to the idea of a special issue that examines how algorithms may -- and may not -- be impacting the attitudes, behaviors, content, discourses, and ethics in journalism,” Zamith said. “Similarly, my methodological interests also drew me to the idea of inviting authors to examine how we might study those algorithms and rethink our research methods as we try to account for these new objects of study.”
Zamith also underscored that studying algorithms is not just a futuristic topic, but very much part of the present. “We have search and discovery algorithms that have had great impact on what journalism gets seen, and we have algorithms that enable entirely new forms of journalism to be seen,” Zamith said. “We already see thousands of news stories written by algorithms that get distributed under major media brands, and journalists must now fight back against an onslaught of disinformation perpetrated by algorithms that are employed toward opportunistic ends.” Their use will only become more pervasive in all fields, but particularly that of journalism, thus highlighting the critical nature of their academic study.
Automating Sport Coverage
In “Automation in Sports Reporting: Strategies of Data Providers, Software Providers, and Media Outlets,” Jessica Kunert focuses on the role of algorithms in German sports coverage, particularly that of football (or what is known as “soccer” in the U.S.). Sports coverage is considered a good candidate for algorithmic reporting, with standard formats for game articles (teams, scores) and a slew of data available about the players, teams and matches. She conducted interviews with sports data providers, software providers, and media outlets regarding their automation activities. The study focused on the production of automated news, as opposed to audience perceptions of the product.
Kunert identified four major themes. The first was “Actors: Working Together,” in which data and software providers work closely with newsrooms, but have a clear delineation from editorial responsibilities. They work in “data handling, the training of journalists and newsrooms, and software development” (Kunert, 2020, p. 9). The second theme, “Content: Additional Coverage,” demonstrated how automation increased the quantity of coverage of underreported areas, like amateur football, that could lead to increased audience satisfaction.
The third theme was that of “Money: Football Only,” identifying that football is the only sport that would derive profit from such automation activities. The final theme was “Quality: The Art of (Human) Sports Reporting.” This is the theme in which respondents articulated the need for human intervention to assure the quality of automated products. “While media outlets generally appreciate the increase in the quantity of articles, some have quality concerns regarding their formulaic structure and language, as well as incomplete con- tent due to limited data availability” (Kunert, 2020, p. 11). These attitudes serve to reinforce the belief in an important role for humans in the future. This theme also addressed the limitations of data, including the availability of structured data and the inability for data to convey the emotional quality of sport.
My Issues with "Interloper" Framing
The study uses language to identify non-journalistic actors from previous research that refers to them as “strangers” and “interlopers” (Belair-Gagnon & Holton, 2018; Eldridge, 2019 ) making the distinction of “implicit interlopers” (those performing vital tasks introducing innovation to the journalistic process) and “explicit interlopers” (outside platforms that challenge journalistic authority) (Kunert, 2020, p. 6). Personally, this is an area in which I have issues with this categorization. We address this conflict of framing somewhat in the invited forum on "Product Management in Journalism and Academia" in Journalism and Mass Communication Quarterly (Royal, et el., 2020). By Merriam-Webster definition, an interloper is “one that intrudes in a place or sphere of activity; an illegal or unlicensed trader.” By framing these participants as “interlopers” and “outsiders,” it marginalizes their contribution and reinforces traditional newsroom power dynamics. I find it unhelpful to the acceptance of their role in media work as these functions move more central to the mission and counterproductive to the purpose of driving innovation. Granted, there is value in recognizing the fresh perspective outsiders bring to innovation work, but I feel there are improved frames in which they can be described. These software and data providers and internal technology personnel are meeting a need for the organizations in which they become involved. More appropriate descriptors might include “consultant,” “value-added resources,” “helpers,” "partners," or other ways consistent with the “assistance” theme from another piece in the special issue (Schapals & Porlezza, 2020) to better reflect the collaborative and assistive nature of the stakeholder relationship that is not captured in the “interloper” phrasing.
Kunert’s study specifically identifies that in most cases, these “implicit” resources are welcome and invited into newsroom processes, thus bringing into question the use of the term “interloper.” The article concludes with the vital nature of these automation providers, and that media outlets don’t feel threatened or in competition with them. But highlighting the shifting power dynamic, there was a fear identified that these implicit resources might at some point turn into “explicit” providers. For example, Fussball.de, run by the German Football Association, provides amateur football reporting, claiming they “don’t call [their coverage] journalism, but rather text generation according to data” (Kunert, 2020, p. 13). This is similar to platforms like Facebook and Twitter denying their editorial responsibility, but these nuances may be less discrete in the future.
“Folk Theories” About Social Media Algorithms
In the piece “Negotiated Autonomy: The Role of Social Media Algorithms in Editorial Decision Making,” Chelsea Peterson-Salahuddin and Nicholas Diakopoulos of Northwestern University studied the attitudes journalists and editors hold toward social media algorithms and the effects these attitudes had on workflows, routines and processes. Diakopoulos has particular experience in this arena, as the author of Automating the News: How Algorithms are Rewriting the Media (Diakopolous, 2019), which was recently honored with the Tankard Book Award at the 2020 conference for the Association for Education in Journalism and Mass Communication.
Using semi-structured interviews with 18 U.S.-based professionals, mostly social media and audience engagement editors and other digital producers, the study identified new considerations for gatekeeping and news selection practices. Their findings suggest that “journalists understand social media distribution algorithms as filters that decide whether or not their audiences see their content based on a variety of factors, including but not limited to engagement or engageability of content, publisher size, payment, and political ideology” (Peterson-Salahuddin & Diakopolous, p. 28). Social media platforms have turned the traditional role of gatekeeping -- the decision-making of editors and key media players to what audiences are exposed -- to a more complex and often opaque understanding of the rules under which content is now disseminated -- the algorithms used. Through experience, both personal and professional, journalists are trying to “guess” how a story might perform on social media. This guessing has led to what is termed in this study as “folk theories” on how they should most efficiently operate on social platforms (Peterson-Salahuddin & Diakopolous, p. 27). There is a lot of experimentation, but there is also a preoccupation with clicks and page-view analytics that can skew content toward the salacious or otherwise provocative.
“Overwhelmingly, interviewees understood social media distribution algorithms as filters that did or did not allow audiences to be exposed to their content” (Peterson-Salahuddin & Diakopolous, p. 30). However, respondents differed on their impression of the effects of engagement, and to a lesser extent, the effects of publisher attributes and specific social media platform. One respondent thought “the algorithm measured all the various facets of engagement, such as liking, sharing, or commenting ‘coming up with some kind of a score for the likelihood that you’ll like some- thing similar.’” But another respondent said that “at different times the algorithm may favor one form of engagement over others" (Peterson-Salahuddin & Diakopolous, p. 30).
Respondents were mixed on whether social media algorithms influenced the editorial process. Several indicated it had no bearing. On the other end of the spectrum, a few had been explicitly instructed to not cover a topic due to its lack of performance on social. But the majority expressed some type of general influence in the middle. “For instance, in some newsrooms, guidelines for social media platform usage issued to news organizations became a factor in the editorial process. In a few instances, interviewees mentioned that these guidelines were re-inscribed into their own newsroom’s editorial guidelines” (Peterson-Salahuddin & Diakopolous, p.33).
Content framing was also used to deal with social media algorithms. One respondent noted, “I think a good story across platforms is a good story. I think that the way you present the story...that’s what changes” (Peterson-Salahuddin & Diakopolous, p. 33). Tweaking headlines and being purposeful with visuals were also referenced. But even so, most respondents felt that there were limits to what they would do to change a story’s frame to meet the expectations of an algorithm. “Platforms are never telling you what to do week to week. It’s more how the algorithms work....We tend to stay guided very much by editorial principles. So, we’re trying to grow, we’re trying to optimize, we’re trying to find ways to engage...but not at the expense of our editorial identity” (Peterson-Salahuddin & Diakopolous, p. 34).
The study concludes with the recognition that catering to social media algorithms is giving way to other metrics to which media workers may have more control and exposure, like search engine optimization and content aggregators.
Freeing up the Human Element
In "Assistance or Resistance: Evaluating the Intersection of Automated Journalism and Journalistic Role Conceptions," Aljosha Karim Schapals and Colin Porlezza further address journalistic roles, how journalists interact with technologies and the agency in which they do so. The authors studied the extent to which journalistic roles are challenged or advanced as a result of automated journalism, with findings supporting the latter, more assistance than resistance. “Thanks to technological innovations such as automated journalism, these relationships find themselves in a state of flux, leading to hybrid arenas in which novel technologies are interwoven within long-held newsroom values and routines” (Schapals & Porlezza, p. 17).
Interviewing ten journalists or editors from four German media organizations, the authors hypothesized that editors in managerial roles would feel positive about automation’s reduced cost, while those in reporter roles would have negative impressions, based on job preservation. However, their results did not support this hypothesis. Reporters expressed that technology could advance their roles. “Both editors and journalists were upbeat about the opportunities automated journalism could bring with itself -- first and foremost, its ability to free themselves from the daily grind of purely factual reporting and to instead devote their resources to profound, in-depth investigations requiring the skills that human journalists embody” (Schapals & Porlezza, p. 23).
Another result of this study of attitudes toward digitization was that journalists’ articulation of the role of journalism emphasized the human, creative aspect. “Journalism as a creative process; journalism as a uniquely individual craft; as well as the need to add background and context in order for recipients to contextualise information accordingly. They also referenced journalism’s core ideals of public service and autonomy and continued to position themselves as authoritative actors in the space of automated journalism” (Schapals & Porlezza, p. 24).
Bias, Credibility, Quality and Readability
In “Automated Journalism as a Source of and a Diagnostic Device for Bias in Reporting” Leo Leppänen, Hanna Tuulonen and Stefanie Sirén-Heikel uncover the myth that automation might produce more unbiased, fair and objective journalism. This descriptive piece explored a range of potential biases in content selection and news language, as well as mechanisms for bias in rule-based and machine-learning systems. The study provides helpful ways to detect bias in various scenarios: full transparency, cooperative operator in a black-box system and output only. They conclude with the observation that “while the mechanisms require an underlying human source for the bias, the biases can emerge in the system without human intention and in very subtle manners” (Leppänen, Tuulonen, Sirén-Heikel, 2020, p. 45).
In the only quantitative piece of the issue, “Automated Journalism: A Meta-Analysis of Readers’ Perceptions of Human-Written in Comparison to Automated News,” Andreas Graefe and Nina Bohlken summarized the results of 12 studies with a total of 4,473 participants. Of the 12 studies, credibility was measured most often (nine times), followed by quality (eight times) and readability (five times) (Graefe & Bohlken, 2020). Their results showed no difference in perceived credibility between human-written and automated news, a small advantage in perceived quality for human-written news, but a large advantage for human-written news with respect to readability. These results offer an opportunity for readability improvement in these systems as the technology matures. They conclude with the recommendation for future researchers to study the more specific factors driving these perceptions.
Final Thoughts
Zamith feels that researchers have only scratched the surface of studying these topics and sees a fertile field of study under development. “I’d love to see further work on how algorithmic actants are being used -- or can be used -- to engage in activities besides newswriting and distribution, such as interviewing sources, organizing knowledge and verifying claims,” Zamith said. “I would also love to see work outside of North America and Europe, which is something that we also struggled with in this special issue.” In addition, Zamith emphasized the difficulty in auditing algorithms. “This is something that is immensely difficult to do given their typically proprietary and black-boxed nature, but is important work.”
This special issue on algorithms and automated journalism provides a comprehensive view of the issues associated with the topic and offers unique theoretical and methodological approaches that may be applied to these concepts. If your research agenda includes algorithms or machine learning, this special issue is a must-read, but it is also relevant for all media educators and scholars to expand their understanding of this important, emerging topic. It is particularly relevant, however, for professionals to improve their understanding of the technical, social and ethical issues associated with algorithms and automated journalism, as they make decisions about their application within their organizations.
References
Belair-Gagnon, V., & Holton, A. E. (2018). Strangers to the game? Interlopers, intralopers, and shifting news production. Media and Communication, 6(4).
Diakopoulos, N. (2019). Automating the news: How algorithms are rewriting the media. Harvard University Press.
Eldridge, S. A., II. (2019). Where do we draw the line? Interlopers, (ant)agonists, and an unbounded journalistic field. Media and Communication, 7(4).
Graefe, A., & Bohlken, N. (2020). Automated Journalism: A Meta-Analysis of Readers’ Perceptions of Human-Written in Comparison to Automated News. Media and Communication, 8(3).
Kunert, J. (2020). Automation in sports reporting: Strategies of data providers, software providers, and media outlets. Media and Communication, 8(3).
Leppänen, L., Tuulonen, H., & Sirén-Heikel, S. (2020). Automated Journalism as a Source of and a Diagnostic Device for Bias in Reporting. Media and Communication.
Royal, C., Bright, A., Pellizzaro, K., Belair-Gagnon, V., Holton, A., Vincent, S., Heider, D., Zielina, A., Kiesow, D. (2020). Product Management in Journalism and Academia. Journalism and Mass Communication Quarterly, Vol. 97(3).
Schapals, A. K., & Porlezza, C. (2020). Assistance or resistance? Evaluating the intersection of automated journalism and journalistic role conceptions. Media and Communication, 8(3).
Zamith, R., & Haim, M. (2020). Algorithmic Actants in Practice, Theory, and Method. Media and Communication, 8(3).