The IDL Index is a human-based judgment metric for assessing the factuality of machine-generated content. It is a language-agnostic tool designed to evaluate content generated by large language model systems in the context of academic research. To what extent do generative AI tools adapt to facts or create “artificial hallucinations”?

The Information Disorder Level (IDL) index is a metric designed to assess the factuality of machine-generated content. Tested on a mix of made-up and real news stories using ChatGPT, the results of this experimental research emphasise the critical role of human judgment.

The ethical use of data-driven systems in journalism can be approached from the perspective of news gathering and news production. They also relate to news distribution, with news recommenders and news personalisation systems. It refers to two distinct periods in the use of AI-based systems in journalism: before and after ChatGPT.

Generative AI, through large language models, has become a cheap and quick method to generate misleading or fake stories. However, producing false or inaccurate results is not always intentional, as machine-generated contents are subject to “artificial hallucinations”. Therefore, defining the (non)human nature of the author seems pointless, especially since detection methods still remain limited. A different perspective is grounded in the tradition of human judgement methods that have been developed in natural language processing (NLP) to assess the qualitative characteristics of machine-generated content.

For decades, journalists have had ambiguous relationships with technological innovations, perceived as promises for a brighter future in journalism, threats to professional authority and identity, or tools to augment professional practices. The growing spread of artificial intelligence systems within newsrooms adds new layers to the debates surrounding the disruptive nature of technology. How does it shape or affect the journalistic meta-discourses and representations?

News automation can be used as a final product that will be delivered to the audiences without any journalistic mediation or as a first draft that journalists will enrich with their expertise. These recommendations aim to provide guidelines to promote ethical practices, considering that news information is a public good that commits all stakeholders to the audiences.