This knowledge article explains how authors can avoid fabricated references in their work and how teachers and supervisors can check reference lists in work they have to assess.

Preventing fabricated references

Fabricated citations are becoming increasingly prominent, both in student work such as BSc and MSc theses, and in scientific publications such as peer-reviewed journal articles and PhD theses. See, for example, this article: Fabricated citations: an audit across 2·5 million biomedical papers - The Lancet.

False or fabricated references may be considered academic fraud and can have serious consequences for students, PhD candidates, and employees. They also undermine the integrity of the scientific literature when these inaccurate references are cited by others and find their way into the scholarly record.

For authors

Reading your source and citing it correctly

It may seem obvious, but the first step in preventing incorrect or fabricated references is simple: do not create or use references that you have not read and verified yourself. Citing a source always means that you must have interacted with it. You have read the text, understood it and internalised it, and you are using it to support your own research or argument.

Before citing a source, you should always:

• Locate the full text publication. If you have difficulties in tracing the source, the Library can help you with this; see Databases and collections.
• Read the publication (or at least the relevant part of it).
• Make sure that the publication does indeed support what you want to say.

These first steps prevent you from relying on non-existent sources (for example, because you found them in a result list from an AI tool such as ChatGPT or Claude; or in a poorly made reference list from another published paper).

While citing your sources and finalising your manuscript, WUR Library advises the following:

• Use reference manager software, but remember garbage in = garbage out. Always import reference metadata from reliable sources. If you must manually input metadata, only do so for sources you know are real. Keep in mind that your reference manager software will not check for you whether a reference is real or not.
• If you use AI to improve grammar and structure in your text, make sure you exclude the reference list from that check. GenAI is notorious for also altering your reference list; it will change titles, authors, DOI, etc. In the prompt, say that it should not alter any in-text citations, and double-check that it was indeed the case.
• As an author, you should always carefully check your reference list for any errors or inaccuracies before submitting your work to your supervisor, a journal, or your teacher. This will ensure that the sources you've used are accurately put together in your reference list and you don’t end up with fabricated sources.

For teachers and supervisors

Detecting fabricated references

You can check two things when going through a reference list:

1. Whether the source does exist or not
2. Whether the claim that is made is backed up by the source or not.

In this article, we address only the first point: whether the article exists or not.

Manually checking references

You can, of course, check the reference list manually. We strongly recommend you ask your students or PhD candidates to include the DOI in their reference lists, or the source’s URL if there is no DOI available.

You can then go through the reference list, open each link and compare the provided information with the actual information.

If the DOI does not resolve or the URL does not work, you can manually search for the source in Google Scholar or Scopus, using the title, author, or journal name. This is a reliable way to verify a reference, but it can be time-consuming.

Checking with Turnitin

Turnitin, by default, highlights the entire reference list of a manuscript as if it is plagiarised text. Turnitin is looking for text similarity, so if a reference has already been included in another work in a similar style, it will match that reference and highlight it. You can use this feature to check reference lists.

You can review the results of the plagiarism check in Turnitin. While Turnitin cannot automatically verify whether a reference actually exists, the results may help you identify potentially suspicious or fabricated references that warrant further checking.

• If the reference is fully highlighted in one colour, it is likely that this reference has been used somewhere else. This doesn’t imply that it is a correct, existent or non- fabricated reference. This could still be the case, although the probability is low. A fake, made-up or incorrect reference can be highlighted by Turnitin if it is cited in a reference list from a published scientific paper.
• If the reference is highlighted in more than one colour, partly highlighted, or not highlighted at all, it can be a fabricated reference. But again, you will need to check this manually, because it could refer to grey literature, a book chapter, something very new; something published in a language other than English, something simply not in the Turnitin database, etc.

A Turnitin match only proves that the text of the reference occurs in Turnitin’s comparison corpus. It does not prove that the reference exists. The absence of a match only proves that the reference is not available in Turnitin’s comparison corpus. However, the source may still exist.

Reference checking tools

Several tools can automatically check reference lists. WUR Library doesn’t have recommendations for a specific tool. We are currently looking into various tools and will update this article when we have more concrete advice.

Below, we list three free and open-source tools you can use to check for fake references. They seem to be working well, but we can’t guarantee the results (false positive or false negative). We also expect poor results when checking poorly formatted reference lists. Note that you should never upload the full text of the manuscript to the tool. Even when the tool is run locally, some information may still be sent to the tool provider, and it is not always clear what data is shared. Therefore, only upload the reference list itself.

• Hallucinator; a tool to detect potentially hallucinated or fabricated references in academic PDF papers. Requires prior knowledge of programming.
• Paper Reference Checker is easier to use. Allows you to do a quick check for a single reference or run a batch check.
• UB-Mannheim/check-fake-references: This is a Python script developed by the Library of Mannheim University, and comes with extensive documentation.

For all these tools, the following always applies: if it can’t verify a source, you need to do a manual check. You will also need to manually check papers that don’t have a DOI.

When you can’t verify a reference

When a source can’t be verified, it can either be a fabricated source or a hard-to-find source.

The first step is to request the student/author of the work to provide you with the details of the source. You can also ask the Library’s information specialists for assistance with locating the source.

When you think you might be confronted with a fabricated reference in a BSc or MSc student’s work, you should follow the guidelines of the Examining Board.  See this knowledge article for more information. For WU doctoral theses, follow these Rules and guidelines.

More information and support

PhD’s and post-docs are welcome to join the course Searching & organising literature, or the course on Scientific publishing. BSc and MSc students can join the free workshops on Citing & Referencing. You can also contact the Library if you need assistance with locating a difficult-to-find source. The Copyright Information Point can help you with questions about plagiarism.