- Paste content
- Review references
- Export
Resolves each identifier and compares what comes back with what the document supplied.
Check LLM reference data before it reaches your CMS
The LLM Reference Auditor inspects reference data produced as part of an AI-assisted publishing workflow so problems can be found before the content is published. Formerly Portable Reference Parser.
Paste the output
Add content or reference data produced by an LLM.
Audit the references
CiteKit parses the reference information and checks whether the data needed for publishing can be recovered.
Review the problems
Find missing, malformed or incomplete reference records before they enter the CMS.
What can the LLM Reference Auditor inspect?
Depending on the reference payload, the auditor can inspect data such as:
- Source identifiers
- URLs
- Titles
- Authors or creators
- Publication dates
- Source types
- Required reference fields
- Malformed reference data
- Missing values
- Whether a usable reference record can be reconstructed
A structurally valid reference does not automatically prove that the source exists or supports a claim.
Good prose does not guarantee good source data
An LLM can produce readable content while still returning incomplete or malformed references.
Common problems can include:
- Missing URLs
- Incomplete author information
- Invalid identifiers
- Malformed payloads
- Inconsistent fields
- Source information that cannot be reconstructed cleanly
The audit step sits between generation and publication.
Put a reference check between the LLM and the CMS
A reference-aware publishing workflow can look like:
- LLM
- Reference audit
- CMS
- Published page
Instead of reducing citations to fixed text inside the generated article, the workflow can carry source information forward and reconstruct the reference when the content reaches the publishing system.
What is a portable reference?
A portable reference carries enough source information for another system to reconstruct the corresponding reference record.
That means source data can travel with generated content instead of being reduced to a citation string that has to be parsed or re-entered later.
In a CiteKit workflow, compatible reference payloads can be reconstructed as Reference Library records when content reaches WordPress.
Parsing is only one part of the job
Portable Reference Parser
The original name, Portable Reference Parser, described the mechanical step of decoding reference data.
LLM Reference Auditor
The LLM Reference Auditor describes the larger publishing job: checking whether the reference data coming out of an LLM is usable before the content moves downstream.
A valid reference payload is not the same as a verified source
A successful audit does not automatically mean:
- the source exists
- every metadata field is correct
- the source supports the associated claim
- the source is authoritative
Those require additional source verification where appropriate.
The auditor checks whether the reference data itself is usable in the publishing workflow.
Build reference-aware LLM publishing workflows
Audit before publication
Catch reference-data problems before the CMS receives the content.
Keep source data separate from formatting
Carry the source information forward rather than freezing one citation string into the LLM output.
Reconstruct references in WordPress
Compatible CiteKit reference data can become Reference Library records during publication.
Generate multiple outputs later
Once reconstructed, the same source record can feed citations, bibliographies and citation metadata.
Put reference checking into the pipeline
Use the auditor as a checkpoint between content generation and publishing.
LLM Reference Auditor FAQ
What is the LLM Reference Auditor?
It is a browser-based tool for inspecting reference information produced as part of an LLM-assisted publishing workflow.
Does it detect hallucinated sources?
It can identify problems in the reference data it receives, but a valid reference payload does not by itself prove that the source exists or supports a claim.
What is a portable reference?
A portable reference contains enough source information for another system to reconstruct the corresponding reference record.
Do I need WordPress to use it?
No. The audit step can be used independently. WordPress becomes relevant when CiteKit Reference Manager is used to reconstruct and publish the references.
Can it be used in an automated publishing workflow?
Yes. It is designed to sit between LLM-generated content and a CMS or other publishing system.
What happens after a reference passes the audit?
The source data can continue into the publishing system and be reconstructed as a reference record for use in the published content.
Already using the LLM Reference Auditor and want the technical detail? Read the LLM Reference Auditor documentation for supported payloads, audit results and integration into an LLM-to-CMS workflow.