Plato's Cave: A Human-Centered Research Verification System
AAAI MAKE 2026, 2026 · University of Florida
Matheus Kunzler Maldaner
Raul Valle
Junsung Kim
Tonuka Sultan
Pranav Bhargava
Matthew Maloni
John Courtney
Hoang Nguyen
Aamogh Sawant
Kristian O'Connor
Stephen Wormald
Damon L. Woodard
What is Plato's Cave?
Plato's Cave is an open-source, human-centered research verification system that converts a paper into a directed acyclic graph of claims, sends web agents to gather evidence for each node, and computes an overall integrity score. A twelve-person University of Florida team built it, and the paper appears at AAAI MAKE 2026.
Plato's Cave started as a weekend project. My team and I won first place overall at Gator Hack IV, the University of Florida's AI Days hackathon, with a prototype that used AI agents to check the claims inside academic papers. Over the months that followed, twelve of us at UF, including Stephen Wormald and Dr. Damon L. Woodard, turned that prototype into a full system and a first-author paper at AAAI MAKE 2026.
The motivation is simple. Papers are published faster than anyone can carefully read them, and reviewers rarely have time to chase down every citation. We wanted a tool that does the tedious part, gathering evidence claim by claim, while a human keeps control of the final judgment. The reviewer stays in the loop the whole way through.
How do you verify claims in a research paper?
Verifying a research paper means checking each claim against independent evidence. Plato's Cave automates this in three stages: it extracts the paper's argument as a graph of typed claims, dispatches web agents to score each claim against online sources, and aggregates those scores into a single paper-level integrity score.
The pipeline has three stages. First, the system reads the paper and extracts its argumentative structure as a graph of typed nodes. Second, web agents verify each node independently, searching the open web for supporting or contradicting evidence. Third, scores flow through the graph using trust-gated propagation, an approach inspired by belief propagation where each node's trust combines its own quality with the weakest link of its upstream support.
The final score also rewards structure. We identify the bridge subgraph connecting at least one Hypothesis to at least one Conclusion, find the single hypothesis-to-conclusion chain with the highest confidence, and combine graph-level components like bridge coverage, redundancy, fragility, and coherence into one integrity score. Structure matters as much as sources.
Interactive demo: one paper through the pipeline
Press play to follow a single paper through the three stages, with the real timing numbers from the evaluation run. The clock is fast-forwarded; the labels are not.
1Extract the DAG
gpt-5-mini turns the paper into a graph of typed claims. Eight candidate graphs sampled, median 43.1 s each.
2Web agents verify
Browser agents score every node against the open web. Eight scoring trials per valid graph, 53.6 s per trial.
3Aggregate the score
Trust-gated propagation plus bridge coverage, redundancy, fragility, and coherence become one integrity score.
Turning a paper into a directed acyclic graph
Plato's Cave uses gpt-5-mini to extract up to 16 nodes per paper into a directed acyclic graph, a structure where every edge points one way and no cycles exist. Each node receives one of ten roles, such as Hypothesis, Claim, Evidence, Method, or Conclusion, and edges encode logical dependency between them.
Extraction runs on gpt-5-mini with a strict JSON schema and a fixed ontology of ten roles: Hypothesis, Claim, Evidence, Method, Result, Context, Assumption, Counterevidence, Limitation, and Conclusion. We validate every candidate graph against three constraints, ontology compliance, referential integrity, and acyclicity, and when a graph fails, the system returns the errors to the model for a bounded repair pass instead of discarding the work.
The approach holds up in practice. Across 832 sampled DAGs in our evaluation, only 22 failed validation, a 2.6 percent failure rate, with unknown role labels and dangling references as the most common causes. Sixteen nodes sounds small for a full paper, and the constraint is deliberate. We sample multiple candidate graphs per paper and score each one separately, which turns a lossy summary into a distribution we can trust more than any single extraction.
Interactive demo: how trust flows through a claim graph
A toy paper on the spacing effect, extracted as a DAG. Click any node to inspect it, then run verification to watch agents score each node and trust propagate through the weakest link.
Node inspector
Click a node in the graph to see its role and the six agent scores.
Web agents that score credibility
Web agents in Plato's Cave are browser-driving LLM agents built on the browser-use library. For each node in the graph they run a search, verify, assess workflow and return six scores between 0 and 1: credibility, relevance, evidence strength, method rigor, reproducibility, and citation support.
Each agent runs inside a Docker container and follows a search, verify, assess workflow: formulate queries from the node's text and role, navigate the results, extract content, and judge what it found against six metrics. Peer-reviewed sources score higher on credibility than blogs, and nodes with available code or data score higher on reproducibility. Careful engineering of the browser sessions gave us a 3 to 5 times speedup over a naive implementation.
The human-centered part is my favorite piece. The reviewer and the verification agent share the same live browser session, so you can watch which pages the agent visits in real time and step in when needed, solving a CAPTCHA or entering university credentials to unlock a paywalled source. The agent does the legwork. The human keeps the judgment.
Results on 104 research papers
On a dataset of 104 papers spanning economics, machine learning, and psychology, each labeled Bad, Neutral, or Good, Plato's Cave reached an AUROC of 0.7594 at separating good papers from bad ones. The full run took 13.99 hours, with a median of 65.4 minutes and roughly 15 cents in API fees per paper.
For each paper we sample eight candidate DAGs and run eight scoring trials per valid graph, resampling the node-quality and propagation weights each time. That produced 810 valid DAG samples and 6,480 scoring trials across the collection. At the refine stage, the configuration we treat as our main result, the system reached an AUROC of 0.7594 and a Spearman correlation of 0.395 against the triage labels. A later sparse search stage nudged AUROC to 0.7658, but we flag it for overfitting risk and lean on the refine numbers.
I want to be honest about what this is: a pilot run meant to validate throughput, artifact quality, and failure modes. The labels are weak supervision from a spreadsheet triage, and 104 papers is modest next to large fact-checking benchmarks. The signal is real. The claim is calibrated.
Interactive demo: can you classify like the extractor?
Five sentences from a fictional paper on urban tree canopy and street heat. Assign each one the role the extractor would give it.
The real extractor classifies against a ten-role ontology (Hypothesis, Claim, Evidence, Method, Result, Context, Assumption, Counterevidence, Limitation, and Conclusion) and validates every graph for ontology compliance, referential integrity, and acyclicity. Across 832 sampled DAGs, only 22 failed validation, a 2.6 percent failure rate.
Try the demo
The Plato's Cave demo at platoscave.matheus.wiki lets anyone upload a PDF, paste a URL, or type a natural-language prompt and watch the verification happen live. The interface renders the extracted graph interactively and shows the six credibility metrics for every node on hover, with confidence breakdowns per edge.
The demo accepts a PDF upload, a URL, or a plain prompt, then streams verification progress as it happens. The graph view highlights the best hypothesis-to-conclusion path, lets you filter nodes by role or quality threshold, and exposes the full reasoning chain from evidence to conclusion, so you can see exactly which nodes are weak and why.
Everything is open source on GitHub, and there is plenty left to build. The direction I care most about is learning the trust-gate parameters and role-specific weights from human annotations, so the system sharpens as reviewers actually use it. Try it on a paper you know well and see where the graph gets it wrong.
Abstract
The growing publication rate of research papers has created an urgent need for better ways to fact-check information, assess writing quality, and identify unverifiable claims. We present Plato's Cave as an open-source, human-centered research verification system that (i) creates a directed acyclic graph (DAG) from a document, (ii) leverages web agents to assign credibility scores to nodes and edges from the DAG, and (iii) gives a final score by interpreting and evaluating the paper's argumentative structure. We report the system implementation and results on a collected dataset of 104 research papers.
Frequently Asked Questions
Why is it called Plato's Cave?
The name comes from Plato's allegory of the cave, where prisoners mistake shadows on a wall for the world itself. A paper's prose is the shadow. The evidence behind it is the world outside, and the system's job is to walk out and check. The metaphor also carries some useful humility, since no single verification pass sees everything.
Is Plato's Cave open source?
Yes, the full system lives at github.com/matheusmaldaner/PlatosCave. The frontend is built with Gatsby.js and React, the graph renders with ReactFlow using a Dagre layout, progress streams over WebSocket connections, and the web agents run in Docker containers with remote browsers controlled through the Chrome DevTools Protocol.
Can Plato's Cave detect fraudulent or retracted papers?
We frame the system as a tunable, auditable ranking scaffold rather than a definitive quality estimator. It ranks papers by how well their claims survive independent verification, which correlates with the Good and Bad triage labels in our dataset, but it issues no verdicts. A low integrity score is a reason to look closer, and the graph shows you where.
How long does each stage of the pipeline take?
Median DAG extraction takes 43.1 seconds and a single scoring trial takes 53.6 seconds. The 65-minute median per paper comes from sampling eight graphs and running eight scoring trials on each, all with real web browsing in the loop. At about 15 cents of API fees per paper, the dominant cost is time rather than money.
Who built Plato's Cave?
A twelve-person team at the University of Florida. I am the first author, alongside Raul Valle, Junsung Kim, Tonuka Sultan, Pranav Bhargava, Matthew Maloni, John Courtney, Hoang Nguyen, Aamogh Sawant, Kristian O'Connor, Stephen Wormald, and Dr. Damon L. Woodard. The project began as our winning entry at Gator Hack IV and grew into the AAAI MAKE 2026 paper.
What is next for automated research verification?
Our future work covers active learning to decide which nodes to verify next, cross-domain tests of whether a model tuned on computer science papers transfers to biology, longitudinal monitoring of preprint servers like arXiv, support for claims spanning multiple papers, and verification beyond web search, such as running a paper's code to reproduce its results.
BibTeX
@misc{maldaner2026platoscave,
title={Plato's Cave: A Human-Centered Research Verification System},
author={Matheus Kunzler Maldaner and Raul Valle and Junsung Kim and Tonuka Sultan and Pranav Bhargava and Matthew Maloni and John Courtney and Hoang Nguyen and Aamogh Sawant and Kristian O'Connor and Stephen Wormald and Damon L. Woodard},
year={2026},
eprint={2603.23526},
archivePrefix={arXiv},
primaryClass={cs.AI}
}