Everyone has heard the pitch: upload your invoices and carrier agreements into AI and let it tell you what you're owed.
We wanted to test that claim. So we hired an independent AI engineering firm and gave them four weeks, a frontier large language model, 52,289 real FedEx charge lines across 8,000 shipments, published rate tables, nearly 5,000 negotiated agreement terms, and one thing DIY teams don't have: Reveel's verified audit results for every charge line.
They ran two tests. The second results matter more.
Test 1: Ask AI to audit invoices directly.
The model received invoice data, rate tables, surcharge schedules, and negotiated terms, then calculated the correct net charge.
Only 47% of charge lines were within a penny of the correct answer. The average error was $7.80 per line, creating more than $407,000 in discrepancies across the file. More concerning, the model frequently missed overcharges while confidently confirming incorrect invoices.
Prompting AI alone simply wasn't reliable.
Test 2: Use AI to build deterministic audit software.
The second approach used the model as a coding assistant to build deterministic audit software: real code that parses invoices, references published rates and applies negotiated discounts in sequence. It worked. Each rule layer sharply reduced errors: $8.34 average error with base discounts, $1.19 after earned-discount logic, and $0.02 after modeling minimum-net floors. The finished engine achieved 99.5% accuracy within a penny and 99.94% on surcharges.
That's an impressive result, but the study uncovered something even more important.
AI Can Build the Engine. It Can't Verify the Answer.
Every accuracy improvement depended on validating results against Reveel's verified audit output. Complex pricing logic wasn't discovered by AI alone — it was reverse-engineered by comparing results to known correct answers. When validation wasn't available, errors persisted unnoticed, including misapplied surcharge discounts that confidently produced incorrect invoices.
AI didn't fail because it was weak. It failed because it had no ground truth.
Where AI Actually Belongs
The study reinforced the approach behind Reveel IQ.
Subject matter experts train the AI. AI extracts discounts, tiers, minimums, and caps into structured data. A deterministic rules engine, runs every rate lookup and discount stack against carrier invoices, in the correct sequence, with a citation trail for every result. And our data and our people validate the output against pricing logic proven across thousands of carrier agreements.
The result is AI-powered answers backed by verified ground truth.
The takeaway is simple: AI is an exceptional tool, but it still needs verified truth. We tested the "just use AI" approach. The real advantage isn't AI alone — it's AI built on expertise that's already been proven.
Read the full report at reveelgroup.com/resources/ai-study and learn more about Reveel IQ at reveelgroup.com/iq.
Jack McCrum leads Reveel’s Optimization and Analytics team, where he helps shippers interpret carrier pricing behavior and control transportation costs through data driven modeling. His work focuses on package level simulation, accessorial forecasting, and identifying strategic levers that improve performance for finance, operations, and logistics leaders. Jack regularly contributes insights to industry publications and webinars, offering practical guidance that helps organizations anticipate change, strengthen parcel strategy, and make confident decisions backed by measurable results.
