DeepSeek V4.1 Flash Tops a Bookkeeping Test, Then Changes Its Answers
A budget AI model beat a pricier rival on 40 bookkeeping checks for under a cent a run, then gave different answers when rerun. Here's why that matters.
A comparison making the rounds among accountants pits a budget AI model against the newest ChatGPT release on the same 40 bookkeeping checks. DeepSeek V4.1 Flash scored higher, and each run cost less than a cent. It also gave a different answer the second time it ran. That last detail matters more than the score, and it is the part most teams will miss.
The scores behind the claim
On the same 40 checks, DeepSeek V4.1 Flash answered 34 correctly on one run and 33 on another. The newest ChatGPT model managed 30. The budget model also produced one miss where the pricier one produced none. Raw accuracy favors the cheap model; reliability does not.
Under a cent changes the math
A penny per run sounds trivial until you multiply it. Push a full year of transactions through a review pass and even modest per-run fees pile up, while a sub-cent model stays near rounding error. That price gap is why these tests get attention, and it is a legitimate reason to evaluate the budget option. It is not a reason to adopt it.
Same prompt, different answer
These models generate text one piece at a time, and small sampling differences can steer a borderline check down a different path on each pass. Easy calls stay stable. Marginal ones flip. In this comparison the drift was one or two answers out of 40. Small, but it repeats.
A model that disagrees with itself has a specific cost in bookkeeping: you cannot file its output as reviewed. If the answer you verified on Monday differs from the one you act on Wednesday, your check expired the moment you closed the window.
Should you switch your checks to a cheap model?
Not on one scorecard. A 40-check suite is a useful smoke test, not proof, and prompt wording plus routing settings move results more than most people expect. Our read of this comparison is narrower: the budget model cleared a real bar once and did not clear it identically twice. Both findings are true, and the second one is the riskier discovery.
Test a model three times before it touches your ledger
Build a check set from your own closed periods: 40 questions about dates, categories, duplicates, and amounts you already know the answers to. Run the full suite twice on identical settings, then again after any prompt or model change. Compare answers line by line, not just the totals. Two agreeing runs build trust; a flip shows you exactly which checks are too close to automate.
Log the model version with every output. Providers swap models behind the scenes, and a workflow that behaved last month can drift without anything changing on your side. Treat every answer as a draft, and let nothing reach the ledger until a person signs off.
If you want a starting point, pull last quarter’s reconciled books, write your own 40 questions tonight, and run your current model twice before you change anything. The comparison that matters is not DeepSeek against ChatGPT. It is either model against your own ledger.