
To measure the ROI of an AI automation, track one business outcome per hour of work (tickets resolved, invoices processed, quotes sent) before and after launch, taken from your records rather than from how the work feels. Subtract the time people spend checking and fixing the AI's output, put a price on its mistakes, and divide the net gain by everything the automation costs to run.
Most guides stop at "hours saved times hourly rate". The formula is fine. The hours are the problem: people misjudge how much time a tool saves them, review time rarely makes it into the sum, and a plain before-and-after comparison also picks up everything else that changed in the business during those months.
What does the return of an AI automation consist of?
The return is the value of what changed, minus what the automation costs, divided by that cost. What changed is usually net time saved, more work finished per hour, fewer errors or faster replies to customers. Cost is everything: the subscription, usage fees, setup hours and the time someone spends keeping it working.
Written out, ROI = (value gained minus total cost) divided by total cost. The tricky part is "value gained". Saved time only turns into money when it goes somewhere: the same team handles more orders, a hire you were planning becomes unnecessary, or quotes go out the same day and win more deals. If the freed hour quietly disappears into the rest of the week, the books won't show it. Decide before launch where the time is supposed to go, and measure that.
That decision also tells you which task to automate first. Picking the right one is the starting point of any AI integration and automation project, and it comes before choosing a tool.
Why can't you trust how much time people say AI saves?
Because people misjudge it, sometimes in the wrong direction. In a 2025 randomized trial by METR, 16 experienced developers expected AI tools to make them 24% faster. With the tools they took 19% longer, and afterwards they still believed AI had sped them up by 20%. Take the time from records instead.
The trial covered 246 real tasks, and METR is careful about what it shows. These were highly skilled people working on large projects they knew well, with early-2025 tools. In February 2026 METR called the result historical, said developers are probably sped up more by newer tools, and admitted its new data was too affected by who agreed to take part to say by how much.
None of that rescues the staff survey. The developers felt faster while the clock said slower, a gap of almost 40 percentage points between belief and measurement. If your ROI case rests on asking the team "how much time does this save you?", you are measuring the feeling. Timestamps are boring and far more honest: when a ticket was opened and closed, when an email was sent, when a lead moved to the next stage in the CRM.
How to measure the ROI of an AI automation, step by step
Pick one outcome the automation should move, record it for a few weeks before launch, then keep recording after. Count review and correction time as part of the job, price the mistakes the automation makes, and compare against similar work done without it over the same period. Each step fits in an ordinary spreadsheet.
1. Choose one outcome per hour of work
Usage numbers are tempting because the tools report them for free: drafts generated, messages the bot sent, how many staff logged in. They tell you the thing is being used. Pick what the business is actually paid for and divide it by hours worked: tickets resolved per hour, invoices processed per day, quotes sent per salesperson per week. The NBER field study of generative AI in customer support, cited below, used this kind of measure: issues resolved per hour.
2. Record a baseline from your systems
Pull the numbers for a few weeks before launch from whatever already logs the work. If nothing logs it, ask the person doing the task to note start and end times for two weeks. It's tedious, and it's the only number you will trust later. Write down the volume too. A task done a handful of times a month rarely pays back a monthly fee, however clever the tool.
3. Count net time, review included
Say a customer reply used to take 10 minutes to write. The AI drafts it in seconds, but reading and editing the draft takes 4 minutes, and one draft in ten is wrong enough that someone rewrites it from scratch: 4 minutes lost plus the original 10. The average reply now takes 5 minutes. You saved half of what the "10 minutes saved" headline promised. Add the cases the automation hands back to a person, and time spent on prompts or settings, and the number shrinks again.
4. Put a price on mistakes
An AI step that quotes the wrong price, books the wrong slot or routes a hot lead to the wrong inbox costs money the time sheet never shows. Check a fixed random sample of outputs every week, say twenty, and log each error with what it cost to fix: a refund, an hour of someone's time, a customer who didn't come back. The other side counts too. If the old manual process made errors, the automation gets credit for fewer of them, but only if you counted errors in the baseline.
5. Compare with work done without it
A before-and-after comparison also measures the season, the new hire and last month's price change. A fairer test runs the automation on part of the work while the rest goes on the old way: alternate weeks, half of the incoming requests, or one team member with it and one without. Small samples are noisy, so treat a difference of a few percent as a tie.
What does a realistic result look like?
Smaller and more uneven than vendor pages suggest. In an NBER field study of generative AI at work, 5,179 customer support agents resolved 14% more issues per hour with an AI assistant. Novice and low-skilled agents improved by 34%, while the most experienced barely changed. An average can hide who actually gains.
The details are worth a second look. According to the NBER summary, agents with the tool spent about 9% less time per chat and handled about 14% more chats per hour, yet they resolved only about 1.3% more chats overall. Speed moved a lot. The success rate barely did. Two side effects were worth money as well: agents with two months on the job performed like six-month agents without the tool, and attrition among agents with access was 8.6% lower. Hiring and training a replacement is a real cost, so a lower turnover rate belongs in the return.
For a small business, that means measuring each person as well as the team. If your staff are experienced, expect less. If you take on new people often, the value may show up as shorter training. And if the automation is an agent that acts on its own instead of an assistant that drafts, mistakes cost more and step 4 matters more; this overview of what AI agents can and cannot do today covers where they are reliable.
Which costs do people forget?
The subscription is the cost everyone counts. The ones that get missed are usage fees that grow with volume, the hours spent setting it up, the review time from step 3, the cost of mistakes, and the time someone spends repairing the automation when an input changes. Over a year, those can outweigh the licence.
Repairs are the least visible. An automation that worked in its first month breaks when a supplier changes the layout of its emails, a form gains a new field, or the tool updates its model and the answers shift in tone. Someone has to notice and fix it, and that person's hours go in the cost column. Chatbots carry a version of this too: their answers are only as current as the documents behind them, which is why training an AI chatbot on your business documents includes keeping those documents up to date.
When should you decide whether it pays off?
Set the review date and the bar before launch, while nobody is attached to the result. Judge after at least one full business cycle, such as a month with a month-end close, because the first weeks mix in setup and learning. If the net is still negative then, switch it off or cut it back.
At onlyspell, AI and automation work is the Visionary Spell. If you want a second pair of eyes on the numbers before you buy or build, talk to us about the automation you have in mind.
Frequently asked questions
What is a good ROI for an AI automation?
There is no universal number. A good result is a positive net return after every cost, review time and mistakes included, within the period you set before launch. Compare it with the cheapest alternative too, such as a template, a simple rule-based automation or leaving the task as it is.
How long does it take to measure the ROI of an AI automation?
Plan on a baseline of a few weeks before launch and a trial of at least one full business cycle after it. The first weeks include setup and people learning the tool, so judging on them alone usually understates or misreads the result.
Does saved time count if nobody's workload changes?
Saved time only shows up on the books when it goes somewhere: more work handled by the same people, a hire you no longer need or faster replies that win deals. If the freed hours simply disappear into the week, record them as spare capacity and decide what to do with it before calling the automation a success.
Sources
- Erik Brynjolfsson, Danielle Li and Lindsey R. Raymond, "Generative AI at Work", NBER Working Paper 31161
- NBER Digest, "Measuring the Productivity Impact of Generative AI", June 2023
- METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity", July 10, 2025
- METR, update on the developer productivity study, February 24, 2026
If you still need additional help in developing and designing your website, our magic is here to help. Contact us today, and our wizards will help you immediately with your digital marketing needs!


