Automation vs AI automation: when a rule is enough

A lot of what is sold as AI automation is plumbing with a model glued to the side. That is not always wrong — sometimes the model is the useful part — but it is worth knowing which is which before you pay for either. The distinction is simple, and it decides how a system gets built, tested, and maintained.

Plain automation: when this happens, do that

Plain automation follows rules. A form is submitted, so a record is created and a confirmation goes out. An invoice is approved, so it moves to the payment queue. It is a weekday at eight in the morning, so the daily job list is assembled from the calendar. The rule can be written down completely, and the system does exactly the same thing every time.

This kind of automation is cheap, predictable, easy to test, and easy to explain to the person whose job it changes. Most of the repetitive work in a small business — the copying, the reminding, the routing, the filing — is this kind.

AI-assisted automation: when the input is messy

AI earns its place where the input is unstructured and a person currently reads and judges. An email that could be a complaint, a question, or a sales inquiry. A scanned document where the fields are in different places on every page. A pile of call notes that need to become a summary. A first draft of a reply, a proposal, or a report.

Here the system is not following a rule; it is making a judgment that a person would otherwise make, and it will sometimes make it wrong. That is why AI-assisted steps usually need a review point, at least until the error rate is known.

The test

Try to write the rule down completely. If you can — every condition, every outcome, no exceptions that depend on reading something — you want plain automation. If the rule keeps needing words like "usually," "depends," or "you can tell when," the task involves judgment, and that is where a model belongs.

Most real tasks are a mix, and the useful skill is separating the two parts rather than choosing one label for the whole thing.

Why AI where a rule would do is worse

• It is not deterministic. The same input can produce a slightly different output on a different day, which is fine for a draft and unacceptable for a routing decision.
• It costs more to run and more to maintain.
• It is harder to test. A rule is tested with cases; a model is tested with an evaluation set and a tolerance, which is more work.
• It is harder to explain. "The system routed it there because the form said X" is an answer. "The model decided" is not.

Why a rule where judgment is needed is also worse

The opposite mistake is just as common. A rule-based system that tries to classify free-text email by keyword will be brittle, exceptions will pile up, and within a month people will route around it. When the input genuinely needs reading, forcing it through a rule produces an automation that everyone quietly ignores.

The pattern that usually works: rules around, AI in the middle

Take an inbox that receives service requests. A plain rule receives the message and creates a record. A model reads the message, classifies it, and extracts the details a person would otherwise type. Plain rules then route it by type and urgency and send the acknowledgement. A person reviews anything the model flagged as uncertain or anything classified as high-risk before it moves on.

The model does the one part that needs reading. Everything around it is deterministic, testable, and cheap. That structure is what most good AI workflow automation looks like once you take the marketing off it.

Testing each kind

Rules are tested with cases: here is the input, here is the required output, run them all, they must all pass. AI steps are tested with an evaluation set: a collection of real inputs with known-good answers, run through the model, with the results compared and the error rate recorded. When the underlying model changes — and it will — you re-run the set and see whether anything regressed, instead of finding out from a customer.

If a vendor cannot describe the evaluation set, they have not tested the AI part.

What to ask a vendor

• Which steps in this are rules, and which use a model?
• For the model steps, what does a wrong answer look like, and who catches it?
• What is in the evaluation set, and can we see the results?
• What happens when the model provider changes the model?
• Could any of the model steps be a rule instead? Why not?