AI chat vs rule-based chatbots

Decision trees still have jobs worth defending. Most of what a shop gets asked is not one of them, and the reason has nothing to do with which mechanism is cleverer.

A rule-based chatbot is a flowchart somebody drew. A model-based one writes each reply from source text you gave it. That is the whole difference in one line. Decision trees still have jobs where they are the right mechanism, and those jobs deserve the argument attached rather than a shrug about use cases. Most of what a shop gets asked, though, does not arrive in the shape a flowchart expects. The axis that matters is who decides consequences. A decision tree welds understanding and permission into the same object, and the half that breaks is understanding. A language model repairs understanding and hands you back a permission problem you now have to solve on purpose.

What each one is doing when it looks like it understands

A rule-based bot is three things: buttons, keyword intents, and a fallback message. The buttons are a menu you wrote. An intent is a bucket with a name like shipping_cost and a list of example phrasings you typed in. When a message arrives, the tool scores it against every bucket and picks the closest. If nothing clears the threshold, the fallback fires and the customer sees some version of "I didn't quite get that, try one of these." Nothing there read the sentence. It matched it. When a rule-based bot appears to understand, the customer's wording was close enough to a phrasing you anticipated.

A model-based assistant is given the customer's message plus the text you supplied, meaning catalog rows and your written policies, and produces a reply from that. The reading is genuine, and you can check it on specific cases: a paraphrase lands, a question asked in a second language lands, a message containing two questions and a complaint gets sorted out. What the model is not doing is checking whether the reply it produced is one you would have approved. It is good at the reading and holds no opinion about the permission.

Two different failures, and two different things to count

A decision tree fails visibly, which is its one mercy. Take your last 200 sessions and count the ones that ended in the fallback message or in a menu loop, meaning a customer tapping three options and closing the window without an answer. If that share is under 10 percent your tree is doing its job and you should leave it alone. Between 10 and 25 percent you are losing conversations quietly. Above 25 percent the tree is not a support channel, it is a form that people fail to fill in. You can pull this number today with a text search, because the failure has a fixed string in it.

A model-based assistant fails invisibly. Its bad output is fluent, correctly formatted, in the right tone, and wrong. No text search finds it, because there is no fallback string to look for. The only measurement is reading: pull 50 transcripts and mark each factual claim as supported by your written source, contradicted by it, or not covered anywhere. The third bucket is where inventions come from. Different problem, different remedy, and the controls against it have their own page.

What it costs to add one more answer

Adding coverage to a decision tree means drawing another branch. Not just the branch: the entry points that reach it, the phrasings that route to it, the back-links so the customer is not stranded, and a decision about which existing branch it takes traffic from. Twenty minutes if the tree is small. The real cost is that every branch you drew last year is still there. Nothing expires. Adding coverage to a model-based assistant means writing one short fact, thirty words saying orders over a threshold ship free within a stated number of days, and the assistant uses it whether the customer asks about shipping, about the threshold, or about why their total changed at checkout. You wrote a fact, not a path.

Look at what a tree becomes after two years of edits. Forty or sixty branches, several written by someone who has left. Two of them quote different free-shipping thresholds, because one was updated when the policy changed and the other was reachable only from a menu nobody remembered. Three exist purely to ask "did you mean shipping or returns?" A handful are orphaned, unreachable since an entry point was rewritten, and you cannot tell which without walking the whole thing. Editing that tree is archaeology and it gets more expensive every quarter. Correcting a written fact costs the same forever: find the sentence, change it.

Where a decision tree still wins

If a sentence has to be an exact sentence, generation is the wrong mechanism. Regulated wording, statutory refund language, anything your lawyer approved word for word: you want a stored string returned unchanged, not a system that produces text which is usually a faithful rendering. "Usually" is not a standard you can defend. The advantage here is structural rather than nostalgic. A tree can only emit strings you typed, so its worst possible output is something you already approved in the wrong place. No amount of source discipline gives a model that property, because a model composes the sentence each time.

The same holds for a single fixed flow with no variation, where the customer's answer is a number or a choice from a short list and there is nothing to interpret. Ring size, delivery slot, which of four warranty options. A model there adds latency and a failure mode in exchange for nothing you needed.

The third case is the one shops get backwards. If a wrong answer costs you a refund dispute and a non-answer costs you an email, then a system that reliably says nothing beats one that usually says something right. Price the two failures before you choose the mechanism. Rules are not legacy technology there. They are the correct engineering choice, and swapping them for a model is a downgrade dressed as an upgrade.

Understanding and permission are separate jobs

A decision tree fuses two things that have no reason to be fused. When you drew the branch you decided both what the customer meant and what the bot is allowed to say about it, and you welded those decisions together in a flowchart node. That is why the tree is rigid where you want flexibility, in the reading, and no more trustworthy than your own drafting where you want rigidity, in the promises.

Split them and both halves get easier. Understanding changes constantly, because customers keep inventing new ways to phrase the same question, and it is what models are good at and rules are bad at. Permission changes rarely, because your refund window and your escalation policy hold for months, and it is what rules are good at and models have no notion of. Give each half to the mechanism suited to it.

The hybrid most shops actually want

The arrangement worth building is not one or the other. The model handles understanding: what did this person actually ask, in what language, and which of their two questions matters. Fixed configuration handles consequences: what may be promised, what must always reach a person, which topics the assistant never speaks about at all. The flexible part reads. The rigid part decides. When a customer asks for a discount that does not exist, the model understands the request perfectly and the fixed rule is what refuses it.

In practice that means answers come from your own material rather than general knowledge, and the consequential behaviors are settings you control rather than sentences the model chose. Starly is built on that split: replies are assembled from the catalog it imported and the cards you typed, while human handoff, business hours and the away message are configuration. Test the split on your own vendor by asking where a promise lives. If your assistant's escalation behavior lives in a paragraph of instructions instead of a setting, you have a model deciding consequences.

What to do with the flows you already built

Do not throw the tree away on day one, and do not port it either. Walk it once and sort every branch into three piles. Keep as hard rules the branches that encode a consequence: escalation triggers, exact approved wording, topics that must always reach a person. Those become settings. Delete the branches that exist only to route around bad understanding: the disambiguation questions, the "please rephrase" prompts, the menus you added because keyword matching kept missing. They solve a problem you no longer have, and carrying them over rebuilds the old failure inside the new tool. The third pile is content, branches whose value was the fact at the end of them. Those become written facts, and the fact is shorter than the branch that carried it, because you are no longer writing the route.

Then run both for two weeks before switching anything customer-facing. Point the new assistant at a test link, replay 50 real questions from last month's transcripts, and read the answers yourself. Gaps cluster: when the same three topics keep producing weak answers, that is three facts you never wrote down, not a model problem. Fix those, then cut over the website widget first and the channels where customers expect a person last.

When you should not do any of this

Two conditions make the whole comparison premature: too little volume for either mechanism to earn its upkeep, and policies that exist only in somebody's head. Both are readiness questions rather than mechanism questions, and they are decided elsewhere. The second is worth one line here, though, because the direction of the failure changes when you switch. A tree with nothing written behind it fails loudly and stays honest, dropping the customer into a fallback. A model with nothing written behind it fails fluently. Migrating before the writing exists does not create a gap you did not have; it converts a visible one into an invisible one.

If your catalog and your policies are the same four sentences on one page, a link beats a chatbot of either kind. And if your existing tree really does answer nine questions in ten, leave it. A tool that works is not improved by being newer. The case for changing rests on your fallback rate, and if you have not counted it you cannot yet decide.

What you give up on the way out

Something goes missing in the migration that nobody warns you about, and it is not a feature. It is your instrument. A decision tree hands you a free failure signal: the fallback is a fixed string, so the share of conversations that hit it is a text search away, and you can watch that share move week to week without instrumenting anything or asking anybody for a report. It is a crude number and it is an honest one, because the tree cannot hide a miss.

Delete the tree and the signal goes with it. There is no equivalent string in a model-based assistant, because its misses come out phrased like hits. The week after cutover your reporting gets quieter, and shops read that quiet as improvement when all that happened is the failures stopped announcing themselves. So plan the replacement before you switch rather than three months after: from here on the instrument is somebody reading conversations, and what to count while reading them is a discipline of its own. The point to carry out of this page is narrower. Do not mistake a number disappearing for the thing it was counting disappearing.

How to tell which one a vendor is selling you

One question does most of the work: what happens when a customer asks something nobody anticipated? If the answer describes a fallback message, a menu or a rephrase prompt, you are being shown a decision tree whatever the marketing page calls it. If the answer is that it will figure it out, you are being sold the risk without the mitigation. A good answer names the source the reply comes from and says what happens when that source is silent. Ask for that demonstration live, with a question you invent on the spot.

The last one is the question that catches hybrids pretending to be one thing. A vendor selling you a model will often insist that everything is generated, which means your approved wording is a suggestion. A vendor selling you a tree will often insist that nothing is generated, which means the paraphrase problem you are trying to solve is still yours. The honest answer is that both mechanisms are in the product and here is which one handles which case, and a vendor who can say that sentence has thought about the same thing you are reading this page to think about.

Common questions

How do I tell if my current chatbot is rule-based?

Type a question it cannot possibly have a branch for, phrased in a way no one would have written down, and see what comes back. If you get a fallback line like "I didn't quite get that" or a menu of options, it is rule-based: it matched your words against a list of intents and nothing scored high enough. A model-based assistant will either answer from source material it was given or say it does not have that information and offer a person.

Is a rule-based chatbot cheaper to run than an AI one?

Cheaper per message, since there is no model call. But the cost of a decision tree is maintenance, not runtime: every new question needs a new branch, and after two years you have dozens of branches nobody fully understands, including contradictory ones. Compare the totals by counting the hours someone spends editing flows each month, plus the conversations lost to your fallback rate.

Can I keep my existing decision tree and add AI on top of it?

You can, and for some branches you should. Keep the ones that encode a consequence, such as escalation triggers, legally approved wording, and topics that must always reach a person, and re-implement those as fixed settings. Delete the branches that only existed to work around bad keyword matching, such as disambiguation questions and rephrase prompts, because carrying them over rebuilds the original failure inside the new tool.

Will an AI chatbot invent answers about my shipping or returns policy?

It can, and that is the real trade you are making when you leave a decision tree. The mitigation is not a better model, it is a shorter source: an assistant whose only material is your catalog and the policies you typed, handing off to a person instead of improvising. The full set of controls against invented answers is a topic in its own right and worth reading separately.

How long should I run both before switching over?

Two weeks is enough for most shops. Run the new assistant on a test link, replay about 50 real questions from last month's transcripts, and read every reply rather than sampling. Weak answers cluster around a handful of topics, and those are almost always facts you never wrote down. Fix them, then move the website widget over first and the channels where customers expect a person last.