How to Tell If a Vendor's AI Is Real or a Marketing Claim

There are products in market right now where the “AI” is a set of if-else statements, a keyword matching function, and a chat bubble UI. There is no model. There is no training data. There is no inference. The “AI” label was added to the marketing materials in 2023 and the underlying system didn't change. The danger isn't just that you overpay for a product that doesn't deliver what you expect — it's that you build a roadmap around a capability that doesn't actually exist and only discover the gap when you try to push the system past what its rules were written to handle.

The vendor isn't necessarily lying to you. In many cases they genuinely believe they have AI because someone on their team used the word once in a Slack message and it stuck. In other cases they're deliberately vague because specificity would reveal limitations. Either way, the technical reality is the same: a rules engine with a chatbot wrapper cannot learn from new data, cannot generalize to inputs it wasn't explicitly programmed for, and cannot improve over time in any meaningful sense. If your use case depends on those capabilities, you're buying something that doesn't exist.

This matters disproportionately for founders making vendor decisions without a technical co-founder to run the evaluation. A seasoned engineer can usually tell in a 30-minute technical call whether a system has real ML behind it. If you don't have that option, the right move is to ask the questions that force clarity — questions that a real ML system can answer and a rules engine cannot.

A real machine learning system has a model — a mathematical structure trained on data that generates predictions or outputs for inputs it hasn't seen before. The defining characteristic is generalization: it can handle inputs that weren't explicitly anticipated during development, because the model learned patterns from examples rather than following rules written by a programmer. A rules-based system, by contrast, can only handle inputs that match the patterns it was explicitly programmed to recognize. When the input falls outside those patterns, it fails — usually silently, with a confident-looking wrong answer or a fallback to a default response.

Understanding accuracy claims is essential, because vendors universally quote accuracy numbers in their decks and those numbers are almost always measured on their best-case data. “97% accurate” means the system got the right answer on 97% of the examples in their test set — a test set they selected, labeled, and optimized against. Real-world accuracy on your data, with your edge cases, in your context, is almost always lower. The question isn't whether they have an accuracy number; it's whether that number was measured on something representative of your situation.

The most useful diagnostic is simple: ask the vendor to describe how their system works without using the words “AI,” “machine learning,” or “model.” If their description sounds like “when the user says X, we return Y” or “we match the input against our database of patterns” — that's a rules engine. A genuine ML system will have a description that involves training data, a model architecture, and inference: “we trained a classifier on 200,000 labeled examples, and it generates a probability score for each possible output.” The specificity is the signal.

Follow up by asking what happens when the input falls outside the training distribution — when a user asks something the system hasn't seen before. A rules engine will either fail silently or return a generic fallback. A real ML system has a principled answer to this question: it may return a low-confidence output, flag the case for human review, or explicitly say the input is out of distribution. How the vendor answers this question tells you a lot about whether they've actually thought about production behavior or whether they've only thought about the demo case.

Finally, ask how the system improves over time. ML systems get better when they're retrained on new data — there should be a concrete answer about feedback loops, retraining schedules, and who controls the update process. Rules engines improve when a programmer updates the rules — which is fine, but it's a different product with different scaling characteristics. A vendor who says “it learns from user behavior” without being able to explain the mechanism is using the vocabulary of ML without the substance.

There's a spectrum of what vendors mean when they say “AI-powered.” At one end: pure keyword matching or regex with a conversational UI. One step up: decision trees or rule-based classifiers that are deterministic and programmer-authored. Another step: GPT API wrappers where the vendor calls a foundation model and presents the output as their own AI. These exist across a wide range of sophistication — some GPT wrappers add genuine value through prompt engineering, retrieval-augmented generation, and fine-tuning on domain-specific data; others are a system prompt and a monthly API bill.

At the genuine ML end of the spectrum, you have systems with proprietary training data, custom model architectures or fine-tuning, rigorous evaluation pipelines, and a feedback loop that improves the system over time. These are expensive to build and maintain, and the companies that have them know their systems in detail — they can tell you training data volume, model architecture choices, evaluation benchmarks, and production performance metrics. If a vendor can't give you this level of specificity, they probably don't have this kind of system.

The practical takeaway: a rules engine or GPT wrapper isn't necessarily the wrong choice for your use case. The problem is buying one while expecting the properties of the other. Keyword matching is fast, deterministic, and auditable — those can be features. GPT wrappers can deliver genuinely useful capabilities at reasonable cost. But you need to know what you're buying so you can evaluate whether it solves your actual problem and build your roadmap accordingly.