Why I Don't Let AI Make the Final Decision

Discover AnythingSignupWrite New StoryWhy I Don't Let AI Make the Final DecisionbyOke Iyanuoluwa EnochbyOke Iyanuoluwa Enoch|@iyanuoluwa-007Robotics and AI Engineer specializing in ROS 2, Computer Vision, Machine Learning, and Embedded Systems.SubscribeJuly 29th, 2026TLDR Your browser does not support the audio element.Speed1xVoiceDr. One Ms. Hacker byOke Iyanuoluwa Enoch@iyanuoluwa-007byOke Iyanuoluwa Enoch|@iyanuoluwa-007Robotics and AI Engineer specializing in ROS 2, Computer Vision, Machine Learning, and Embedded Systems.SubscribeStory's CredibilitybyOke Iyanuoluwa Enoch|@iyanuoluwa-007Robotics and AI Engineer specializing in ROS 2, Computer Vision, Machine Learning, and Embedded Systems.SubscribeStory's CredibilityEveryone wants to talk about how smart the model is. Nobody wants to talk about what happens when it is wrong. That gap, between model performance and system reliability, is where most AI products quietly collapse. And it is the thing I think about most when I am building anything that is supposed to work in the real world at production quality. I have worked across robotics perception, embedded systems, autonomous navigation, and product-scale AI systems, including Sentinel Quant, a human-in-the-loop portfolio management platform, and Signlytic, a bidirectional British Sign Language translation system combining computer vision, large language models, and voice synthesis. In every single one of those contexts, the same pattern appeared: the most reliable systems were never the ones with the best models. They were the ones whose designers understood, early, that a model is not a decision-maker. It is a component. That distinction sounds small. It changes everything.The core problem with "trust the model." AI models are probabilistic by nature. That is not a flaw. It is just what they are. But probabilities become dangerous the moment they are mistaken for authority. A model can be: highly accurate on your test set and still fail on inputs that sit slightly outside the training distribution confident and wrong at the same time, because calibration and accuracy are not the same thing fast in development and completely unusable on edge hardware after deployment correct 98% of the time and catastrophically incorrect the other 2%, specifically in the cases where correctness matters most This is why "trust the model" is not a system design principle. It is an absence of one. Consider even something as conceptually clean as object detection. In my CARLA-based perception work using YOLOv11 with ByteTrack and StrongSORT, the model outputs a detection with a bounding box, a class label, and a confidence score. That confidence score is not a guarantee. It is a probability estimate. And in a live robotics system, acting on every detection above some arbitrary threshold without downstream validation is how you get a robot that makes confident, fast, wrong decisions. A responsible pipeline does not treat model output as truth. It treats it as a hypothesis that still needs to survive a gauntlet.What a governed AI system actually looks like The architecture I return to across robotics, fintech, and language systems is always some variation of this: Raw input | [Model inference] <-- probabilistic, fast, flexible | [Confidence threshold] <-- gate: is this output trustworthy enough? | [Deterministic rules] <-- hard constraints: safety, policy, physics | [Human review layer] <-- triggered when uncertainty is above threshold | [Logging and audit] <-- full trace, always | Action or output Each layer does something different. The model is good at pattern recognition and generalisation. The deterministic layer is good at enforcing constraints that must never be violated, regardless of what the model thinks. The human layer handles uncertainty that neither of the above can resolve reliably. And the logging layer makes the whole thing inspectable. Remove any one of these, and you have a wea