Edge AI is not a Raspberry Pi calling a chatbot

ClearBlade founder and CTO Aaron Allsbrook on why large language models are an expensive way to solve a problem for the first time, and why the real work in IoT is turning that first answer into something small, local and repeatable.

‍

Every industrial team is being asked the same question this year: where is the AI? The fastest answer is also the most expensive one. Wire a hosted large language model into the data flow, send it every reading, and let it decide.

‍Aaron Allsbrook, ClearBlade's founder and CTO, has seen where that leads. It works in a demo. It does not hold up across a fleet. IoT runs on thousands or millions of devices, each sending data many times a minute. Put a hosted model behind every message and the bill grows with every device, every decision waits on a round trip to someone else's data center, and the intelligence stops whenever the link drops.

‍"I don't really feel good about putting a Raspberry Pi that makes 10,000 calls to a hosted chatbot every time the lights come on," Allsbrook says. "It won't sustain as the truck drives around."

‍

21.9B: connected IoT devices expected worldwide by the end of 2026 IoT Analytics

3x: how much more organizations will use small, task-specific AI models than general-purpose LLMs by 2027 Gartner

90%+: expected drop in provider cost per token by 2030, while overall inference spending still rises Gartner

‍

Why one model call per message breaks down

The usual reply is that model prices keep falling. They do. Gartner expects the provider cost of running a trillion-parameter model to fall more than 90% by 2030. In the same forecast, it expects token consumption to grow faster than prices fall, so overall inference costs still go up. Its advice is to send routine tasks to small, domain-specific models (Gartner) and save frontier models for complex reasoning.

‍In IoT, nearly every message is a routine task. A temperature reading or a valve state does not need a frontier model to interpret it a million times a day. "It's an incredibly expensive way to solve a problem," Allsbrook says of the large hosted models. They are excellent at figuring something out the first time, and the wrong tool for repeating it ten thousand times a second.

Cost is only half of it. Large language models are non-deterministic, so the same request can come back with different answers. That is fine for brainstorming. It is not fine when the output decides whether a pump shuts off.

‍"You can't get a purple elephant and a green hippopotamus as your two options when you're just trying to talk to an old machine," Allsbrook says. "You've got to take the non-determinism into something deterministic, which is why you need rules to really do IoT."

‍

"A lot of what AI will generate can turn into old-fashioned rules."

Aaron Allsbrook, Founder and CTO, ClearBlade

‍

What edge AI looks like in practice

Allsbrook's approach uses the large model where it is strong and keeps it out of the data path:

1. Build with the big model. Use a large language model to understand a new machine, data source or question.
2. Compile the answer down. Turn what it learned into a deterministic rule, or a small open-source model tuned for one job.
3. Run it next to the asset. Execute the rule or small model at the edge, with no third-party API call per message.
4. Revise as conditions change. Bring the large model back to update the logic when the environment or requirements change.

‍

Two examples from his own work:

• Power outage detection. The generator is running and there is no inbound power on the main. A rule can raise that flag in milliseconds, on site, without asking a hosted model anything.
• Reading a gauge. A camera grabs a still image, a local OCR model reads the needle and the text, and the value is processed like any sensor reading. Allsbrook built this in about an hour with AI's help, and it runs without sending every frame to an outside API.

‍

Why this approach pays off

Flat cost per device message. No per-message fee that grows with the fleet.

The same answer every time. Rules are deterministic and auditable, so an engineer can see why a decision was made.

‍Keeps working when the link drops. Logic next to the asset turns a lost connection into an inconvenience, not an outage.

‍Rules that still learn. The large model can rewrite rules as conditions change, and the edge picks up the new version.

‍

None of this is new to ClearBlade. "We were doing AI before AI was a thing," Allsbrook says. "We just were calling it rules." ClearBlade has built and run rules for years on a platform with a full domain model for them, and it runs inference in the data flow to keep the "overall cost per device message as low as possible." What has changed is how fast those rules can now be written.

‍

The unglamorous conclusion

Most valuable edge AI looks a lot like rules. "Take the abstract, hard-code it into rules, run that at the edge," Allsbrook says. "It still looks like AI. People might even call it AI." The models will keep getting larger. For operations, the useful part keeps getting smaller and closer to the machine.

‍