The Failure of IoT’s Frankenstein Monsters: By Aaron Allsbrook
March 15, 2022
If you’re in IoT, you’ve most likely heard rumors of the corporate IT or big service integrators’ Frankenstein Monster. These Frankensteins are often proudly presented in strategy sessions or digitalization meetings. They imply that your IT partner group has all the answers for your business. These shallow green monsters take many names. “The IoT Hub”, “ACME IoT Platform”, “Corp IoT Cloud”, “All-In-One IoT Portal” “Organizational Control Center. Despite their different names these monsters built by wild haired scientists overwhelmingly grunt, moan, and stumble about the same way.
Today’s IT departments are made up of engineers who are excited to try new things and escape the old processes of business. IoT (Internet of Things) represents a new frontier, a new cool opportunity, and definitely an opportunity to get sponsorship of something that will be critical for all business units. IT departments do what all good software architects do, they attempt to imagine everything in their business getting connected up and for some reason think that all that information should be managed by an amalgamation of tools. They then read about the Netflix architecture and its chaos monkeys with great admiration. They study Kafka and load balancing techniques. Then they go into a cloud and look across hundreds of cloud services to try to use as many as they can for no clear reason. The end result is an architecture diagram so big and full of tiny boxes that no one really has the time or will power to read. Layers and layers of boxes upon boxes for “Ingestion”, “Persistence”, “Caching”, “Service Application”, all to get a simple sensor feed. Thus begins your Frankenstein IT’s journey to build the IoT Monster. Like Frankenstein’s monster, your IoT monster will share 5 failures.
1. Frankenstein’s monsters are hard to keep alive.
These architectures, while maybe broken up into isolated microservices, ultimately create a constant cacophony of failures from updates, to new dependencies, to changing requirements. You will spend all your time trying to bring them back to life. In time you will find that your efforts are focused on reviving the monster rather than building much needed new business features.
2. Frankenstein’s monsters are slow.
When you build a system where you just follow the example put forward by others you don’t understand all the constraints. Unlike an ecommerce site or a video stream service or a high speed trading platform your IIoT data has different sizes, cadence, and processing needs. While there is much to learn from others, your needs can’t simply be an ugly copy of others.
3. Frankenstein’s monsters can’t be trusted.
IIoT data comes from many sources and must respond at operational speeds. The traditional tolerances of your IT for acceptable response times will not meet your operational speed needs. Waiting 2-10 minutes for an asset to be provisioned or a ticket to be opened or a transaction to take place does not work for the emerging needs of IIoT. Unfortunately, the monsters being built today by IT are using older technologies that aren’t used to the speeds and requirements that the operations environment requires.
4. Frankenstein’s monsters are expensive to build and maintain.
Medieval laboratory cloud services are just the beginning of your Frankenstein’s monster. To bring your monster to life it takes lightning bolts of energy, resources, and funding to gather data scientists, devops engineers, backend engineers, front end developers, QA, testing frameworks, and devops tools to jolt your monster to life. Once alive all of those costs continue as creating new features and expanding the digital initiatives have no end in sight.
5. Frankenstein’s monsters don’t understand what you need to do.
As these monsters are built and put into place they are built only with the eyes of cloud architects and front end developers. While these roles have strong and valuable skills the reality is they don’t know anything about operations being performed and processes being run in the field. The application they build represents this lack of field knowledge and will only be able to moan and grunt back at you when you ask them to perform business specific tasks.
How did we get to this point? Fundamentally our IT and big SI’s groups are playing follow the leader. Amazon Web Services has driven the micro service model to go on ad infinitum. The unfortunate reality is AWS is trying to find ways to charge you more money, not help you build a good solution. Cloud services become confusing, complicated, and a complete lock-in. Netflix has told the world about their Kafka / Flink model for managing subscribers watching TV. Their metrics are impressive but for your corporate architect the Netflix pattern doesn’t solve your IoT problem. Unless you’re building a streaming TV service starting with technologies from 10 years ago, you shouldn’t blindly follow their design.
Over 80% of IoT projects are failing today following this Frankenstein approach. Transparently, we at ClearBlade built a Frankenstein 6 years ago and that implementation was fragile, performed poorly, and was rigidly trapped. Microservices overloaded, engorged on devops containers everywhere and a heavy implementation of cloud services. Instead, ClearBlade threw out the Frankenstein and built a small, reliable, highly scalable single binary application just for IoT transactions. We use, but don’t abuse technologies like containers and microservices. We scale across compute generically, agnostic to the infrastructure provider. The principle of KISS (keep it simple stupid) applies now more than ever.
If you are a business user reading this, beware… I daily talk to businesses that are blocked, waiting for endless updates to these IT experiments. If you’re IT and reading this, beware… just because everyone in the industry is talking about doing this, your Frankenstein will most certainly FAIL. If you’re in management trying to decide a course of action, you need to ask yourself if you should invest yet another year into a never ending project. Should you keep building something that you will forever have to maintain? And how many years and how much money has already been spent? Maybe it’s time to stop following the masses. After all, Dr. Frankenstein was undoubtedly mad, and ultimately his monster failed.