Today we are going to look at how to detect anomalies in IoT solutions. As enterprises adopt IoT, value comes not from simply getting visibility into the state of their assets, but rather the ability to act upon and resolve important events. For many use cases, being able to simply monitor and report things that aren’t normal provides enormous opportunity. In this tutorial we are going to quickly add the ability to watch for anomalies in our IoT Solution.

Concepts Demonstrated:

• Integrate device data with real-time, anomaly detection with the ClearBlade Edge Platform

Prerequisites:

• ClearBlade Sandbox Account (Create)

1. Start by going to the ClearBlade IoT Package Manager (IPM) to find a package that will get you started. Searching the IPM for “anomaly detection”, you’ll see two packages. One is “anomaly-detection-template” and the other is “anomaly-detection-library”. In this case, select the “anomaly-detection-template” because it has both the core reusable libraries and a nice demonstration associated with it.

2. Click in the “anomaly-detection-template” IPM and copy the URL. In the ClearBlade platform developer console perform the normal IPM install process. To install the anomaly-detection-template package as a new system click “Install” on the left hand side, paste the URL, and then click “Fetch!”. Make sure all the assets are selected and click “Import”. Once imported you will see “anomaly-detection-template” show up under Systems.

3. This system is ready out of the box, but in order to leverage some of the integrations, you need to update the Twilio configuration so that you can show text messages integrated by editing the constants library. Start this process by clicking “Code” in the system, then clicking “AnomalyDetectionConstants” under Libraries. Insert credentials for “USER”, “PASS”, and “SOURCE_NUMBER” and click “Save”.

4. Now begin using the anomaly detection by opening the associated portal by clicking “Portals” on the left hand side, then clicking “AnomalyDectection”. With the portal open you can see that you are getting a simulated data stream. In this template system there is a timer generating that data. The raw data shows in the Sensor Feed pane, and is visualized in the graph as the middle blue line. The red lines represent the threshold at which a sensor feed datapoint is considered an anomaly. The anomaly detection algorithm is powered by a data science technique called ‘Moving Medians Decomposition’, and is widely used for Preventative Maintenance solutions.

5. What is interesting is the library is easily reused for different tolerances. If you wanted to, you could easily change the allowed strictness in the detection algorithm. You can update the detection rules in the user interface. Notice this anomaly detection is self-training, meaning it can determine on its own what value corresponds to LOW, MEDIUM, HIGH strictness. Setting it to LOW strictness allows for high variability in the data feed before reacting. You can also change the field of interest to something other than temperature if you are instead interested in acceleration, vibration, decibels, or humidity.

With the detection rules matching your needs, you can see the results appearing in the alerts section. Here you can see the latest anomalies that have been identified. In this case, it is set to keep a list of all the anomalies that have ever occurred which can be reviewed and studied by your data scientist.

Lastly, you can make these anomalies alert you via text or email. In this demo, since we already configured twilio, lets go ahead and add my phone number and let the alerts begin to roll in.

Here we can see the alerts showing up alerting me to temperatures that are statistically abnormal.

6. Before we wrap up, let’s look at how you might integrate the library in your own IoT solution. Back in the developer console click “Code” on the left hand side. Under “Libraries” you’ll see several different types. “jStat” is a statistics library doing the math and the detection library that is implementing the specific algorithm. The “AnomalyDetection” library is actually going to use the mathematics of the “jStat” library and define and trigger alerts.

7. Now go look at the example service to see how easy it is to leverage. Under “Services” click “ExampleDetectAnomaly”. First, you’ll see the Strictness variables with numerical values that you can change for your use case. Under the Strictness you’ll see the sample inputs, which would normally be coming directly from your sensors, and finally a place where you run the library to detect.

8. Click “Save and Test” and you will see the calculated outliers. This box shows from the example 3 points that were perceived as anomalies that will send alerts. This library is very simple and may be used across any of your device attributes or message streams, or collections to let you know things like if the soil temperature is too dry or if the device has been dropped to the ground.

That’s it, thanks for tuning into the ClearBlade Technical Blog Series.

Aaron Allsbrook, Chief Technology Officer at ClearBlade