External Integration with Webhooks Demo – Technical Blog Series
June 26, 2018
Today we are going to discuss integrations with external systems using web-hooks. Web-hooks are a very popular way to allow one system to anonymously send data to another system.
We see this in many out of the box vertical IoT solutions, proprietary connectivity solutions like LoRA or SIM cards, and from things like payment processors validating transactions. Essentially, by configuring a web-hook it means we have created a web server available to catch what are most often http calls using the get/put or post methods. The concept is a great pattern and out of the box all ClearBlade services are configured to be available via http post calls, so ClearBlade certainly encourages with the implementation.
That said, many services that use the web-hook model get lazy when it comes to security. Meaning, the services sending data over a web-hook don’t have features to implement security calls, which ultimately make your IoT solutions vulnerable to things like spoofing and DDOS attacks.
- Accordingly, today we are going to demonstrate how you can still maintain the integrity of your IoT solution in ClearBlade while still opening up an endpoint for one of these less sophisticated web-hook implementations.
- To do this we are going to deploy a web-hook adapter. Adapters handle all sorts of protocol translations for us. This specific web-hook adapter we are demoing today will run on its own compute instance where we direct external traffic and protect our core platform system.
- We will use an Edge to deploy and monitor the adapter. We can also use some local edge logic to parse the incoming data and validate that it is coming from a trusted source with clean information. This architecture design is a great way to control your exposure to threats.
- ClearBlade Sandbox Account (Create)
1. Start by going to IoT Package Manager (IPM) and find the “webhook-adapter” package, and click into it.
2. Copy the “webhook-adapter” URL by click the “Copy” button to the right of it.
3. This next step is different from most of the other ClearBlade Technical blog series because you will actually import the package into one of our existing systems as opposed to a new system. For this demo we are installing the “webhooks-adapter” into “Intelligent Transportation” system. Click “Data” underneath “Intelligent Transportation”.
4. From this page click “Install” on the left tab bar. Paste the “webhooks-adapter” URL and click “Fetch”.
5. Make sure the assets you want to import are selected then click “Import”.
6. Now that the IPM package has been imported you need to update the constants library with our platform URL. Click “Code” in the left hand tab, under “Library” click “webada_constants”, change the URL to your platform URL, and click “Save”.
7. Next, you need to run the setup service for creating your portal users that will allow you view your web-hook results. Click “Code” again on the click and click “webadaSetup” under services. Here it will show the the credentials you will be creating. Feel free to change the credentials here. Then click “Save and Test”.
A pop up saying “Success” will open if everything is correctly installed.
8. With this done, you need to deploy the web-hook adapter. You will need a standalone web server for the web-hook. For this demo we will be creating a new compute instance in a Google Cloud project area. ClearBlade is cloud agnostic, so you can do this in any cloud such as AWS, Azure, or Softlayer.
What’s interesting from the ClearBlade perspective is that you are going to use an edge in the cloud. While we usually imagine an edge running on a gateway device in the factory, the same value of being able to push processing and grooming of data to another compute instance is valuable here. You will need to edit the cloud network settings to allow for http requests.
9. With the compute created, go back to your system and click on “Deploy” and then click on “webhook-adapter-amd64” to see that you have another artifact created in the system by the IPM installation. You can see you have a deployment specifically for the web-hook. If you click “Adapters” in that deployment you can also see the web-hook adapter and an edge.
10. Click on “Adapters” in the left tab then click “webhook-adapter-amd64” to go to the adapters page. In the adapter page you can find the basic information about the web-hook startup parameters, and its current state of not running on any edge.
11. Click the edit tool and a pop up will come up. Under “Start” enter in the platform URL, messaging URL, port, system key, and system secret. Click “Update Adapter”.
12. Now you need to deploy your edge. In the left tab click “Edges” and then click “Setup instructions”. Under “Target” you’ll want to select the architecture you want to use. In this demo we are using “Linux 64bit – AMD”.
13. Run these install commands in a newly created compute instance by copying, pasting, and downloading the URLs in the following order: Download, Unzip, Install, Permission.
14. The last thing you’ll want to copy and paste from the pop-up into the compute instance is the startup command at the bottom. This will download all the adapter data to the local client and will take a few minutes.
15. To validate that it is running and connected close out of the edge view by clicking “Done”, click “Adapters” in the left hand tab, and then click “webhook-adapter-amd64”. If you scroll down you should see one edge under the “Running” tab. Click “View” and you can see the logs and double check it is listening to the correct port.
16. Click on “Messaging” in the left hand tab and double check that there are no messages yet.
17. A simple way to test if your web-hook is working is by trying out a browser hit directly against that adapter by copying the external IP into your browser followed by a colon and the portal your web-hook is set to.
18. If you refresh your dashboard and click “webhook-adapter/received” you’ll see that it has received two messages. One for the favicon and one for trying to load the webpage.
19. So that you can see the results of your web-hook adapter, go ahead and open the demonstration portal by clicking “Portals” in the left tab, then clicking “webada Incoming Webhooks”, and logging in as your default user. In all reality, you will probably never need to look at this portal in production, but it helps validate that everything is working. Once logged in, you now have a UI see external calls. You have “Incoming Requests” on the left and if you click into it, you can see the details of each request.
20. With everything set up in the demo, I want to go the extra mile and play with a chrome plugin called Postman which lets me issue different types of http requests. In postman set the method to “POST” a payload into our newly created compute IP address which you’ll paste into Postman. For our demo payload I’ll do something interesting like show an update from a third party IoT service for a connected vehicle.
21. Hit “Submit” and the http request will leave the computer and go to the web-hook adapter. The web-hook adapter then converts that request into an MQTT message and sends it into the ClearBlade Edge. The Edge then relays that message up to my cloud system and you can see the results in your messaging feed almost instantly. If you go back to our demonstration portal you can see each web-hook call and inspect its results historically.
With the web-hook available, we have made it easy to let external third party systems provide us data securely and allow us to enrich the context of everything that’s happening in our ClearBlade IoT Solution.