Cross Platform IoT: Troubleshooting IoT Hub using IoT Hub Diagnostics

Cross Platform IoT: Troubleshooting IoT Hub using IoT Hub Diagnostics

This post is part of a Cross-Platform IoT series, to see other posts in the series refer here.

In the previous posts we looked at the Azure IoT CLI and IoT Hub Explore which assisted us in the management of Hub, Devices as well as sending a receiving messages. In this post, we discuss a troubleshooting utility that further assists in IoT Hub development and support.

Why may we need a diagnostics utility?

There are times when things go wrong and you are trying to figure out what could be the issue. The trickiest of all are generally network connectivity issues, whether its a firewall blocking a port or a bandwidth issues resulting in intermittent connections. While IoT Hub supports Operation Monitoring through the Azure Management Portal as well as the IoT Hub Monitoring endpoint, digging into the logs and crash dumps can take multiple support tickets and valuable time. The IoT Hub Diagnostic utility was a recent addition to the IoT Hub toolkit and is a useful “ping” type utility that provides some early feedback on probable issues in case IoT Hub is not behaving as expected.

The tool is also written in node.js and is available as a Open Source (MIT) solution. To install the tool on your machine, run the following command in a terminal:

npm install -g iothub-diagnostics

Initiating the diagnostic tool is fairly simple, all you need is your IoT Hub connection string. There are no other options or commands :).

From my tests, it seems the tool does require the iothubowner permission, I did not try with all policies so it may be possible to restrict access or use a lesser rights policy.

iothub-diagnostics "HostName=youriothub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=="

Once the connection is established, the tool kicks in to perform a series of tests:

  1. The firsts are network tests to ensure DNS resolution, the utility is just pinging a URL here to ensure connectivity to internet is possible. The default pingURL is at the moment hardcoded to www.microsoft.com. The code does have the option to provide your own DNS or Addresses so you can customize this as desired.
  2. It then checks for ports availability and TLS Cipher requirements, again these tests include a ping on HTTPS (443), the default httpsRequestUrl is hardcoded to https://www.microsoft.com/
  3. The final checks are performed on the IoT Hub itself where first a temporary device is registered and a series of tests are run to ensure D2C (device to cloud) and C2D (cloud to device connectivity using all supported protocols (HTTPS (443), AMQP (5672 and 5671) and MQTT (1833 and 8883) ).

Once the tests are completed, you should see a similar result:

2017-03-16T07:38:51.193Z - info: *******************************************
2017-03-16T07:38:51.196Z - info: * Executing the Microsoft IOT Trace tool. *
2017-03-16T07:38:51.196Z - info: *******************************************
2017-03-16T07:38:51.197Z - info:  
2017-03-16T07:38:51.198Z - info: --- Executing network tests ---
2017-03-16T07:38:51.222Z - info:  
2017-03-16T07:38:51.223Z - info: Starting DNS resolution for host 'www.microsoft.com'...
2017-03-16T07:38:51.254Z - info: --> Successfully resolved DNS to 23.204.149.152.
2017-03-16T07:38:51.255Z - info:  
2017-03-16T07:38:51.255Z - info: Pinging IPV4 address '23.204.149.152'...
2017-03-16T07:38:51.291Z - info: --> Successfully pinged 23.204.149.152
2017-03-16T07:38:51.291Z - info:  
2017-03-16T07:38:51.291Z - info: Sending https request to 'https://www.microsoft.com/'
2017-03-16T07:38:51.444Z - info: --> Completed https request
2017-03-16T07:38:51.445Z - info:  
2017-03-16T07:38:51.445Z - info: --- Executing IOT Hub tests ---
2017-03-16T07:38:54.731Z - info:  
2017-03-16T07:38:54.732Z - info: Starting AMQP Test...
2017-03-16T07:38:59.141Z - info: --> Successfully ran AMQP test.
2017-03-16T07:38:59.142Z - info:  
2017-03-16T07:38:59.142Z - info: Starting AMQP-WS Test...
2017-03-16T07:39:03.460Z - info: --> Successfully ran AMQP-WS test.
2017-03-16T07:39:03.466Z - info:  
2017-03-16T07:39:03.466Z - info: Starting HTTPS Test...
2017-03-16T07:39:08.036Z - info: --> Successfully ran HTTPS test.
2017-03-16T07:39:08.059Z - info:  
2017-03-16T07:39:08.060Z - info: Starting Mqtt Test...
2017-03-16T07:39:11.828Z - info: --> Successfully ran Mqtt test.

In case of any test failure, the appropriate exception are logged and displayed.

Overall, the diagnostic tool is a nifty little utility for quickly checking your environment for common problems around Connectivity and Device operations. Note that the tool does not provides traces or network monitoring support for the IoT Hub servers, it is only for client connectivity and testing your IoT hub basic operations.

There is definitely scope for adding more features and logging support so we can see more details about the types of tests are being run. I would also want to see this integrated into a CI/CD pipeline after extending the basic operation tests with our own custom test suites.

This post is part of a Cross-Platform IoT series, to see other posts in the series refer here.

the “internet of things” is the next big bang!

What’s new about this, so you would ask?

Internet of Things aka IoT is already making a big impact in our day-to-day lives, the fact that the IoT industry became a $1.24 trillion (Source: Markets and Markets) business in 2013 proved that IoT is big and here to stay. However the reason I say this is not because how many under 25 billionaires it will create but how it is going to change our lives forever.

This realization came to me just recently …

the other day my wife was making tea and she realized we were out of sugar (yeah I forgot it during the last visit to the grocers, how that ended for me is another story) and said “I wish someone can fill this up magically every time!” … Now if this was some years back I would be thinking of Aladdin and his Genie but with the advent of IoT I started thinking this might just be possible …

Consider a “Smart Jar” that has sensors which track its quantity, the consumer can set a configuration of sending alerts whenever the quantity reaches below a minimum limit, the device can then send notifications to the user or add the item to their favorite grocery list app. Taking this a step further the “Smart Jar” can connect to an external provider like Amazon or Target which the user has a subscription for (in USA) and then automatically schedule the item for next delivery. Also for people like me who constantly forget the location of items in the kitchen a mobile app lets you find the appropriate jar based on its co-ordinates. From a producer perspective the jar may send telemetry on usage pattern for families which can show demographics on how products are used.

While the above example may seem a little overreached (btw some start ubig-bangp might just be working on a solution for this right now!) the point I was trying to make with the example above is that many such tasks that touch our daily lives will be simplified and automated with the use of connected devices. This, by itself will be a revolution not just in our homes but for corporate as well. It will change how we eat, drink, shop … live and that is why I say it is the next big bang!

Now the next question that comes to mind is … are we ready for it?

Many governments and companies are investing generously in the research and development for IoT solutions (Industry 4.0). This is great for the IoT industry, however any industry needs consistent sales to sustain and prosper. We have seen successes in certain domains such as connected homes and thermostats (Nest) , automobile (BMW), wearable etc but there is still so much untapped potential that this just seems like the tip of the iceberg.

One of the big challenges for any industry is adoption; since the events we are talking here are life changing there are certain principles that should be followed when building devices targeted for IoT to enable mass adoption:

Intuitive: When the iPad was launched several years back it lacked a lot of features but one thing that was prominent in the device was that it looked “sexy” and simplified a lot of tasks that were cumbersome on a laptop or a desktop (aka the dinosaurs that existed some years back) and that was one of the main reasons of its success. History was repeated when Nest launched its smart thermostat which though may not be completely accurate from a temperature or humidity perspective provides great intuitive features that has compelled major manufacturers in this field to release similar variants. So any device that will be launched under the IoT umbrella needs to be intuitive and simplified rather than sophisticated. Some key features that any device may exhibit:

    • Does the job it is meant to do ALL the time and without errors: if my wife had to call support for a simple device like the “Smart Jar” she would not use it next time.
    • Multiple sensors to predict user actions: ease user life with determining what they want to do
    • Easy installation and upgrades: if not my grandma then at least my wife should be able to configure it :).
    • None or minimal maintenance: No frequent battery changes, wired connections, connectivity failures

Privacy and Security: Security is an obvious concern and has been highlighted in many articles. With multiple devices running all the time and sending telemetry data back to manufactures they can literally predict what you are doing right now in your house. This is an invasion in privacy which both consumers and corporate will oppose to. Simply put, convenience at the stake of privacy will not sell!! We are still in an immature stage in this space but work is being done to define policies around data privacy and end-user security. This is an area I would be watching before placing my bets on IoT.

dilbert-140511

Cost: A device by itself does not achieve an IoT scenario, it needs to be backed with the power of cloud and data analytic so when evaluating the cost of a device multiple auxiliary items need to be accounted for such as:

    • Hardware (sensors, MCU, RAM etc.),
    • Communication and network interface
    • Messaging channel transactions
    • Cloud compute for analytic and business systems
    • App development
    • Data storage

These are just high level items, there are multiple hidden costs apart from these that need to be constituted in the selling price of the device. Now all these details make a compelling reasoning for the costs to be higher than a normal device, however the consumer does not care about what goes in the device or that a sophisticated cloud platform is backing the solution (at least not from a cost perspective), a light bulb is a light bulb and if a consumer gets it for 10x of the average price only a small community would be interested in it most likely for experimental purposes. It is thus essential to keep the costs to the minimum.

An effective way can be to keep the device price low and provide subscription based solution for enabling more features on the device. Also as the hardware manufacturing costs keep coming down we will see many of these devices become reachable to the masses.

Interoperability: This is always a hot and controversial topic for me, Hot because just thinking of interoperability as part of IoT standards opens up a plethora of opportunities and can introduce huge gains to consumers. Controversial because a lot of companies today bet their business on their platform, example Windows for Microsoft and iOS for Apple, so from a business perspective if you have a closed platform people will get tied to it and most likely will stick to it for years to come. Besides once you get people on your platform it is easy to sell them supplement services that are optimized for that platform, for example YouTube experience on an Android device is far better as compared to YouTube on a Windows phone.

Now platform dominance was OK for the PC, tablet and phone market primarily because of their controlled production and legacy proficiency but the IoT introduces a whole new generation of devices and it seems almost impossible that companies can become successful without providing an open and inter-operable interface.

Think of an example, your microwave needs to get input from your refrigerator so it knows at what temperature it should warm the food at and for how much time (example shamelessly stolen from the ebook published by CoAP sharp team). Now the microwave is from Samsung but the refrigerator is from LG, the only way they can talk to each other is through a common platform. Now apply this to the hundreds of devices that you will have in your house, automobile, and offices in the coming years that are developed in different countries and by different manufacturers, unless they all have some common form of Interfaces to communicate the IoT vision and goals will not be achieved. Agreed that there will be companies that will create some middle ware such as Belkin Wemo Smart Switch but I would consider these as alternatives to legacy devices, anything new should be inter-operable by design.(period)

I will talk more about work being done in this space in a future post.

Infrastructure: Cloud Computing has been a game changer in how business’s work, it enables organizations to have the impression of unlimited resources for any of their computing needs and that too at a cheaper cost. Well unlimited is a stretch right now since most cloud providers have some restrictions and bars defined on your usage but these limits are still too high for most of business’s. Moreover if required organizations can put in a bag of money to have their own silo data centers in order to achieve their scalability targets, this is still cheaper than hosting and maintaining everything in their in-house data centers.

This all works very well for almost 99% of the Web and Mobile scenarios today however the IoT space is different from how Web and Mobile applications operate:

Most Web and Mobile solutions require human intervention so the spike or bursts in the server load is intermittent or during standard schedules such as peak hours, in case of devices however this load is constant, for example, if a thermostat is configured to send telemetry data back to the server every 30 seconds that is a constant activity that the device will continue to perform unless it breaks due to some failure, it does not stop for lunches or take bio breaks it just keeps transmitting data every 30 seconds. Now consider 10 Million of these thermostats deployed across continents and each thermostat transmits around 10 KB of data (including headers and payload) . We are talking about 100000000 KB (95.4 GB) of data being sent for processing and storage at a constant rate of 30 seconds. This is huge and while the cloud might still be able to accommodate such loads through Big data and Auto Scaling the small and medium business’s would have challenges managing the costs of  running such solutions. Note that I have described a simple ingestion scenario above, add the data coming from mobile application commands, device inquiry etc. and this number just keeps growing.

There is no silver bullet to the explosion of data and how back ends will manage it while still keeping the costs low, as we mature more in the Cloud space we should see price drops in Cloud computing and storage solutions which should bring the overall costs down, also optimization on the device payload and messaging can ensure minimal data being sent over the wire (Protocols like MQTT and CoAP are being designed for such type of solutions)

So where do we go from here?

IoT is one the best things that has happened in the our space, it has blurred the lines between the hardware and software industry, enabled scenarios that we could only see in movies or dream about however there is a need for standardization of how companies design and implement solutions, instead of working in silos we need to work towards a consistent and effective reference solution that may apply to most if not all scenarios. Of course there will be tweaks and turns for each sub domain but if the basic principles and not violated with we are looking towards a new world that will change how we work today … this will truly be a big bang!