Creating a Scalable MQTT Environment

Creating a Scalable MQTT Environment

MQTT, a light-weight IoT messaging protocol over TCP/IP, is designed for large-scale telemetry environments with remote locations where a small code footprint is required and/or network bandwidth is limited.  It operates as a producer-consumer paradigm, where many “Publishers” (for example, sensors) send out messages to a small set of “Subscribers” (for example, client applications), which consume the messages. This Publish-Subscribe arrangement is mediated by a message broker. The broker is responsible for distributing messages from the publishers to interested subscribers based on the topic of each message. The client application can subscribe to specific topics from certain devices. They can use the messages for monitoring or managing the devices. Each subscriber can receive messages from many sensors. For purposes of configuration and management, a subscriber can also act as a publisher and vice versa.

There are many sensors available from a variety of vendors. Each of them has its own identity, type, address, operating system, etc. If they support MQTT, they each send out messages on a different topic and at a different frequency. The broker receives messages from all connected sensors and sends those to the right client application(s) depending on the topic.

More about Building a Specification on top of MQTT to Meet Your Industry Requirements

To ensure correct functioning, both broker and Subscriber applications need to be tested for scalability and performance before the deployment in a IOT solution. They need to be tested against the expected number of sensors sending a range of messages at different frequency. How can an organization be certain that its new broker or client application will live up to performance requirements? This can be very difficult to achieve. With the number of devices available, it is expensive to try to maintain a lab with even an adequate sampling of these devices. Also, if a sensor is supposed to send out critical information, it is very important that receiving and processing of those alarms is tested thoroughly. In real-time situations where reaction time is critical, for example if temperature reaches a boiling point, or pressure is above the admissible level, surpassing the response time will result in costly disaster. Now imagine a rare, disastrous situation where thousands of sensors are publishing these types of alarms. A comprehensive disaster preparedness and recovery plan is required. The application should be tested to handle those correctly. It is also necessary for staff training, to ensure that the proper process is in place to handle and to recover from a disaster in a timely manner with least impact on production.

It is not cost effective to have a large scalable lab just for testing. The best alternative is the use of a MQTT simulator. A simulator can create a real-world test lab with thousands of IoT sensors and devices. It allows application vendors to design, develop and test their Brokers, Load balancers and client applications using secured connections, in a virtual and scalable IoT network environment. Using a simulator, they can assure their customers that their applications will be able to handle the scale of connections, topics and varieties of messages to work properly when deployed across heterogeneous environments. The organizations deploying those applications can also try a variety of pathological scenarios before deploying them.

An IoT Simulator can help create a virtual IoT Smart City in their test lab. Each developer, tester or staff member can have their own private city simulation. They are able to simulate many thousands of MQTT based Publishers and Subscriber, each with their own IP addresses Port, Client ID and Authentication. Each Publisher can publish multiple, unique topics. Each Subscriber can subscribe to multiple topics, including wild-card topics.

Further Reading: MQTT Enables IIoT Security Best Practices within the Purdue Model

Realistic testing of the application against the simulation early on significantly reduces problems found in later stages of testing. In addition, it is easy to do future planning by creating a lab with current and new device simulations, instead of waiting for a new device purchase. It is easy to configure sensors to generate the type of alarms at any needed frequency. All these can be done without running around the lab and worrying if all the devices are connected and configured properly. Imagine the time this would save in the testing procedure! With a simulator, application developers can enhance the scope of regression testing. Testing proceeds faster and companies can have higher confidence in the level of software testing, and software products can be released to customers more quickly. And in this way companies can win the time-to-market battle with the competition while assuring a high level of quality!

More about What’s best for IIoT: An integration hub or an MQTT broker?

 

This article was written by Pankaj Shah the founder and CEO of Gambit Communications. He has 20+ years of experience in corporate building and engineering. He holds a Master’s degree in Computer Science from the University of Massachusetts and a Bachelor’s degree in Electronics & Telecommunications from the College Engineering Pune.