Clever Geek Handbook

IoT, fog and clouds: talk about technology?





The development of technologies in the field of software and hardware, the emergence of new communication protocols have led to the expansion of the Internet of Things (IoT). The number of devices is growing day by day, and they generate a huge amount of data. Therefore, a need arises for a convenient system architecture capable of processing, storing and transmitting this data.



Now they use cloud services for these purposes. However, the ever-growing foggy paradigm (Fog) is able to complement cloud solutions by scaling and optimizing the IoT infrastructure.



Clouds can close most IoT requests. For example, to provide monitoring services, fast processing of any amount of data generated by devices, as well as their visualization. Misty computing is more effective in solving real-time problems. They provide fast response to requests and the minimum delay in data processing. That is, Fog exactly complements the "clouds", expands its capabilities.



However, the main question is different: how should all this interact in the context of IoT? What communication protocols will be most effective when working in a unified IoT-Fog-Cloud system?



Despite the apparent dominance of HTTP, IoT, Fog, and Cloud use a large number of other solutions. This is because IoT must combine the functionality of a variety of device sensors with the security, interoperability, and other requirements of users.



Here are just a single idea of ​​the reference architecture and communication standard is simply not there. Therefore, the creation of a new protocol or the refinement of an existing one for specific IoT tasks is one of the most important tasks facing the IT community.



What protocols are used now and what can they offer? Let's get it right. But first, let's discuss the principles of an ecosystem in which clouds, fog and the Internet of things interact.



IoT Fog-to-Cloud Architecture (F2C)



You must have noticed how much effort has been put into exploring the benefits and benefits of managing IoT, clouds and fog in a rational and coordinated manner. If not, then there are already three standardization initiatives: OpenFog Consortium , Edge Computing Consortium and mF2C H2020 EU project .



If earlier only 2 levels, clouds and end devices were considered, then the proposed architecture introduces a new level - fog computing. In this case, the fog level can be divided into several sublevels, depending on the specifics of resources or a set of policies that determine the use of different devices in these sublevels.



What might this abstraction look like? Here is a typical IoT-Fog-Cloud ecosystem. IoT devices send data to more powerful servers and computing devices to solve tasks that require low latency. In the same system, clouds are responsible for solving problems that require a large amount of computing resources or storage space.







Smartphones, smartwatches, and other gadgets can also be part of the IoT. But such devices, as a rule, use proprietary communication protocols from large developers. The generated IoT data is transmitted to the fog level through the HTTP REST protocol, which provides flexibility and interoperability when creating RESTful services. This is important in light of the need for backward compatibility with existing computing infrastructure running on local computers, servers, or a server cluster. Local resources, which are called "nodes of the fog", filter the received data and process it locally or send it to the cloud for further calculations.



Clouds support various communication protocols, among which AMQP and REST HTTP are most often found. Since HTTP is well known and imprisoned for the Internet, the question may arise: “But should I use it to work with IoT and fog?”. However, this protocol has performance issues. More about this later.



In general, there are 2 models of communication protocols suitable for the system we need. This is a request-response and publication-subscription. The first model is more widely known, especially in the server-client architecture. The client requests information from the server, and he receives the request, processes it and returns a response message. The REST HTTP and CoAP protocols work on this model.



The second model arose because of the need to provide asynchronous, distributed, weak communication between the sources that generate data and the recipients of this data.







The model involves three participants: the publisher (data source), broker (dispatcher) and subscriber (recipient). Here, the client acting as a subscriber should not request information from the server. Instead of sending requests, he subscribes to certain events in the system through a broker responsible for filtering all incoming messages and routing them between publishers and subscribers. And the publisher, when an event occurs concerning a certain topic, publishes it to the broker, who sends the subscriber data on the requested topic.



In essence, this architecture is event driven. And this interaction model is interesting for applications in IoT, cloud, fog because of its ability to provide scalability and simplify the interconnection between different devices, to support dynamic many-to-many communication and asynchronous communication. Among the best-known standardized messaging protocols using the publish-subscribe model are MQTT, AMQP, and DDS.



Obviously, the publication-subscription model has a lot of advantages:





However, the request-response model also has its own strengths. In cases where the capabilities of the server side for processing requests of several clients are not a problem, it makes sense to use already proven reliable solutions.



There are also protocols that support both models. For example, XMPP and HTTP 2.0 that support the server push option. The IETF has also released CoAP. In an attempt to solve the messaging problem, several other solutions were created, such as the WebSockets protocol or using the HTTP protocol through QUIC (Quick UDP Internet Connections).



In the case of WebSockets, although it is used for real-time data transfer from the server to the web client and provides constant connections with simultaneous bidirectional communication, it is not intended for devices with limited computing resources. QUIC also deserves attention, because the new transport protocol provides a lot of new opportunities. But since QUIC is not yet standardized, it is premature to predict its possible application and impact on IoT solutions. So we leave WebSockets and QUIC in memory with an eye to the future, but we will not study in more detail yet.



Who in the world is sweetest: we compare protocols



Now let's talk about the strengths and weaknesses of the protocols. Looking ahead, we immediately make a reservation that there is no one clear leader. Each protocol has some advantages / disadvantages.



Response time



One of the most important characteristics of communication protocols, especially with regard to the Internet of things, is response time. But among the existing protocols there is no standout demonstrating a minimum level of delay when working in different conditions. But there is a whole bunch of research and comparisons of protocol capabilities.



For example, the results of comparing the effectiveness of HTTP and MQTT when working with IoT showed that the response time for requests from MQTT is less than that from HTTP. And when studying the reception and transmission times (RTT) of MQTT and CoAP, it turned out that the average RTT of CoAP is 20% less than that of MQTT.



Another experiment with RTT for the MQTT and CoAP protocols was carried out in two scenarios: a local network and an IoT network. It turned out that the average RTT is 2-3 times higher in the IoT network. MQTT with QoS0 showed a lower result compared to CoAP, and MQTT with QoS1 showed a higher RTT due to ACK at the application and transport levels. For different QoS levels, network delays without congestion for MQTT were milliseconds, and for CoAP, hundreds of microseconds. However, it is worth remembering that when working in less reliable networks, MQTT running on top of TCP will show a different result.



Comparison of the response time for the AMQP and MQTT protocols by increasing the payload showed that with a small load the delay level is almost the same. But when transmitting large amounts of data, MQTT demonstrates less response time. In another study, CoAP was compared with HTTP in a machine-to-machine communication scenario with devices deployed on top of vehicles equipped with gas sensors, weather sensors, location (GPS) and a mobile network interface (GPRS). The time it took to send a CoAP message over a mobile network was almost three times shorter than the time it took to use HTTP messages.



Studies were conducted that compared not two, but three protocols. For example, comparing the performance of the IoT protocols MQTT, DDS, and CoAP in a medical use case using a network emulator. DDS outperformed MQTT in terms of experienced telemetry latency in various poor network conditions. UDP-based CoAP worked well for applications that needed a fast response, but because it was UDP-based, there was a significant unpredictable packet loss.



Throughput



Comparison of MQTT and CoAP in terms of bandwidth utilization was carried out as a calculation of the total amount of data transmitted per message. CoAP showed less bandwidth than MQTT when sending small messages. But when comparing the effectiveness of the protocols in terms of the ratio of the number of useful information bytes to the total number of bytes transmitted, CoAP was more effective.



When analyzing the use of the MQTT, DDS (with TCP as the transport protocol) and CoAP bandwidth, it turned out that CoAP, as a rule, showed a relatively lower bandwidth consumption, which did not increase with an increase in network packet loss or an increased network delay, unlike MQTT and DDS, where in the mentioned scenarios there was an increase in the use of channel capacity. In another scenario, a large number of devices transmitting data at the same time were involved, which is a typical case in IoT environments. The results showed that for a higher load it is better to use CoAP.



With a light load, CoAP used the least bandwidth, followed by MQTT and HTTP REST. However, when the payload size increased, REST HTTP had the best results.



Energy consumption



The issue of energy consumption is always of great importance, and especially in the IoT system. If we compare the power consumption of MQTT and HTTP, then HTTP “eats” much more. And CoAP is more energy efficient than MQTT, allowing you to manage power. Moreover, in simple scenarios, MQTT is more suitable for exchanging information on the Internet of things, especially if there are no restrictions on power.



Another experiment, which compared the capabilities of AMQP and MQTT on a test bench for a mobile or unstable wireless network, showed that AMQP offers more security options, while MQTT is more energy efficient.



Security



Security is another critical issue raised when studying the topic of the Internet of things and foggy / cloud computing. The security mechanism is usually based on TLS in HTTP, MQTT, AMQP and XMPP, on or DTLS in CoAP, and also supports both versions of DDS.



TLS and DTLS begin with the process of establishing a connection between the client and server sides to exchange supported cipher suites and keys. Both parties negotiate kits to ensure that further communication takes place in a secure channel. The difference between the two is in small modifications that allow UDP-based DTLS to work over an unreliable connection.



In test attacks on several different TLS and DTLS implementations, it turned out that TLS did a better job. The attacks on DTLS were more successful due to its tolerance for errors.



However, the biggest problem with these protocols is that they were not originally designed for use in IoT and did not involve work in fog or cloud. Through coordinated handshaking, they add extra traffic with each connection that depletes computing resources. On average, there is an increase of 6.5% for TLS and 11% for DTLS in the workload compared to communication without a security level. In resource-rich environments that are typically cloud- based, this will not be a problem, but this becomes an important limitation between the IoT and the fog level.



What to choose? There is no definite answer. MQTT and HTTP seem to be the most promising protocols, as they are considered relatively more mature and more stable solutions for IoT in comparison with other protocols.



Unified Communications Protocol Solutions



The practice of a single-protocol solution has many drawbacks. For example, a protocol that satisfies a limited environment may not work in a domain that has strict security requirements. With this in mind, it remains for us to discard almost all possible solutions based on one protocol in the Fog-to-Cloud ecosystem in IoT, except for MQTT and REST HTTP.



REST HTTP as a single-protocol solution



There is a good example of IoT-to-Fog REST HTTP request and response interactions: an intelligent farm . Animals are equipped with wearable sensors (IoT client, C) and controlled via cloud computing by a smart farming system (Fog server, S).



The title of the POST method indicates the resource to change (/ farm / animals), as well as the HTTP version and content type, which in this case is a JSON object representing the livestock farm that the system should manage (Dulcinea / cow). A response from the server indicates that the request was successful by sending an HTTPS 201 (resource created) status code. The GET method should indicate only the requested resource in a URI (for example, / farm / animals / 1), which returns the JSON representation of the animal with this identifier from the server.



The PUT method is used when you need to update a specific resource record. In this case, the URI is indicated in the resource for the parameter to be changed and the current value (for example, indicating that the cow is currently walking, / farm / animals / 1? State = walking). Finally, the DELETE method is used equally for the GET method, but simply deletes the resource as a result of the operation.



MQTT as a single-protocol solution







Take the same smart farm, but instead of using REST HTTP, we use the MQTT protocol. The local server with the Mosquitto library installed acts as a broker. In this example, a simple computer (referred to as a farm server) Raspberry Pi serves as the MQTT client, implemented through the installation of the MQTT Paho library, which is fully compatible with the Mosquitto broker.



This client corresponds to the IoT abstraction layer representing a device with detection and computing capabilities. The intermediary, on the other hand, corresponds to a higher level of abstraction, representing the computing node of the fog, which is characterized by large capacities in terms of data processing and storage.



In the proposed smart farm scenario, the Raspberry Pi connects to the accelerometer, GPS, and temperature sensors and publishes data from these sensors in the fog node. As you probably know, MQTT treats topics as a hierarchy. One MQTT publisher can post in a specific set of topics. In our case, there are three of them. For a sensor that measures the temperature in an animal barn, the client selects a theme (animalfarm / shed / temperature). For sensors that measure GPS location and animal movement through the accelerometer, the client will publish updates (animalfarm / animal / GPS) and (animalfarm / animal / movement).



This information will be transmitted to the broker, who can temporarily store it in a local database in case another interested subscriber appears later.



In addition to the local server acting as a MQTT broker in the fog and to which Raspberry Pi, acting as MQTT clients, send data from sensors, there may be another MQTT broker at the cloud level. In this case, the information transmitted to the local broker can be temporarily stored in the local database and / or sent to the cloud. The foggy MQTT broker in this situation is used to associate all the data with the cloud MQTT broker. With this architecture, a mobile application user can be subscribed to both brokers.



In the event of a connection failure with one of the brokers (for example, cloud), the end user will receive information from another (foggy). This is a feature of combined fog and cloud computing systems. By default, the mobile application can be configured for the first connection to the foggy MQTT broker, and in case of failure, to connect to the MQTT broker in the cloud. This solution is just one of many in IoT-F2C systems.



Multiprotocol Solutions



Single protocol solutions are popular because of their easier implementation. But it is obvious that in IoT-F2C systems it makes sense to combine different protocols. The point is that different protocols can work at different levels. Take, for example, three abstractions: IoT, fog, and cloud computing levels. IoT devices are generally considered limited. For this review, let's look at the IoT levels as the most limited, the cloud the least limited, and the fog calculation as “somewhere in the middle”. Then it turns out that between IoT and fog abstractions, the current protocol decisions include MQTT, CoAP, and XMPP. Between fog and cloud, on the other hand, AMQP is one of the main protocols used along with HTTP REST, which due to its flexibility is also used between IoT and layers of fog.



The main problem here is the interoperability of the protocols and the simplicity of translating messages from one protocol to another. Ideally, in the future, the architecture of the IoT system with cloud and fog resources will be independent of the communication protocol used and will provide good interoperability between different protocols.







Since this is not so at the moment, it makes sense to combine protocols that do not have significant differences. To this end, one potential solution is based on a combination of two protocols that adhere to the same architectural style, REST HTTP and CoAP. Another proposed solution is based on a combination of two protocols that offer publishing-subscription interaction, MQTT and AMQP. Using close concepts (both MQTT and AMQP use brokers, CoAP and HTTP use REST), simplifies the implementation of these combinations and requires less integration effort.







Figure (a) shows two models based on request-response, HTTP and CoAP, and their possible placement in the IoT-F2C solution. Since HTTP is one of the most well-known and adapted protocols in modern networks, it is unlikely that it will be completely replaced by other messaging protocols. Among the nodes representing powerful devices that are between the cloud and fog, HTTP REST is a smart solution.



On the other hand, for devices with limited computing resources that communicate between fog levels and IoT, it is more efficient to use CoAP. One of the great advantages of CoAP is actually its compatibility with HTTP, since both protocols are based on REST principles.



Figure (b) shows two publication-subscription interaction models in one scenario, including MQTT and AMQP. Although hypothetically both protocols can be used for communication between nodes at each level of abstraction, their position should be determined based on performance. MQTT was developed as a simplified protocol for devices with limited computing resources, so it can be used for communication between IoT and fog. AMQP is more suitable for more powerful devices that would ideally position it between the nodes of fog and cloud. Instead of MQTT, IoT can use the XMPP protocol, as it is considered lightweight. But it is not so widely used in such scenarios.



findings



It is unlikely that one of the considered protocols will be enough to cover all communication in the system, starting from devices with limited computing resources and ending with cloud servers. The study showed that the two most promising options that developers use more often are MQTT and RESTful HTTP. These two protocols are not only the most mature and stable, but also include many well-documented and successful implementations and online resources.



Due to its stability and simple configuration, MQTT is a protocol that over time has proven its excellent performance when used at the IoT level with limited devices. In parts of the system where limited communication and battery consumption are not a problem, for example, in some areas of fog and most cloud computing, RESTful HTTP is an easy choice. CoAP should also be taken into account, as it is also developing rapidly as an IoT messaging standard, and it is likely that in the near future it will reach a level of stability and maturity similar to MQTT and HTTP. But the standard is being developed now, which is associated with short-term compatibility issues.



What else is useful to read on the Cloud4Y blog



→ The computer will make you tasty

→ AI helps study animals in Africa

→ Summer is almost over. Almost no data leaked

→ 4 ways to save on backups in the cloud

→ On a single federal information resource containing population information



Subscribe to our Telegram channel so as not to miss another article! We write no more than twice a week and only on business.


More articles:


All Articles