(1) reliable and scalable business-critical IoT
application;
(2) high data throughput to meet expectations of
the end-user for responsible IoT products;
(3) lower cost of operation through efficient
hardware network and cloud resources;
(4) integrating IoT data into existing TBP
platform.
4 DISCUSSION
In the case of the testbed platform for Czech Post, the
proposed solution had to meet various requirements
intended to modernise the existing eSystem. In
general, IoT data streaming and horizontally scalable
analytical platforms intended to advance eCommerce
and eGovernment systems can be evaluated in terms
of achieved performance, security, interoperability,
and data increase before the subsequent horizontal
scaling-up eSystem reconfiguration.
Regarding the estimate on how many virtual
clients should be connected to the MQTT broker to
generate up to 150.000 messages, we found that ten
clients/represented as IoT devices worked well and
without any performance degradation.
As a limitation of this study, we have:
Experimented on a single node MQTT broker.
Set the maximum number of generated and
processed messages (up to 150.000
messages/second).
Set the zero-tolerance for message loss (not
requested by the client) since we assumed
there might be no ‘Acknowledgement’
response between some sensor network
devices and the edge gateway.
Not reported the technical details of
Middleware component, developed as the
proprietary solution, which is non-disclosable
due to commercial arrangements and
expectations between the authors and the
Czech Post as our client.
Within the scope of the project and
horizontally scalable architecture, we did not
emulate large number of IoT devices.
Regarding the single node MQTT broker
configuration that could be categorised as a low-cost
‘commodity hardware’ (see Table 1), we reported:
(1) How many sensor devices was handled in
emulations without horizontally scaling up the
platform; and
(2) The number of messages per second, without
experiencing message loss during our
experiments.
As a result of multiple “cold start” experiments
(required to avoid experimental bias associated with
memory caching), we have determined that 150.000
is the number of messages per second where there is
no message loss. Contrary to our expectations, there
was no non-linear decline in message throughput
performance (Figure 4).
We have also considered the possibility of
messaging broker components by testing HiveMQ
VerneMQ and Mosquitto but have only presented the
benchmark for HiveMQ on a single virtual node as an
edge gateway. The VerneMQ and Mosquitto brokers
were excluded from the reported case study due to
lower performance. Note that Mosquitto broker is not
designed for distributed multi-node MQTT broker
operation.
The added contribution of this study is developed
and shared as an open-source Java application
(https://github.com/stufim/mqtt-performance). For
this and related projects, the Java application is
designed to be connected to the testbed platform
architecture running on a single node MQTT broker
for platform benchmarking, using an assigned IP
address with the default port of 1883. This application
can also be used for multi-node distributed MQTT
brokers for similar projects without any major
changes. The results demonstrate that HiveMQ
showed the best throughput with no message lost in
our scenario. Moreover, it allows analysing the
interplay between docker container and container
orchestration on Kubernetes (K8s).
We found that: (1) IoT sensors connected to the
messaging broker via MQTT protocol affect its
performance. For example, MQTT explorer
(http://mqtt-explorer.com) slows down performance
by about 30% (depending on which Apache Kafka’s
topic has been connected to as a client). (2) TBP can
handle and proceed up to 150.000 messages per
second for a limited time (less than a minute) on
HiveMQ messaging broker in real-time.
The paper presents an overall design of a
streaming framework aligned with the predefined
parameters for benchmarking real-time data
processing. To ensure (near) real-time data exchange
and processing on the testbed platform, with data loss,
the individual components (Figure 2) are responsible
for: (1) event streaming, transformation, and
processing, (2) data storage to collect, store and
manage analytical processing requests and (3)
application layer. In addition, the middleware
component handles business logic, object models,