28 March, 2019

How MQTT can help tailor messages for your telematics projects

Transforming a telemetry message to the custom format via an MQTT-powered pipeline.

Rarely will your application need all the wealth of data GPS trackers collect. On the contrary, a solution for a specific business scenario will rely on a handful of parameters smartly processed to generate valuable insights. However, some additional parameters may come from other sources (sensors, platforms, equipment) and it would be helpful to aggregate them in a single object. It may be something as simple as tagging a specific device with the customer.id field or converting a parameter into the sensor value that may come in different formats from different types of equipment; or involve a complex multi-step logic.

Let us tell you about the MQTT-based pipeline that can perform custom message transformations to fit your project’s requirements.

MQTT entities involved

MQTT broker

Once the data from devices flows into the flespi platform through a channel, it gets into the message bus and becomes immediately available via MQTT API (see below). If message TTL > 0 (meaning that the message should be stored in flespi for a specified period), the data is saved into a container and becomes available via REST API. Hence, if your project does not imply storing messages (TTL=0), you access the data via MQTT API or forward it to specific platforms using streams (here's a detailed explanation of the ways to access data in flespi).

MQTT API

MQTT API gives you easy access to device messages, platform logs, connections information, and more. 

You get the data instantly and can filter out the desired subsets of messages juggling the hierarchical topics structure. 

MQTT channel

MQTT channel is a special type of the inbound connection capable of receiving messages directly from MQTT broker. The channel subscribes to the MQTT topic (or topics) you specify and saves the data for further processing. 

If you create flespi devices with the idents matching the ones coming in the messages to the channel, they will automatically get into respective devices storage.

Common use case

MQTT channel is handy in situations when there is a need to modify the original message coming for the tracking device, e.g. leave out certain parameters or add custom data coming from external sensors. We have a detailed example of such modification to reach GDPR compliance here

Here’s the generic scheme of the message transformation process:

flespi message customization pipeline

  • The message from a physical GPS tracker arrives at the flespi channel receiving data over the respective protocol

  • The message is saved in the virtual flespi device (if TTL>0)

  • The pipeline script subscribes to the channel (e.g. flespi/message/gw/channels/{channel_id}/+) or device (e.g. flespi/message/gw/devices/{device_id}) via MQTT

  • The script performs required data modifications — it’s open-source so you can customize it as needed. E.g. you can append some parameters which values originate from external IoT sensors.

  • The script publishes the modified message into the specific topic (e.g. custom/topic/customized)

  • The MQTT channel subscribes to this resulting topic with modified messages

  • A virtual device of the MQTT type is created for each ident (or another unique id) in the modified message

  • The target platform or application gets the data in the expected format via MQTT API and applies the intended business logic.

The algorithm may look a little bulky but it’s just because of a fine-grain explanation — it works really fast due to MQTT under the hood.

***

With the above pipeline integrated into your projects, you become in control of the content and format of the incoming data and get extended message customization capabilities.