11 July, 2019

Custom naming scheme for MQTT message parameters [UPD: June 13, 2023]

Map original MQTT message parameters to any notation.

MQTT channel can be set up to receive messages directly from any MQTT broker. The message payload should be a valid JSON object (or array of objects) with only one requirement — there should be an ident value somewhere in the JSON payload or in the message topic. Other than that, you can put any data in.

One problem with the data you send is that it will most likely have parameter names different from the ones you want to use in your solution (e.g. different message sources publish messages in different custom formats, and you want to force them to a unified naming scheme). We decided to take the conversion task off your shoulders (after several clients faced with it).

We have implemented the possibility to define a custom naming scheme for message parameters in the mqtt channel configuration. It means that you create a mapping of parameters from the original MQTT message into your preferred naming. You can refer to the value sitting deep inside an original complex object and map it to a new parameter name (that will be a part of the resulting plain message). The value of the resulting parameter can be of any type, even a complex one (e.g. an array or object).

FYI, we've later implemented the possibility to define a custom naming scheme for  the HTTP channels as well.

Mapping configuration

To configure the mapping, open the dialogue for creating or modifying a new channel in the flespi panel, pick the MQTT protocol, and find the Message parameters field at the bottom of the configuration object:

flespi mqtt channel configuration

Note: if you put the URI of the flespi MQTT broker, you must paste a valid flespi token into the Username field.

In manual mode, you can manually specify the custom parameter name and point to a parameter value at the specific value path, e.g.:

mqtt channel parameter mapping manual

In JSON path mode, you can specify the path to a parameter name and the path to the parameter value, e.g.:

mqtt channel parameter mapping value path

You can map up to 128 MQTT message parameters. 

Parameters are applied to the original message in the listed order, so it’s possible to add several parameters with the same name but different value paths (e.g. value1 and value2). This allows for the following logic: if value 1 and value 2 are available, value2 will be mapped; if only value1 is available, map value1 (fall-back value).

flespi mqtt channel message parameters fall back scheme

Name field is the resulting parameter name in the channel message and value path is where to take the value of this parameter, e.g.:

Name: "timestamp"              Value path: "/metadata/time"

Name: "position.latitude"    Value path: "/payload_fields/gps/1/latitude"

etc.

The value of "timestamp" property can be a number with Unix time or UTC time string in full ISO 8601 format (YYYY-MM-DDThh:mm:ss).

The value path can point to any level in the current message parsed as a JSON object.

When the Required flag is ticked for the mapping, each incoming message will be checked to the presence of the given parameter. If the Skip invalid messages option is enabled in the configuration, messages without the given parameter will be skipped, otherwise, they will generate a channel error.

Important! Pay attention that if the Message parameters option is enabled, all messages will go through this scheme, therefore, you will have to specify all parameters you want to receive, otherwise, you may be losing some data.

Possible uses

  • The discussed feature allows the conversion of the received message parameters to the flespi-friendly notation. This enables seamless use of different flespi tools, e.g. to use TrackIt and visualize the device on the map you will need timestamp, position.latitude, and position.longitude parameters.

  • Another possibility is to switch parameters to a custom naming scheme using the fact that channel messages are also published to another topic — flespi/message/gw/channels/#. This way you can convert the message content to the desired format and forward it to a third-party platform without any additional scripts. Flespi platform offers streams to send data to major cloud platforms (e.g. Google IoT Cloud, Amazon AWS, Microsoft Azure) and in generic formats (such as HTTP and MQTT).

  • You can also minimize message size and, consequently, storage by listing only the required parameters. The unlisted parameters will be omitted.

***

Having the freedom to name the MQTT message parameters the way you want gives vast opportunities for adapting the naming notation to the needs of your solution without extra coding efforts on your side. Don’t forget about this helpful feature next time you embark on an MQTT-based project.