At the beginning of 2020, we realized that our platform was stable in its functionality and complete in its logic — we finally released all features we initially planned three years ago and were ready to move on.
Our daily work mostly consists of optimizing existing functionality, making it more CPU/RAM/IOps-efficient and we are constantly enhancing inbound flows of telemetry data by introducing more and more integrated protocols and devices. Optimization is a great thing — it is a foundation for everything that comes later, but we still want to move further and not only optimize but develop new interesting features. At the same time, we do not want millions of features — this is usually inherent to mature products — too many features and their complexity often make it increasingly hard to use the product. And we want flespi to be as simple as possible for most of our users. That’s why we introduced the plugins mechanism.
The need for plugins
flespi is everything about message storage and processing. A generic in-depth overview of what flespi does with messages and how you can access them is located here. Messages data flow can be described with this simple diagram:
flespi user is an application that consumes messages from telematics devices in this or that form. The content of the telematics message is shaped as follows: a source message comes from a telematics device located somewhere in the field directly into the flespi channel and the flespi channel generates a JSON representation of the message using the pvm technology. Then the message can be stored in the flespi database, can update the last state of the device (we call it device telemetry), or can be streamed to the flespi user for further processing.
With this pipeline, our users do not have much control over the message content. The only field they could customize was the device name present in each message and specified in a special device.name parameter. And nothing else. We are very restrictive here to make everything very simple.
But part of our users wants more. And the demand is growing. Almost every day we have more and more new projects that want to inject their own custom data into the pipeline and complement message JSON with their own keys. From the very beginning we provided special tools to achieve this functionality:
But we want to make it simpler, to make it an integral part of the platform, not a band-aid. And this is the way we’ve taken with plugins — their primary mission is to inject user-defined properties and fields into the standard flespi pipeline process.
How to use plugins
To start using a plugin you need to create one in the Telematics Hub -> Plugins section:
Initially, we started from plugins for devices but later we also plan to enhance other types of flespi entities with plugins. For example, for channels, it may be possible to specify a list of message parameters to add/remove from default message with own parsing logic or mathematics.
The device plugins can be used to:
Store custom information in devices and use it in REST selectors. For example, add your customer id or phone number field to all devices.
Publish custom information into device telemetry.
Add custom information into JSON message representation that is published into MQTT and streams. Thus your system will be able to receive some special device attributes or operate based on this data. We are using this for implementing a tollbg plugin and stream. Another use case for this is to stream multiple devices under the same identifier.
Add custom information into JSON message representation that is stored in the flespi database. This allows you to post-process your own data with flespi analytics — build reports and trigger notifications.
Plugins for devices define the schema of fields, how to validate field values, and what to do with them on different stages of the flespi message pipeline process.
For example, we defined a “my plugin” plugin for devices with customer_id field that is a positive integer, a required field, and will be published to MQTT and streams:
After creating the plugin we can use plugins REST API or flespi.io GUI to attach devices to the plugin with the specified fields:
The good thing here is that flespi validates field data according to the provided schema — you cannot specify invalid data for the fields or change plugin specification in such a way that can corrupt the device data integrity.
The current configuration of devices attached to the plugin can be retrieved via the REST API or by subscribing to flespi/state/gw/plugins/+/devices/+ MQTT topic.
How much do plugins cost
Plugins add significant value and great flexibility to the flespi pipeline, so there is a price tag associated with them after a certain point.
You can create one plugin with the Free flespi account (no extras possible) or two plugins with the Commercial account (included in the base price). Each extra plugin for the Commercial plan will cost $20/month.
Remember, that you can assign multiple devices to one plugin, so even with one or two available out of the box you can complement your entire fleet with extra parameters.
Plugins for devices are just the beginning of a new era of user-defined extensions in flespi. One of the interesting things we have in mind is not only to calculate expressions or do parsing in place but to generate MQTT events and wait asynchronously for the additional message data from a third party. Then we will be able to inject data from remote locations into the flespi message pipeline, which will open a whole new world of advanced applications for us and for you.