At the beginning of 2020, we realized that our platform was stable in its functionality and complete in its logic — we finally released all features we had initially planned three years before and were ready to move on.
Our daily work mostly consists of optimizing existing functionality, making it more CPU/RAM/IOps-efficient and we are constantly enhancing inbound flows of telemetry data by introducing more and more integrated protocols and devices. Optimization is a great thing — it is a foundation for everything that comes later, but we still want to move further and not only optimize but develop new interesting features. At the same time, we do not want millions of features — this is usually inherent to mature products — too many features and their complexity often make it increasingly hard to use the product. And we want flespi to be as simple as possible for most of our users. That’s why we introduced the plugins mechanism.
The need for plugins
flespi is everything about message storage and processing. A generic in-depth overview of what flespi does with messages and how you can access them is explained here. Conceptually, the messages data flow can be described with this simple diagram:
flespi user is an application that consumes messages from telematics devices in this or that form. The content of the telematics message is shaped as follows: a source message comes from a telematics device located somewhere in the field directly into the flespi channel and the flespi channel generates a JSON representation of the message using the pvm technology. Then the message can be stored in the flespi database, can update the last state of the device (we call it device telemetry), or can be streamed to the flespi user for further processing.
With this pipeline, our users did not have much control over the message content. The only field they could customize was the device name present in each message and specified in a special device.name parameter. And nothing else. We are very restrictive here to make everything very simple and universal.
But part of our users wanted more. And the demand was growing. Almost every day we got more and more new projects that wanted to inject their own custom data into the pipeline and complement message JSON with their own keys. From the very beginning we provided special tools to achieve this functionality:
- MQTT and HTTP protocols for channels that are capable of storing custom data;
- externally hosted flespi_pipeline based python script;
- the custom naming scheme for the MQTT channel;
- special pipe-cache-params protocol for caching some missed parameters.
But we want to make this process more organic — to make it an integral part of the platform, not a band-aid. And this is the way we followed with plugins — their primary mission is to inject user-defined properties and fields into the standard flespi pipeline process.
Possible uses of plugins
Initially, we started from plugins for devices but later we also plan to complement other flespi entities with plugins. For example, for channels, it may be possible to specify a list of message parameters to add/remove from the default message with its own parsing logic or mathematics.
At the moment device plugins empower you to can be used to:
- Store custom information in devices and use it in REST selectors. For example, add your customer id or phone number field to all devices.
- Publish custom information into device telemetry.
- Add custom information into JSON message representation that is published into MQTT and streams. Thus your system will be able to receive some special device attributes or operate based on this data. We are using this for implementing a tollbg plugin and stream. Another use case for this is to stream multiple devices under the same identifier.
- Add custom information into JSON message representation that is stored in the flespi database. This allows you to post-process your own data with flespi analytics — build reports and trigger notifications.
- Validate data received from the device and retain validated parameters value. If the message doesn't pass validation expression, the plugin won't be applied to the message at all.
- Delete specified parameters from the received messages.
- Add custom parsing based on the user-defined parsing scheme written using our special PVM language.
Plugins for devices define the schema of fields, how to validate field values, and what to do with them at different stages of the flespi message pipeline process.
flespi validates field data according to the provided schema — you cannot specify invalid data for the fields or change the plugin specification in such a way that can corrupt the device data integrity.
The current configuration of devices attached to the plugin can be retrieved via the REST API or by subscribing to flespi/state/gw/plugins/+/devices/+ MQTT topic.
How to use plugins
Plugins have a dedicated section in the Telematics Hub drop-down in the left-side menu of the flespi panel:
We address the practical applications of plugins in our dedicated knowledgebase article.
How much plugins cost
Plugins add significant value and great flexibility to the flespi pipeline, so there is a price tag associated with them after a certain point.
You can create one plugin with the Free flespi account (no extras possible) or two plugins with the Commercial account (included in the base price). Each extra plugin for the Commercial plan will cost €20/month.
Remember, that you can assign multiple devices to one plugin, so even with one or two available out of the box you can complement your entire fleet with extra parameters.
***
Plugins for devices are just the beginning of a new era of user-defined extensions in flespi. One of the interesting things we have in mind is not only to calculate expressions or do parsing in place but to generate MQTT events and wait asynchronously for the additional message data from a third party. Then we will be able to inject data from remote locations into the flespi message pipeline, which will open a whole new world of advanced applications for us and for you.