27 September, 2017

flespi streams overview: telematics data and commands sent without a hitch

Streams reliably transport telemetry data from flespi to the software platforms and commands to the devices over specific protocols
Disclaimer: flespi registry streams API has changed, so everything talking about streams in this article is outdated. This article applies an “each device has its own streams” model, i.e., streaming messages from several devices to one endpoint requires creating the same stream for each device.
We’ve got numerous requests to implement streams for channels and also to enable sending HTTP-requests with messages to the third-party platform instead of sending messages as a response to HTTP calls via our REST API. The solution was to redesign our streaming architecture. Which we did. Please meet and learn more about the brand new streams here.

According to internetlivestats.com the Internet traffic for 2017 has exceeded 1,000,000,000,000 GB which is 1 Yottabyte. Have you heard of such measure unit? Everything is connected to everything and myriad of tiny information flows permeates the space around us.

flespi streams overview

Being at the top of the Internet progress flespi team makes our contribution to the huge amount of Internet rivers, creeks, and streams. And flespi streams is what we are going to talk today about.

Having been announced in flespi registry module streams functionality was briefly covered in the how-to article showing the process of connecting a WiaTag soft-tracker to Wialon platform. But the most interesting question “What can I do with this stuff?” was left behind. Let's take the red pill of knowledge and look deeper into the flespi streams.

flespi streams concept

To feel the idea of a stream let’s visualize the data path from the tracking device to a third party platform through flespi:

  1. data comes from tracking devices to flespi via connection to the channel's listening IP:port
  2. flespi channel parses data and saves messages with corresponding identification string to a separate flespi device storage container.
  3. as soon as the message appears in the device storage container, streams handler picks it and pushes through the stream into the third-party platform.

flespi streams scheme

According to the scheme above, a stream is an instance responsible for connecting a flespi device to another platform; similarly a channel is in charge of connecting the tracking device in the real world. When commands or configuration data arrive at the stream, the flespi device forwards them to a channel. The channel's job is to send data to the connection matching the device identification string. One of the key advantages of flespi devices is that you shouldn’t worry if the tracking device or the third-party platform is online — the flespi platform will automatically update the data once the connections are established in both directions.

For now, we support four stream types:

  • wialon_ips — a generic protocol by Gurtam used mostly to connect to Wialon-compatible platforms. This use case was covered in flespi test-drive article
  • aws_iot and thingsboard – two MQTT-based streams to IoT platforms. Amazon IoT hub needs no introduction, and communication with thingsboard.io will be discussed in a separate article (coming soon). For Amazon IoT hub streams you can specify a “thing name” and an MQTT topic. Thing name will store the “shadow” data of the device, and MQTT topic is used for further internal logic. But this is also a (not MQTT) topic for future article :)
  • abque – is used to stream messages from many devices to a single flespi storage abque. For example, you have a bunch of devices operating via a single protocol, but want messages from several groups of devices to be stored in separate buffers. Sure, you can create a channel for each group of devices, but then devices in groups need to send data to different IP:port and additional channels incur extra costs. Creating a flespi storage abque for each group of devices and setting up abque streams will be a more elegant and cost-effective solution.

flespi storage abque scheme

Do you know other streaming protocols that can be useful for you and our future partners? Give us a poke, and in a little while, you will notice a new option in the stream type drop-down list.

Stay tuned for new articles!

Jan Bartnitsky
Software Engineer

Get the latest updates and monthly newsletters from flespi in your inbox

7 February, 2018 | flespi features | Anton Kulichenko

Connecting a tracker to Wialon via flespi and why you may want it

What are the benefits of putting flespi between your trackers and Wialon and how to do it? Step-by-step guide with Queclink GV300.

11 January, 2018 | flespi integrations | Anton Kulichenko

How flespi adds protocols and easy device setup to Traccar

Integration of flespi allows Traccar users to connect more tracker types, see more message parameters, and configure devices from different manufacturers in a single interface.