18 February, 2020

Fill in the gaps in telemetry data via flespi pipe

Add the missing parameters in the incoming messages to ensure consistent notifications triggering.

Partial data in messages is a common pain point in fleet management since it compromises accuracy and timeliness of related operations. Oftentimes, status messages and events from sensors, panic buttons, etc. come without position data. Flespi has come up with the solution to fix this inconsistency — you can now complement incomplete messages with the missing parameter values.

The process is simple — you create a special pipe-cache-params channel, specify the source channel (or channels) where messages will come from, say which parameters have to be included in each message, and let flespi do the magic.

Below are some of the specific scenarios when using the pipe-cache-params channel may be beneficial.

Applicable cases

  • Say, the notification triggers when the sensor expression based on several parameters equates to a certain value. Then if one of the parameters in the expression is missing, the result will be invalid and you will miss your notification.

  • When the alarm goes off, it must notify the driver currently behind the wheel, thus it should know his driver ID. But the driver.ID parameter might not be present in the message that triggered the alarm. Using the pipe-cache-params channel you can add this parameter to all incoming messages thus making it readily available when needed.

  • Pressing the panic button will send the respective parameter but it will oftentimes come without position coordinates which makes it hard to figure out where the critical situation took place. Injecting the position data to all messages will easily solve the issue.

  • If position is sent in equal intervals of, say, one minute and the sensor detecting the arm of the garbage collector going up sends its parameter in the middle of this interval, it won’t be possible to determine the exact position of the vehicle.

  • The bed of a dump truck can lift only when inside a quarry geofence, otherwise the alarm will go off. To check whether the truck is inside or outside the geofence, messages from the sensor should contain the position data.

Channel configuration details

One obvious prerequisite to using the pipe-cache-params channel is having at least one other active channel that receives messages from devices.

Once you have it, create a new channel and pick the pipe-cache-params protocol_id.

Then check the configuration item and populate the Channel IDs, Token (either ACL type with access to the listed channel messages or Standard; avoid using Master token here — it's excessive), and Parameters to cache fields with valid values.

Pay special attention to the “Generate messages only when ALL listed parameters are cached” toggle — this one can dramatically affect the behavior of the channel. With this option enabled, if the cache is cleared either because it expires or our servers are rebooted or you disable and enable the channel, all listed parameters in it will be lost. Consequently, newly received messages will be discarded until ALL listed parameters are cached again.

Note: the caching of parameters is performed on a per-ident basis, which means that if earlier messages for a given device (ident) contained certain non-empty parameters, the pipe-cache-params channel will be able to inject their values into subsequent messages in place of the missing values.

flespi pipe cache params channel create

As you can see on the screenshot below, our Dataflow channel sometimes gets messages with the position data missing:

message with missing parameters

While our Pipe channel has this issue fixed seamlessly:

message with fixed missing parameters

You can specify several channels as sources for the pipe channel. In this way, you can fix missing values for the same parameters across multiple device types (and manufacturers) since they have common names. 

Note: Avoid having devices with the same ident in different channels — this will corrupt your cache for this ident and will lead to wrong data being pasted from cache.

Using with Wialon

Wialon fleet management platform relies a lot on events to trigger notifications. Messages from sensors are often missing the position data and are only sending the parameter that has changed its state. flespi offers a fix to such a situation using the pipe-cache-params channel:

fixing missing parameters in wialon scheme

The sequence of actions should be as follows:

  1. Create a flespi channel of the wialon_retranslator type to receive messages from Wialon

  2. Use a retranslator in Wialon to send data with potentially problematic messages to flespi.

  3. Create a flespi channel of the pipe-cache-params type to fix messages from Wialon. Configure it to add the missing data into the arriving messages.

  4. Create a flespi stream to forward the fixed data back to Wialon.

  5. Create a unit of flespi gateway type in Wialon. Important: add notifications that you want to benefit from position (or other missing) data for this new unit; remove or disable such notifications for the original unit receiving messages with missing parameters to avoid getting notifications without the position (or another missing) data.

  6. Never fail to trigger notifications because of the missing position data.

A more straightforward but not always viable alternative is to have the trackers initially send data to flespi, have it fixed there, and then forwarded to Wialon in all completeness. Then you will avoid the loop and duplicate units needed for the above scenario.

fixing missing parameters in flespi before wialon

This way requires re-pointing your trackers to the new IP:port, so if you have the existing infrastructure in place, it might mess up its operation.

***

The new pipe-cache-params channel by flespi offers a quick cure to the often-heard complaints about occasional parameter absenteeism. It ensures a consistent set of non-empty parameters across all incoming messages thus ensuring stable operation of notifications triggered upon sensor status change.