Analytics basics

A comprehensive guide on the key concepts and principles of the flespi analytics engine.

The main goal of analytics is to aggregate incoming messages into intervals based on specific (user-defined) logic.

Intervals

Intervals contain aggregated metrics (configured in interval counters) calculated from sets of messages meeting the criteria specified in the interval selector(s) of the calculator.

flespi analytics intervals scheme

Here’s a sample interval JSON:

flespi analytics interval json

Calculators

Calculators carry all the logic needed for interval generation.

Calculators can be nested — one calculator can operate on the intervals already generated by another calculator.

To create a new calc, open the flespi panel -> Telematics hub -> Calculators -> “+” button.

flespi panel create new calc

Calculator templates

Before we go into detail with calculator configuration in the sections below, there is a shortcut way to create a calculator — template. We have prepared a few calculator templates for the most popular usage scenarios. If you find the right one for your needs, you can save some time.

calculator templates

The templates dialog will show when you click on the "+" button to create a new calculator (you can check the "don't auto-offer templates anymore" option to prevent this from happening again). You can always open the templates list by clicking the Templates button at the top of the new calculator dialog.

Interval selectors: criteria for data aggregation

The criteria for selection can be:

  • A logical expression
    flespi analytics expression selector

  • A period of time to split intervals by
    flespi analytics datetime selector

  • A geofence — e.g. a circular, corridor-shaped, or polygonal zone (up to 1024 points)
    flespi analytics geofence selector

  • An inactivity period exceeding the specified threshold value (usually used to detect the device going offline or losing connection):
    inactive interval selector

  • Intervals or gaps between them generated by another calculator. For example you can define parking interval as a gap between trips:

Important! You can create up to three selectors for a calculator. The conditions will be combined using AND logic.

Example: find messages with speed>0 to detect trips

Say we have the following sequential messages from the tracking device:

Consequent device messages received by flespi (simplified for demo purposes)

t=1

speed: 5

battery: 99

t=2

speed: 20

battery: 90

t=3

speed: 37

battery: 85

t=4

speed: 54

battery: 79

t=5

speed: 9

battery: 73

t=6

speed: 0

battery: 66

t=7

speed: 25

battery: 61

t=8

speed: 80

battery: 54

And the following interval selector:

flespi expression selector example

The analytics engine will compare the speed parameter in each message against “0” and while the condition holds true, it will assign these messages to the same interval. In our example, the only finished interval will include messages with t=[1..5]:

Completed interval (based on "speed>0" selector)

t=1

speed: 5

battery: 99

t=2

speed: 20

battery: 90

t=3

speed: 37

battery: 85

t=4

speed: 54

battery: 79

t=5

speed: 9

battery: 73

t=6

speed: 0

battery: 66

t=7

speed: 25

battery: 61

t=8

speed: 80

battery: 54

Since messages with t=9 and further have not arrived yet, the interval including messages with t=[7,8] will have the state “ON” meaning that the interval can continue.

Fine-tuning selectors: additional settings to customize behavior

Use these settings to make the configuration of the selector more precise.

flespi calculator selector settings

  • min_active — skip intervals shorter (in seconds) than the specified value.

  • max_active — start new interval when specified interval duration is exceeded.

  • max_inactive — merge intervals if the period of inactivity between them is less than the specified number of seconds.

  • min_duration — save intervals of the specified minimum combined duration; shorter intervals will be skipped.

  • max_messages_time_diff — maximum timestamps difference between two messages to include into the interval and apply merge_xxx options (see below).

  • merge_message_after/merge_message_before — extend the interval by one message with OFF/UNKNOWN state in the respective direction. Usually used together with extend=true in GET /gw/devices/{selectors}/intervals REST API call.

  • merge_unknown — append message with the unknown state to the active interval.

  • invert — invert the result of interval state evaluation (e.g. ON will become OFF).

Interval counters: what data to include

Since the interval contains aggregated metrics, you should specify the exact metrics you need. This is done in the interval counters. Below are the types of counters you can configure.

Expression

Applies a specific expression against all messages in the interval.

Example: calculate engine hours based on din2 duration

flespi calculator expression counter

Note: din parameter is a bitmask; you can learn more about working with bitmasks here.

Example: how much the battery drained during the interval

flespi calculator expression counter difference

Dataset

Include an array of name:value pairs to the interval.

Example: save all points in the interval

flespi calculator dataset counter

Route

Accumulate position.latitude and position.longitude parameters from each message in the interval into the Google polyline format. Use these routes in your solution.

You can check the “route” counter result in intervals Toolbox by clicking the map icon next to the specific interval:

flespi intervals toolbox route

Example: get the route for your trip

flespi calculator route counter

Datetime

Add start or end time of the interval in the text format (e.g. "%Y-%m-%d"). See the list of available formats here.

Example: add the trip start time into the interval

flespi calculator datetime counter

Parameter

Add the parameter value from the message into the interval. You can add the first, last, each or distinct value. With ‘each’ an array of values will be added. With 'distinct' you may enumerate visited geofences, activated DTC codes, tracked BLE beacons and so on during the interval.

Example: save the battery level at the end of the trip

flespi calculator parameter counter

P.S: You may also use counter type "parameter" to add device.id, ident or any other parameter from device message to the interval JSON generated by calculator. It will allow you to retrieve intervals from multiple devices in a single API call and distinguish to which device each specific interval belongs to.

Interval

Use reference to the value of another counter in the given interval in an expression. Note: counters used in the interval expression should be defined above it.

Example: calculate average speed based on mileage and duration

flespi calculator interval counter

Active

Returns the state of the current interval — “true” if active (unfinished), “false” if non-active (finished).

Example: indicate if the given interval is active

flespi calculator active counter

Geofence

Saves the name of the triggered geofence into the interval. Note: use only with the corresponding geofence selector.

Example: save the name of the geofence in/out in the interval

Here’s the “warehouse” geofence selector:

flespi calculator geofence selector

And here’s the geofence counter adding the “warehouse” value if entry to the respective geofence was detected:

flespi calculator geofence counter

Specified

Saves the value under the specified name into the interval.

Example: add “delivered” to the new_delivery field if inside geofence and speed=0

The selectors will be:

flespi calculator circular geofence selector

And the counter to add the specific value looks like this:

flespi calculator specified counter

P.S: You may also use counter type "specified" to add calculator id or name to interval JSON. It will allow you to retrieve intervals from multiple calculators in a single API call and distinguish to which calculator each specific interval belongs to.

Accumulator

Adds up the values of a numeric counter across multiple intervals.

Example: sum up daily mileage values into the monthly mileage

flespi calculator accumulator counter

Calculator

Adds an array of intervals generated by another calculator within the time frame between the begin and end times of the given interval. You can also calculate and add aggregated data from intervals.

Example: add the list of stops for each trip

flespi calculator counter

Assigned devices: source of data for calcs

Once the calculator is created, it needs a source of messages to apply its algorithms to. To provide such a source, you should assign a device (or several devices) to the calculator. Open the Devices tab on the calculator card, click the green “+” button, and select the proper devices from the list:

flespi analytics assign device to calc

After you assign a device, the calculator will automatically try to generate intervals based on the available messages. Calculation and synchronization take place in the background and may take up to one minute to complete. The green check means that everything went smoothly:

flespi analytics assigned device synced

If it won’t sync (you see the orange  icon), try disabling and enabling the calc.

Note: as new messages for the assigned device come, the intervals will recalculate automatically — no need for any actions from your side.

Toolbox: checking the generated intervals

You can see the generated intervals by clicking on the Toolbox icon on the assigned device card:

flespi assigned device card open toolbox

In Toolbox, you can see all generated intervals, messages, and interval details including all parameters, its JSON representation, and a corresponding track (if the interval contains a “route” counter):

flespi analytics intervals toolbox

Analytics API: extracting intervals from your solution

Ultimately, you want the interval data from flespi to end up in your app. The options to deliver the data where it’s expected are as follows:

  • REST API — a comprehensive way to manipulate calculators, intervals, assigned devices, and logs. E.g. here’s the request to get intervals from a particular calculator for a particular device:
    curl -X GET  --header 'Accept: application/json'   'https://flespi.io/gw/calcs/{calcs.selector}/devices/{calc.devices.selector}/intervals/{calc.device.intervals.selector}'
    You may also attach your own data to these intervals with PUT https://flespi.io/gw/calcs/{calcs.selector}/devices/{calc.devices.selector}/intervals/{calc.device.intervals.selector}/{interval-id} call.

  • MQTT API — includes two types of topics:

    • related to interval events — to get the last calculated interval of a certain calculator for a certain device subscribe to flespi/state/gw/calcs/{calc-id}/devices/{device-id}/last. The interval in the topic will update automatically as the new fresher intervals are generated.

    • related to calculator state changes — you can also subscribe to events (created, deleted, updated) triggered by the interval state change using the following topic: flespi/interval/gw/calcs/{calc_id}/devices/{device_id}/{event}.

  • Webhooks — to deliver analytics events to your application using HTTP request from flespi to your lambda.

Find more MQTT topics for analytics here.


To continue you may watch an analytics overview video from our conference or check different analytics use cases in our KB.


See also
How to set up a calculator using the flespi analytics engine to aggregate device messages by day and count the total number of messages.
Using advanced intervals aggregation capabilities to solve non-trivial tasks with flespi calculators.