Analytics — flespi reports engine

Apply custom mathematics to extract information from device messages

Friendly advice: if you are new to flespi analytics, please check this comprehensive guide first or watch an analytics overview video from our conference. Below is more technical description of it. And if you are interested in analytics architecture under the hood please check this blog article.

How it works

Each registered device on the platform will accumulate its messages received via channels until these messages fall within the specified messages_ttl period. It is possible to access the current device state by fetching its telemetry or reading received historical messages via GET /gw/devices/{selector}/messages REST API call

Both telemetry and historical device messages operate with the parameters sent by the device in raw format — the one defined by the protocol. Let's take a look at a sample message from a device:

{
  "ident": "123456789012345",
  "timestamp": 1490347944.893743,
  "din": 9,
  "channel.id": 123,
"device.id": 345,
  "position.altitude": 568.49,
  "position.direction": 297,
  "position.hdop": 0.9,
  "position.latitude": -21.328481,
  "position.longitude": 47.562136,
  "position.speed": 0,
  ...
}

If the values you need are available directly in the message JSON, you can call GET /gw/devices/{selector}/messages with a specific filter (e.g. filter="position.speed>0") or limit data output to a list of specific parameter names you want to receive. This gives direct and fast access to device historical data.

To perform calculations with received messages, e.g. calculate the duration of an input sensor state, detect geofence ins and outs, count mileage or engine hours, and even split all messages into trips and parkings flespi database engine provides you with the Analytics system.

Our analytics system performs interval calculations automatically in the background based on calculator configuration and devices assigned to them. Once the device sends a message (even from a black box or in reverse mode), the system will recalculate intervals and their counters and merge these updated intervals with the previously calculated ones. It will also recalculate intervals if the calculator configuration changes. 

It enables you to:

  • apply custom logic or make injections to interval data from your own database
  • keep calculated device data for years with minimal storage volume for raw messages
  • have instant report results even for thousands of devices
  • handle notifications about complex device state changes: e.g. current daily mileage or parking started events

The internal algorithm used by the analytics system is quite simple. We have a range of messages and our task is to split these messages into intervals and calculate counters for each interval.

Each interval is represented as a JSON message with a set of fields, where "begin", "end", "duration", "id" and "timestamp" are always present and other fields appear only when a counter with such name is defined in the calculator configuration:

{
  "begin": 1490347944,
  "end": 1490347948,
"timestamp": 1490347950.1234,
"id": 1,
  "duration": 4,
"mileage": 190.2,
"work_hours": 0.12345,
"max_speed": 120,
"positions": [{"t":1490347944, "lat":53.0923, "lon":27.12}, {"t":1490347944, "lat":53.0923, "lon":27.12}, ...]
  ...
}

To split messages into intervals you need to define interval selectors. And to calculate information about each interval you need to specify counters.

The best way to start with analytics is to read this article and for real usage samples of applications on top of it please read corresponding articles in our blog or more technical articles in KB and blog:


See also
Apply webhooks to events from calculators to invoke your lambda upon an aggregated event happened to the device