How to extract trips from device messages?

Imagine that you have a task to extract trips from device messages. There are two ways to achieve it.

First option — you may fetch all device messages with GET /gw/devices/{selector}/messages REST API call, analyze them with your script to split these messages into trips/parkings. With this method, you have full control of mathematics but it can take a while since devices can easily have a few GB of messages.

Second option — use flespi analytics to generate such report with the POST /gw/devices/{selector}/calculate REST API call or by configuring calculator with corresponding configuration and assigning device to it for automatic calculation in background. This is not only faster in processing but also consumes much less traffic with only a minimum set of information sent by the platform.

We recommend to carefully read conceptual information on flespi analytics before continuing with the next steps.

There are multiple ways to detect an active trip state. In any case, the interval selector will have type="expression" but the actual expression and how to evaluate it will depend a lot on the hardware. Some devices, like teltonika, send engine.ignition.status parameter, some devices send non-zero mileage of movement from the previous point in segment.vehicle.mileage, sometimes even CAN data can be used, e.g. fuel consumption parameters. 

So the approach to trips calculation greatly depends on the device and the way you want trips to be detected. Here we will use the simplest method based on pure GPS/GLONASS satellites — detect the trip when our instant speed is above zero. And on top of this, we will add a check for extra flags in a message indicating that the current trip is on.

So the proposed interval selector configuration is:

{
    "type": "expression",
    "expression": "$position.speed>0 || $engine.ignition.status || $segment.vehicle.mileage>0",

    "merge_message_before": true,
    "merge_message_after": true,
    "merge_unknown": true,
    "max_messages_time_diff": 300,

    "max_inactive": 120,
    "min_duration": 60  
}

So the "$position.speed>0 || $engine.ignition.status || $segment.vehicle.mileage>0" expression will equal to non-zero when detected instant speed from satellites will be above zero OR when engine ignition status is ON OR when the vehicle has moved from a position in the previous message by internal tracker algorithm. We can also use the simplest expression "position.speed>0" as an option — it will skip all messages that do not contain position.speed parameter at all or just zero values from the trip. Read more about expressions and how to reference parameters.

We would also like to prepend and append ending messages with zero speed (merge_message_after and merge_message_before) so that we do not lose any point on the trip. At the same time, if the time difference between messages is more than 300 seconds, we want to break the trip and restart it again as some information may be already lost. We will ignore stops less than 120 seconds (max_inactive parameter) and require our trip final duration to be not less than 60 seconds (min_duration parameter).

For counters, we can calculate maximum speed, average speed, distance traveled based on GPS coordinates with another distance counter for mileage coming from CAN bus and actual points to render them on the map. As an alternative to an array of points, we have added route counter to dump trip coordinates in Google encoded polyline format - in general, you need just one of these counters - either points or route. Rendered points will only be present when the "position.valid" message parameter has 'true' value. Average speed for the trip calculated from interval message by dividing the distance by duration:

[
    {
        "name": "max.speed",
        "type": "expression",
        "expression": "position.speed",
        "method": "maximum"
    },
    {
        "name": "distance",
        "type": "expression",
        "expression": "mileage()",
        "method": "summary"
    },
    {
        "name": "distance_can",
        "type": "expression",
        "expression": "segment.vehicle.mileage",
        "method": "summary"
    },
    {
        "name": "points",
        "type": "dataset",
        "fields": [
            {"name": "tm", "value": "ceil(timestamp)"},
            {"name": "lat", "value": "position.latitude"},
            {"name": "lng", "value": "position.longitude"},
        ],
        "validate_message": "position.valid"
    },
    {
        "name": "route",
        "type": "route",
        "validate_message": "position.valid"
    },
    {
        "name": "avg.speed",
        "type": "interval",
        "expression": "(distance*3600)/duration"
    }
]

The resulting set of intervals will look similar to this:

{
    "begin": 1554176866,
    "end": 1554176969,
    "distance": 0.820,
    "distance_can": 0.900,
    "duration": 103,
    "max.speed": 10,
    "points": [
        {
            "lat": 47.099231,
            "lng": 17.547296,
            "tm": 1554176866
        },
        {
            "lat": 47.099385,
            "lng": 17.547104,
            "tm": 1554176899
        },
        {
            "lat": 47.099891,
            "lng": 17.546685,
            "tm": 1554176922
        },
        {
            "lat": 47.099926,
            "lng": 17.546731,
            "tm": 1554176924
        },
        {
            "lat": 47.100431,
            "lng": 17.547931,
            "tm": 1554176969
        },
        {
            "lat": 47.100431,
            "lng": 17.547931,
            "tm": 1554177030
        }
    ],
    "route": "_p~iF~ps|U_ulLnnqC_mqNvxq`@"
}

We recommend you to play with interval selectors and counters to find mathematics suitable for your needs.

Next step will be to copy this configuration to calculator in order to enable automatic intervals calculation in the background.