Automatically translate this page?

Expressions — the way to analyze message

How to use expressions to extract information from telematics message using own mathematics

Analytics system uses expressions everywhere — in interval selectors, in interval counters, and even in message and interval validators. Expression operates with a message and the result of an expression is a typed number, string, boolean or null value. All numbers are operated as floating-point with double precision thus limiting integers to 53 bits in size.

An expression consists of numbers, operators, parameters references, and functions. If an expression contains a reference to a parameter that is not present in the message or division by zero, its result is undefined or OFF in most boolean expressions. In the case of boolean-type expressions (e.g. validation or interval test expressions), a non-zero resulting value means the expression is valid and zero, false or null will deny validation of message or interval. 

For example, we sequentially analyze the following messages from devices:

{
  "device.name": "123456789012345",
  "timestamp": 1490347944.893743,
  "din": 9,
  "channel_id": 123,
  "altitude": 568.49,
  "direction": 297,
  "hdop": 0.9,
"speed": 10,
  "lat": -21.328481,
  "lng": 47.562136,
  "accelerations": [1, 2, 3.3, 0],
  ...
}

Expressions sample that can be used for such kind of messages are:

  • device.name == "12*45"
  • timestamp > 1490347940 && timestamp < 1490347950
  • (hdop < 1 && (din & 0x1)) || speed == 0
  • if(altitude > -500 && altitude < 500, altitude, error())

In an expression, all references to message parameters that contain a number, boolean, textual or null value will equal to this value. All references to other types of parameters (e.g. array, nested object or binary) will make expression evaluation impossible.

It is possible to use true, false, and null values in the expression. Type of value can be tested with functions: isnull, isnumber, isstring, isboolean and safely converted to another type with functions tonumber, toboolean, tostring.

In mathematical comparison operators, the maximum level of automatic type casting is used: null and false are cast to zero, true or any valid string — to one.

For textual parameters you can use comparison operators like '==' or '!=', for example: "device.name == '123456789012345'" to check actual value. When comparing two strings you may use wildcards, for example: "device.name == 'Telto*'". For case insensitive comparison use a special operator "~", e.g. "device.name ~ 'TeLTo??kA'".

If you need to access nested (JSON) objects or arrays, use a function json('object-key', 'json-path'). See examples and explanations in the table below.

To reference the value of the same parameter in the previous message, prepend hash (#) to the parameter name. For example, you can use "speed - #speed" to calculate the speed change from message to message. 

Important note on using references to previous message parameters: as the system does not know the value of the previous parameter for the very first message, it means that the very first message in such reference will always return an unknown or OFF value. In order to correctly and consistently calculate intervals, we recommend using merge_message_before=true in selector configuration which expression using the reference to previous message parameters.

Another form of parameters specification is by preceding them with a dollar sign ($). In that case, if the parameter is not present in the message, a null value will be put instead and expression can be evaluated further. For example "abs($speed)>=0" will be always valid and return boolean true.

You can use mathematical operators in expression and add brackets to control the order of operations. The priority of operations is the same as in the C programming language and notation, in general, is also very similar to what the C language defines. You can use the following operators:

operator
explanation
+
sum
-
diff
/
divide
*
multiply
|
binary OR
&
binary AND
&&
logical AND
||
logical OR
^
binary XOR
==
equal; for strings will match with wildcards
!=
not-equal; the inverted value of equal operator
~
case insensitive match of a string with wildcards
<
less
<=
less or equal
>
greater
>=
greater or equal
>>
right shift by the specified amount of bits (in the range from 0 to 64)
<<
left shift by the specified amount of bits (in the range from 0 to 64)

If you need more operators, please contact us

Here are some sample expressions and their resulting values:

expression
evaluation
1 + 2 * 4 / 2
5
(1 + 2) * 4 / 2
6
(4 & 1) > 0
false, because 4 AND(binary) 1 is 0
(4 && 1) > 0
true, because 4 AND(logical) 1 is 1

It is also possible to use some functions inside expressions, for example, "abs(speed-#speed)>10" will always be true if the instant speed value change between two sequential messages is different by more than 10 km/h in any direction. Here is the full list of functions:

function
explanation
abs(X)
an absolute value of X
round(X)
round X to the closest integer
ceil(X)
round X up to the closest integer
floor(X)
round X down to the closest integer
sqrt(X)
the square root of X
min(X, Y)
return a minimum value between X and Y
max(X, Y)
return a maximum value between X and Y
if(X, Y, Z)
if X is non-zero, evaluate to Y otherwise evaluate to Z
mileage()
mileage in kilometers from the previous message, calculated using position.latitude, position.longitude and position.altitude parameters
month(X)
month value for X with ranges 1-12, where X is UNIX timestamp, e.g. month(timestamp) will return month number for the current message
day(X)
day of month value for X with ranges 1-31, where X is UNIX timestamp
hour(X)
hour for X with ranges 0-23, where X is UNIX timestamp
minute(X)
minute value for X with ranges 0-59, where X is UNIX timestamp
strftime(X, Y)
format date into text, where X is UNIX timestamp and Y is a format string similar to strftime.
json('X', Y)
fetch the value from the sub-element inside a JSON object or array.

Sample JSON: {"x":{"y":1, "z":2}, "a":[3, 5, {"keyA":"valueA"}]}

To fetch value under key "y" inside object under key "x" use json('x', 'y') which will return 1.

To access arrays use json('a', 0) or json('a', '/0') which both will return 3.

JSON path can be used for as well for fetching values from complex objects/arrays: json('a', '/2/keyA') will return "valueA".
hex(X[, Y, Z])
Convert string value X from hexadecimal format into 64-bit unsigned integer and take Z bits starting from Y. If Z is not defined, all bits will be taken, if Y is not defined, zero is assumed (first bit).

Samples: 
  • hex("FF") will return 256.
  • hex("08", 0, 2) will return 0.
  • hex("F8", 2, 2) will return 2.
exists('X')
return 'true' if the specified parameter with name 'X' exists in the message and 'false' otherwise
error()
generate an error and fail expression evaluation. Can be used in conditional evaluation like: if(position.valid, mileage(), error())
not(X)
invert argument value - if zero/false, return non-zero/true and otherwise
tonumber(X), tostring(X), toboolean(X)
convert value to another type
isnumber(X), isstring(X), isboolean(X), isnull(X)
test if value is of the specified type and return 'true' in such case

If you need more functions, please contact us.


See also
A comprehensive guide on the key concepts and principles of the flespi analytics engine.