Expressions in flespi

How to efficiently use expressions in flespi

Expressions in flespi are used everywhere: in calculators, REST API selectors, MQTT topic filtering, webhooks, plugins, streams and various validators. Whenever you see a word "expression" in flespi it means an expression that is constructed in the specific flespi expression syntax which evaluates some provided item (usually this is a message in JSON) and outputs expression evaluation result over this item as a number, string, boolean or JSON.


Essentials

  • Expression operates with a JSON value (mostly object) and the result of an expression is a typed number, string, boolean, null or JSON value. An expression consists of numbers, operators, parameters and functions.

  • You construct expressions using C-style syntax with C-style operators priority and round brackets to manually control operations order with 2 exceptions: value inversion is implemented via not(X) function instead if "!" operator and conditional evaluation is implemented via "if(COND, IF_TRUE, IF_FALSE)" function instead of "COND ? IF_TRUE : IF_FALSE" construction. Other operators in expressions are mostly the same as in C.

  • If an expression contains a reference to a parameter that is not present, performs division by zero or provides incorrectly typed arguments to functions - the expression evaluation will return error. To overcome this limitation we advise always preceding parameters with a dollar sign ($). In that case, if the parameter is not present in the object, a null value will be returned instead and the expression can be evaluated further. For example, the expression "abs($speed)>=0" will always be valid and return a boolean true.

  • In the case of boolean-type expressions (various validators, filters and REST API selectors) a non-zero resulting value means the expression evaluates to true. And zero, false, null or error will be evaluated to false.

  • All numbers are internally converted to floating-point with double precision, thus limiting integers to 53 bits in size.

  • In mathematical operators automatic type casting is performed: null and false are cast to zero, true or any valid string — to one.

  • You can wrap textual values in single or double quotes - whatever is more convenient for you. For textual fields you can use comparison operators like '==' or '!=', for example: "device.name == '123456789012345'" to check the actual value. When comparing two strings, you may use wildcards, for example: "device.name == 'Telto*'". For case-insensitive comparison, use a special operator "~", e.g. "device.name ~ 'TeLTo??kA'".

  • If you need to access nested JSON objects or arrays inside some parameter, use a json function: "json(parameter-name, 'json-path')", for example "json(ble.sensors, '/0/rssi')" to access RSSI value for the first BLE sensor in ble.sensors array. If value under the path is not possible to access, it will return null.

  • Expression evaluation can be stateless or stateful. Stateful expression evaluation with previous("X") function access is available in certain applications only - such as calculators or filtering logs and messages. In stateless expressions state functions such as previous("X") will not be available or will return null.

Using Expressions

Suppose our object over which expression is applied is a typical JSON message from device:

{
    "battery.voltage": 3.938,
    "channel.id": 429,
    "device.id": 182083,
    "device.name": "Vehicle",
    "device.type.id": 744,
    "engine.ignition.status": false,
    "event.priority.enum": 0,
    "external.powersource.voltage": 12.64,
    "position.direction": 281,
    "position.latitude": 43.955218,
    "position.longitude": 37.661918,
    "position.speed": 0,
    "position.valid": true,
    "protocol.id": 14,
    "server.timestamp": 1678346072.792816,
    "timestamp": 1678346071,
    "vehicle.mileage": 36713.703,
    "ble.beacons":[
      {"battery.voltage":0,"id":"08351B002203","index":1,"model":"MLD BLE TPMS (ATP100/ATP102)","name":"RF","status":true,"temperature":19,"timestamp":1706872284,"tire.pressure":235},
      {"battery.voltage":0,"id":"08351B002119","index":2,"model":"MLD BLE TPMS (ATP100/ATP102)","name":"RR","status":true,"temperature":13,"timestamp":1706872135,"tire.pressure":245}
    ]
  }

Its parameters, for example "battery.voltage", "device.name", "timestamp" can be referenced inside the expression by the parameter name. There are 2 options accessing parameter value inside expression:

  • Directly by name, e.g. ‘channel.id’ - will evaluate to parameter value or return an error if parameter is not defined;

  • Name with preceding ‘$’ sign, e.g. ‘$channel.id’ - will evaluate to parameter value or to null if parameter is not present;

We suggest always reference parameters with preceding $ to prevent error generation when some parameter is not present in the object.

Examples of expression evaluation over the object with fields above:

  • "protocol.id + channel.id": will evaluate to "14 + 429" => 443;

  • "(position.direction - 81) / 100": will evaluate to "(281 - 81) / 100" => 2;

  • "if(exists(‘position.speed’), position.speed, 0)": will use function if(condition, when-true, when-false) and evaluate to "if(true, 1, 0)" => 1;

  • "protocol.id & 1": will use binary AND and evaluate to: "14 & 1" => false;

  • "protocol.id && 1": will use logical AND and evaluate to: "14 && 1" => true;

  • "device.something": will try to access the parameter which is not exists in the object and evaluate to error;

  • "$device.something + 100": will try to access the parameter which is not exists in the object and evaluate to: "null + 100" => 100;

  • "$device.id + 100": will evaluate to: "182083 + 100" => 182183;

  • "device.name == ‘eh’": will use case-sensitive wildcards for string match (you may use single-quoted or double-quoted strings inside expressions) and evaluate to "‘Vehicle’ == ‘eh’" => true;

  • "device.name ~ ‘v*’": will use case-insensitive wildcards for string match (~ is a special operator used in flespi expressions responsible for case-insensitive comparision) and evaluate to "‘Vehicle’ ~ ‘v*’" => true; 

  • "json(ble.beacons, ‘/0/id’) + json(ble.beacons, ‘/1/id’)": will access data inside ‘ble.beacons’ JSON array specified by JSON-path and after that concatenate 2 strings. It will evaluate to: "‘08351B002203’ + ‘08351B002119’" => "08351B00220308351B002119";

  • '"not(engine.ignition.status)": will evaluate to "not(false)" => true;

You can construct and test your own expressions with the Expressions Testing tool.

Using expressions in REST API selectors

To use expressions in REST API selectors you wrap it in curly brackets: "{your expression}".

Parameters that are available to you are fields of the item that is currently analyzed - for example for REST API methods over devices it will be device fields, for REST API methods over calculators it will be calculator fields. You can list all fields available as parameters in the expression by checking ‘fields’ parameter for the corresponding REST API call selector.

In expressions inside REST API selectors you can use all functions including telematics functions such as distance(lat1, lon1, lat2, lon2). However stateful functions such as mileage() and previous("X") will return zero and null accordingly as there is no state between expression evaluation in REST API selectors.

Few examples of expressions in REST API selectors for device API methods:

Using expressions in plugins

Usually you are using expressions in the "validate_message" field of plugin configuration to apply the plugin only to device messages that conform to the expression. Also msg-expression plugin allows you to create or update device message parameters.

In expressions inside plugins you have access to all functions including telematics ones, such as mileage() which for messages with the latest timestamp will return the mileage device traveled from the position currently stored in telemetry.

Stateful functions such as previous("X") are also available in expressions inside plugins. However previous parameter values are provided directly by the device telemetry, not from the message that was previously registered for this device. Only telemetry parameters that were registered by the message with a timestamp that is older than the timestamp of the currently registering message will be accessible with the previous("X") function. Due to this, for any re-registered message previous("X") will return null.

If there are geofences assigned to the plugin you can test if the message is inside or outside of any such geofence with "geofence() != null" or "geofence() == null" expressions.

Using expressions in streams

Usually you are using expressions in the "validate_message" field of stream configuration to stream only those device messages that conform to the expression.

If there are geofences assigned to the stream you can test if the message is inside or outside of any such geofence with "geofence() != null" or "geofence() == null" expressions.

Functions that depend on the state state, such as previous() or distance() are not available in streams and will return null.

Using expressions in calculators

Expressions in calculators are used everywhere - in message and interval validators, selectors, counters.

Except in very rare cases all expressions in calculators are stateful and with the previous("X") function you can access parameters from the device message with an earlier timestamp. However in counters for each new interval detected by selectors the state is reset and for the very first message in the interval previous("X") function will always return null.

If there are geofences assigned to the calculator you can test if the message is inside or outside of any such geofence with "geofence() != null" or "geofence() == null" expressions.

Using expressions in webhooks

In webhooks you use expressions to apply webhook only for certain MQTT messages. Also expressions are widely used to format webhook URI, headers and BODY.

There is no state in webhooks between each MQTT message processing. That’s why functions such as previous("X") are not available there. Also telematics functions such as distance(lat1, lon1, lat2, lon2) can not be used in webhooks.

For templating you wrap expressions in % characters: %your-expression%. We also recommend wrapping it additionally into curly brackets to allow the use of % character inside expression: %{your-expression}%. For example to add local time to the webhook use the following template: %{strftime(timestamp, "%h:%m:%s")}%.

Operators

You can use mathematical operators in expression and add brackets to control the order of operations. The priority of operations is the same as in the C programming language and notation, in general, is also very similar to what the C language defines. You can use the following operators:

operator

explanation

+sum
-diff
/divide
*multiply
%
reminder
|binary OR
&binary AND
&&logical AND
||logical OR
^binary XOR
==equal; for strings will match with wildcards
!=not-equal; the inverted value of equal operator
~case insensitive match of a string with wildcards
<less
<=less or equal
>greater
>=greater or equal
>>right shift by the specified amount of bits (in the range from 0 to 64)
<<left shift by the specified amount of bits (in the range from 0 to 64)

Functions

Inside expressions you may call functions. All function arguments are strongly typed and if you attempt to call a function with invalid type of argument expression will return an error. Some function arguments are optional, this is specified with square brackets in function specification.

Math functions

function
explanation
abs(X)an absolute value of X
round(X)round X to the closest integer
ceil(X)round X up to the closest integer
floor(X)round X down to the closest integer
sqrt(X)the square root of X
min(X, Y)return a minimum value between X and Y
max(X, Y)return a maximum value between X and Y

Time functions

function
explanation
month(X)month value for X with ranges 1-12, where X is UNIX timestamp, e.g. month(timestamp) will return month number for the current message
weekday(X)return day of week value for X with ranges 1-7, where X is UNIX timestamp.
day(X)day of month value for X with ranges 1-31, where X is UNIX timestamp
hour(X)hour for X with ranges 0-23, where X is UNIX timestamp
minute(X)minute value for X with ranges 0-59, where X is UNIX timestamp
strftime(X, "Y")format date into text, where X is UNIX timestamp and Y is a format string similar to strftime - e.g. "%Y-%m-%d %H:%M:%S".
now()current time as UNIX timestamp with microseconds granularity

Text functions

function
explanation
hex(X[, Y, Z])convert string value X from hexadecimal format into 64-bit unsigned integer and take Z bits starting from Y. If Z is not defined, all bits will be taken, if Y is not defined, zero is assumed (first bit).

Samples: 
  • hex("FF") will return 256.
  • hex("08", 0, 2) will return 0.
  • hex("F8", 2, 2) will return 2.
strescape(X)escape all control and printable characters in string X with backslash.
urlencode(X)URL-encode string X.
Function is useful for generating correct arguments with complex contents in webhook calls.

JSON functions

function
explanation
json(X, "Y")from JSON object or array X fetch the value from the sub-element specified by json path "Y" .

Sample JSON: {"x":{"y":1, "z":2}, "a":[3, 5, {"keyA":"valueA"}]}

To fetch value under key "y" inside object under key "x" use json(x, 'y') which will return 1.

To access arrays use json(a, 0), json(a, '/0') which all will return 3.

JSON path can be used as well for fetching values from complex objects/arrays: json(a, '/2/keyA') will return "valueA".

If unable to extract value under path function will return null.
json_array_count(X)return number of elements in JSON array X

Typecast functions

function
explanation
typeof(X)return type name of argument X: 'number', 'string', 'boolean', 'null', 'json'.
tonumber(X), tostring(X), toboolean(X)convert value to another type

Conditional functions

function
explanation
if(X, Y, Z)if X is non-zero, evaluate to Y otherwise evaluate to Z
exists('X')return 'true' if the specified parameter with name 'X' exists in the message and 'false' otherwise
not(X)invert argument value - if zero/false, return non-zero/true and vice versa
isnumber(X), isstring(X), isboolean(X), isnull(X), isjson(X)test if value is of the specified type and return 'true' in such case
error()generate an error and fail expression evaluation. Can be used in conditional evaluation like: 'if(position.valid, mileage(), error())'.

Telematics functions

function
explanation
previous(X)previous known value of X or null if not yet available.
This function can be used in stateful expressions (inside calculators and plugins) only.
mileage()mileage in kilometers from the previous message, calculated using position.latitude, position.longitude and position.altitude parameters.
This function can be used in stateful expressions (inside calculators and plugins) only.
distance(lat1, lon1, lat2, lon2)distance in kilometers between two points X and Y with coordinates lat1, lon1 and lat2, lon2 where latX is latitude of point X and lonX is longitude of point X
geofence()name of geofence with the highest priority in which device is currently in. To return other geofence's field than  name you may specify it in function argument, e.g. geofence("metadata.max_speed") to fetch the maximum allowed speed defined in geofence's metadata when device is in that geofence.

To test if device is in any geofence you may use this form: geofence() != null

To test if device outside of all geofences use: geofence() == null
geofences()array with the names of geofences sorted by the priority in which device is currently in. To return other geofence field than name you may specify it in function argument, e.g. geofences("name,priority,metadata.max_speed") to fetch the name, priority and maximum allowed speed defined in geofence's metadata for each geofence device is currently in.
variable('X')return latest calculated value of variable X or null otherwise. In flespi analytics variables are created using counter of type variable.
This function can be used in calculator counters expressions only.
metadata('X')return device metadata under key or JSON path 'X'.
This function is available in calculators and plugins only. In expressions within REST API selectors you can use metadata.X syntax to access metadata.

Troubleshooting

To create and test your expression we suggest using expression testing tool as a great UI interface that highlight expression syntax, suggest functions to use and can be used to test expression evaluation over custom JSON objects.

The tool is integrated in flespi in Logs and Messages viewers for devices, channels, plugins, streams, calculators, webhooks, platform logs - elsewhere.

Just select corresponding logs or messages with Ctrl or Shift click followed by Right click to show the context menu and select "Test Expression" to activate the tool with these messages. This is especially helpful for testing selector expressions in calculators.

And finally you can ask our AI Assistant (codi) to generate correct expression for your task.


See also
UI options to control the order of items received via REST API
How to set up a calculator using the flespi analytics engine to aggregate device messages by day and count the total number of messages.