- What is PVM?
- AI assistance with PVM
- PVM sandbox
- Main conventions
- Structure of a PVM code
- Atomic syntactic elements of PVM
- PVM value types
- Control operators
- Math operations
- String operations
What is PVM?
PVM stands for Parsing Virtual Machine - special is a programming language to deal efficiently with protocols. Like any programming language, PVM contains syntactical constructions for variables, operators, numbers, string literals, functions, and other abstractions which you may use to define the data transformation algorithm.
PVM was designed to write declarative code. It means that PVM code says not “do this, then that”, but declares “what data we have” and “how it should look at the output”. Data conversion and transformation is a very practical task and the primary reason for PVM existence. Some cases cannot be described effectively with declarative code, that’s why even though PVM code looks declarative, it's still an imperative code. As a result, PVM has a very expressive syntax to declare data format and at the same time, it has variables, loops, conditions, etc. to be able to handle all possible data transformation cases with imperative code.
AI assistance with PVM
We are providing two AI tools to simplify your coding experience with PVM.
The simplest way to explore PVM capabilities is with the help of our AI Assistant (codi). Just open HelpBox by clicking on the CHAT button in top-right corner of your flespi.io account, enable AI Assistant and give it a task to generate some PVM code for you.
Another option is to use PVM generator tool directly. Click on TOOLS in top-right corner and than select PVM Generator and explain your task. Then click on the "Generate" button and copy generated code into the plugin or sandbox:
Generated PVM code you may test in PVM plugin sandbox. Sometimes it is wise to split your bigger message transformation task into smaller standalone pieces and feed them into AI separately. However it can be quite capable to provide you with whole big solution sometimes.
PVM sandbox
We have a convenient UI testing tool to experiment with the PVM plugin code: pvm plugin sandbox.
It helps to easily identify errors and detect discrepancies between the the data and the plugin code. This together with our AI assistant should be the primary creating tool and playground for your pvm scripts.
Main conventions
- PVM is a case-sensitive language. unset and UnSet are not the same thing
- Only English symbols can be used as identifiers. Full Unicode support is available only in "string literals"
- Nested elements are separated from the parent with increased indentation (just like in Python), and only horizontal tabulation symbol (0x09 ASCII char, \t) can be used for it
- C++-style comments: /* multiline comment */ and // one-line comment
Structure of a PVM code
Thanks to its declarative look, the PVM code can be treated as structured data in text form (like an XML or YAML file, for example). In other words, the PVM code is knowledge written in structured text form about how data should be parsed and transformed.
Here is the sample of PVM code with comment lines explaining code lines that follow after each comments block:
// take payload.text parameter if present
optional .payload.text ==> input:
// and split by commas,
// in next section each comma-separated element is retrieved with "item ==>" section
split[",", error=false]:
// store first element in sentence_id variable
item ==> $sentence_id
// switch between sentence_id variable values for different action flows
switch[$sentence_id]:
// execution flow when $sentence_id variable value is "$GPGGA"
"$GPGGA":
// ignore second element
item ==> skip
// convert third element from text to double
// and store into message as a parameter position.hdop
item ==> %double ==> #position.hdop
// convert next element from text to 1-byte unsigned integer
// and store into message as a parameter position.satellites
item ==> %uint8 ==> #position.satellites
// next is kind of constant and check
// expected that next element will always be "M",
// if not - plugin will generate parsing error
item ==> <"M">
// remove parameter payload.text from the message
unset .payload.text
// initialize variable variable1 with zero value
0 ==> $variable1
// parse hex string stored in parameter can.data.frame.1 as a binary
// hexadeciam string from value will be loaded into binary bufer with %hex transformer
// and then contents of this binary buffer will be parsed as bits
optional .can.data.frame ==> %hex ==> bits:
// skip first 8 bits
8 ==> skip
// store next 16 bits into variable variable1
16 ==> $variable1
// store value of next 4 bits multiplied by 100 into message parameter named voltage
4 ==> this * 100 ==> #voltage
// format value in $variable1 value as hex string and store into parameter data.version
format["V-%x", $variable1] ==> #data.version
// extract 4 bits from parameter din
// and store as boolean into separate parameter for each digital input
// with name din.X where X is from 1 to 4
optional .din ==> bits:
1 ==> %boolean ==> #din.1
1 ==> %boolean ==> #din.2
1 ==> %boolean ==> #din.3
1 ==> %boolean ==> #din.4
More samples of how PVM code looks like you may find in KB and blog tagged with pvm.
Atomic syntactic elements of PVM
Below is a list of common syntactic elements used to denote atomic parts of the pvm code, like variables, numbers, string literals, etc.
- Number: For example, 0, 1, -100, 3.14, 0x42
- String: One of "string literal", `fixed string literal`
- Binary: One of <"binary string with zero byte" 00>, <01 0203>
- Constant: One of true, false, null
- Word: For example, if, switch, repeat, any_word_without_spaces, pvm4parsing, camelCaseWord
- Variable: For example, $var
- Property: For example, .fixed_property.name
- Property with variable name (aka varprop): For example, {$property_name}
- Type: For example, %uint32
- Parameter: #position.latitude
- Attribute: [key="value", 2, 3, error=false]
- Label: @label_word
PVM value types
The PVM basic value types are based on JSON basic types:
- number to operate with numeric values
- string to contain fixed text or binary information
- boolean to denote a true or false value
- null to denote special null value in JSON
- JSON - JSON object or array
One of the main PVM tasks is to parse binary data, and the double value type used by device message JSON for numeric types lacks the accuracy to represent all possible 64-bit integer values as only first 53-bit are available. To overcome this limitation in device messages such 64-bit parameters should be represented as hexadecimal strings during the initial parsing of device message in flespi channel.
In PVM code you may access current values of device settings, device telemetry or metadata.
PVM code sample accessing device telemetry to detect when parameter value has been changed:
// take from device telemetry last known value of engine.ignition.status
// and add to message new parameter engine.ignition.changed if its value is changed
if optional #engine.ignition.status != optional telemetry[#engine.ignition.status]:
// param in the message is not equal to its last known value
true ==> #engine.ignition.changed
// Here is an advanced example demonstrating check for the message timestamp,
// required if your device can send messages from the black box/history:
if is_set #engine.ignition.status && optional telemetry[#engine.ignition.status].timestamp < #timestamp:
// we have "engine.ignition.status" parameter in current "fresh" message (not from black box)
if #engine.ignition.status != telemetry[#engine.ignition.status]:
// param in the message is not equal to its last known value
true ==> #engine.ignition.changed
PVM code accessing device metadata:
// store metadata value under 'field' to message parameter param.name
optional metadata["field"] ==> #param.name
if is_set metadata["field"]:
... // code will be executed only if field are set
if optional metadata["field"] == 42:
... // code will be executed only if field are set to value 42
PVM code accessing device settings:
// store setting value of named setting settingname
// with field 'field' to message parameter param.name
optional :settingname.field ==> #param.name
if is_set :settingname.field:
... // code executed only if setting field is set
Control operators
Despite the declarative look, PVM is an imperative programming language, executing its code from the top to bottom, and every chain from the left to the right (in the direction of the ==> arrows). Programmers need operators to control execution flow — condition and loop.
Condition
There are two operators for condition — if and switch. Both of them require writing a new section in code to work. Here is an example of the if operator:
if <condition>:
<actions-if-...>
<...-condition …>
<...-becomes-true>
Of course, there are an else and else-if (elif) branches for multiple conditions:
if <condition-1>:
<actions-if-condition-1-becomes-true>
elif <condition-2>:
<actions-if-condition-2-becomes-true>
...
elif <condition-N>:
<actions-if-condition-N-becomes-true>
else:
<actions-if-all-conditions-becomes-false>
There might be any boolean-resulting expression in place of <condition>, as in any general-purpose programming language. For example:
if .temperature < 0:
"Sub-Zero" ==> #wins
Boolean variables can be also used:
.property[boolean] ==> $variable
...
if $variable:
"property is true" ==> #answer
Note that there is no implicit types conversion to boolean. Unlike JavaScript and C/C++, 0 and "" (empty string) will not be treated as false values. There is no "falsy"/"truthy" concept in PVM.
The switch operator can also be used as a special form of the conditional operator. It’s usable when you have one value and many variants of action to do depending on that value:
.property[number] ==> $property
switch[$property]:
1:
<actions-if-$property-is-1>
2, 3:
<actions-if-$property-is-2-or-3>
7 ... 10:
<actions-if-$property-is-from-7-to-10>
default:
<actions-for-all-other-values-of-$property>
Note that every switch case is obvious and can be represented with a set of possible values, or even with a range. The default branch is optional.
Loop
Out practice has shown that one kind of loop operator is enough in most cases. In PVM it’s a repeat[N] placed in the section subject. It repeats the code in section N times. N can be any expression with number type:
repeat[$count]:
<loop-body-repeated-$count-times>
In addition to N, you can specify a counter-variable that will be incremented automatically for each iteration and its starting value:
repeat[$count, counter=$i, from=1]:
<loop-body-repeated-$count-times>
The repeat loop operator has a special form in the binary context (when you parse binary data). It will be covered in the next article.
Math operations
Along with basic arithmetic operators like + - * / you can use following mathematical functions:
math.round, math.floor, math.ceil, math.abs, math.acos, math.asin, math.atan, math.cos, math.exp, math.log, math.log2, math.log10, math.sin, math.sqrt, math.tan, math.trunc, math.pow[...]
Here's an example of how to apply these math functions in PVM code:
.battery.level ==> math.round ==> #battery.level
.value ==> math.pow[.power] ==> #value_in_power
When dealing with number values, PVM stores them as int64_t, uint64_t or double type. When you write a constant number value in source code, the way how it's written is important.
If it's written as decimal value without fractional part (as integer), then values from -9223372036854775808 to 9223372036854775807 will be stored internally as number value with int64_t type.
If you write number which is outside of that range, or it's written with fractional part (even with zero fractional part), then it will be stored internally as number value with double type.
For example:
10 ==> $var // stored as int64_t
10.0 ==> $var // stored as double
This is particularly important when performing division operation - operator / will implicitly do an integer division (discarding fractional part) in case both of its operands are integer values.
For example:
3 / 2 ==> #result // {"result": 1}But in most cases you use usual (not integer) division.
Thus if you don't need integer division and one of the division operands written as constant number value, it will be a good practice to always write that number with a fraction part (even if it's zero).
For example:
3 / 2.0 ==> #result // {"result": 1.5}
3.0 / 2 ==> #result // {"result": 1.5}
3.0 / 2.0 ==> #result // {"result": 1.5}String operations
PVM supports several string transformation functions. Some of them does not require any additional arguments to work:
strescape, lowercase, uppercase.
Just use them in the chain middle like this:
.payload.text ==> lowercase ==> #payload.text.in.lowercase
Please note that lowercase and uppercase work only for ASCII-letters.
Next transformation functions can be used to trim white-space or any other 1-byte symbol from the string:
trim, trimleft, trimright.
Here is an example of their usage:
.payload.text ==> trim ==> #payload.text.trimmed
.driver.id ==> trimleft["F"] ==> #driver.id.without.leading.f
.payload.text ==> trim["0"] ==> #without.zeros.from.both.sides
Substring replace can be performed with function replace[<what>=<which>].
For example, to replace char "+" with "-" use following pvm code:
.payload.text ==> replace["+"="-"] ==> #payload.text
To remove only first occurrence of the substring "foo" you can add option max=1:
.payload.text ==> replace["foo"="", max=1] ==> #payload.text
You can specify several replace pairs like in the code below:
.payload.text ==> replace["foo"="bar", "baz"="quux"] ==> #payload.text
In this case replace will be performed sequentially - all occurrences of "foo" will be replaced with "bar", then all occurrences of "baz" with "quux", and so on for all specified pairs. In this multiple-pair mode there is no way to specify max=... argument.