Custom parsing in plugins or a sneak peek to PVM

I believe you are already familiar with plugins — the super-powerful supplements to devices that let you modify parameter values, add custom parameters, delete unwanted parameters, and so much more. If you haven’t had a look at them yet, you may want to start by checking specific examples of plugins use.

Psst, this is not what you are here for.

We have something special for those of you who are not afraid of getting their hands dirty writing code and are ready to master a new language. We let you voluntarily try yourself as a flespi engineer. But only if you want to.

Do you say I can set up custom parsing for device data?

Yes. Now you can use plugins to write custom instructions to the flespi parsing engine to interpret the payload in your device messages.

Let’s look at a real example straight away. Below is the plugin that removes the location data from device messages to comply with the GDPR regulations:

The code on the screenshot will be applied to all messages from devices assigned to that plugin and will perform the following logic:

• If the message contains the “private.mode” parameter and its value is true, then...
• … then unset (remove) several parameters related to the device position in the message.

To add your own custom code to plugins you need to understand how to write programs in PVM. Let us give you some basics here.

PVM introduction

PVM — is a domain-specific language (DSL) created in flespi to describe data shape and its transformation.

The main purpose of PVM is to parse raw binary or text data into a JSON message. But dealing with bytes of raw data is not the only thing PVM can do. It is designed to potentially transform any structured data entities like HTTP requests, MQTT messages, or JSON objects.

A program in PVM describes how data should be transformed, piece by piece. Most of the PVM program consists of the lines like this:

<get value> ==> <maybe transform it> ==> ... ==> <store value>

And assuming that data source and target is a JSON message:

<get property> ==> <maybe transform it> ==> ... ==> <store property>

The transformation part in the middle is optional (if you just want to copy value).

Super simple example

Let's imagine we need to get the analog input 1 voltage (typically, the parameter “ain.1”) in millivolts in a separate message parameter “ain.1.millivolts”. Here is a PVM code we should put into the plugin:

.ain.1 * 1000 ==> #ain.1.millivolts

Yes, it’s simple. You just write:

1. Which JSON property value you need as a source: ”.ain.1” means “take property ain.1”
2. Then convert it: ”* 1000”, i.e. multiply it by 1000 to get millivolts, and finally…
3. Store the converted value: ”#ain.1.millivolts”, means “save to ain.1.millivolts parameter in the message”

If you assign a plugin with the code to the device, then the following message:

{ "ain.1": 3.14 }

will be converted into this one:

{ "ain.1": 3.14, "ain.1.millivolts": 3140 }

More advanced example

The example above will work properly only if each device message contains the “ain.1” parameter. If it's absent in some messages, the message registration stage will fail because PVM code execution in the plugin will finish with an error: no property 'ain.1' in JSON object. To avoid that, your PVM code should include the conditional logic that will perform the conversion only if the message contains the “ain.1” parameter. Here are two ways to accomplish this:

1. Use condition operator (if):

if is_set .ain.1:
	.ain.1 * 1000 ==> #ain.1.millivolts

2. Use property reading with optional operator:

optional .ain.1[number] ==> this * 1000 ==> #ain.1.millivolts

In case 1 we use a condition operator ("if") with property set check (is_set .ain.1). Note the increased indentation on the second line: it will be executed only if the “if” operator gets true value for the “is_set“ check.

And in case 2 we use the “optional” operator to instruct that the whole line should be skipped if the “ain.1” parameter is absent in the message. Note that in this case, we have to explicitly specify the property type (number) and move our millivolts transformation to the next step (after arrow ==>).

Note the difference between the two approaches: the first one tells us what should be done, and the second one defines how the data looks at both ends (in and out). The second approach shows that PVM can be used as a descriptive programming language. This means that in PVM you can describe how the data looks and how it should be transformed. This helps the code to be clean and expressive. You want to stick to the second approach if you use PVM.

Another cool example

There are common parameters representing a GPS fix quality — position.fix.type and position.fix.type.enum:

The position.fix.type is a numeric parameter with values 0, 1 or 2. And if your messages do not contain the human-readable parameter position.fix.type.enum you can add it using PVM code in a plugin like this:

optional .position.fix.type[number] ==> map[0="no fix", 1="2D", 2="3D"] ==> #position.fix.type.enum

Here we’re using a handy PVM operator map[] that helps easily convert values. Please note that by default it will throw an error if it receives an unspecified value (none from the 0, 1, 2). If it’s your case, you need to add an “, error=false“ in square brackets after all possible values.

Here is another form of this operator, useful if your conversion table is big:

optional .position.fix.type[number] ==> map ==> #position.fix.type.enum:
	0 ==> "no fix"
	1 ==> "2D"
	2 ==> "3D"

Complex conditions and mappings example

PVM plugins can also be helpful in implementing logic based on complex conditions depending on several values. For example, you want to convert some event.enum and event.function value pairs into human-readable messages. Here's how the code may look like. 

if is_set .event.enum && is_set .event.function:
	.event.enum[number] ==> $enum
	.event.function[number] ==> switch:
		0:
			"general" ==> #log.type
			switch[$enum]:
				0, 4, 166, 240:
					"alert" ==> #log.level
				default:
					"info" ==> #log.level

			$enum ==> map ==> #log.msg:
				0 ==> "Software reset"
				1 ==> "Power supply was plugged in"
				4 ==> "Power supply problem"
		104:
			"tacho" ==> #log.type
			switch[$enum]:
				38:
					"alert" ==> #log.level
				default:
					"info" ==> #log.level

			$enum ==> map ==> #log.msg:
				29 ==> "CAN communication started"
				36 ==> "Remote authentication successful"
				37 ==> "Remote download access granted"

Next steps

If you understand the above examples, you are ready to start writing simple transformations for your needs.

To make you more equipped, we’ve released a more comprehensive PVM language documentation.

***

Plugins in flespi represent a whole new layer — the transformation layer — that enables you to modify message content exactly as you want. The possibility to add PVM code to a plugin to instruct flespi to perform custom parsing of specific parts of a message gives great flexibility and freedom — should your device carry extra data from versatile peripherals (sensors, BLE tags, etc.), you will be able to extract the specific values by writing your own processing logic.

Stay tuned and wait for more.