How to split HEX string parameter into bits with a PVM code plugin?

Some trackers send custom bytes of information that can’t be parsed for the whole protocol and are stored as a hexadecimal string parameter.

For example, Teltonika Manual CAN elements are stored as indexed parameters can.data.frame.X. If you need to extract specific bytes from such parameters, the PVM code plugin may help you with this. Here’s how to do it.

Here is a message containing the can.data.frame.11 parameter — a hexadecimal string that carries 8 bytes of information:

{
  "battery.current": 0,
  "battery.level": 100,
  "battery.voltage": 4.129,
  "can.data.frame.11": "47544B0100014AE1",
  "din": 0,
  "din.1": false,
  "din.2": false,
  "engine.ignition.status": false,
  "event.priority.enum": 0,
  "external.powersource.voltage": 48.601,
  "ident": "555444333111222",
  "movement.status": false,
  "position.altitude": 567,
  "position.direction": 344,
  "position.hdop": 0.6,
  "position.latitude": 17.833562,
  "position.longitude": 87.149355,
  "position.pdop": 1.2,
  "position.satellites": 15,
  "position.speed": 0,
  "position.valid": true,
  "server.timestamp": 1649922838.082161,
  "timestamp": 1649922834
}

PVM plugin configuration

To arrange custom parsing of the can.data.frame.11 parameter, we need a “msg-pvm-code” plugin with the following script:

Here’s the PVM code in plain text in case you decide to copy it:

optional .can.data.frame.11 ==> %hex ==> bits:
	8 ==> skip    
	16 ==> #bms.capacity.remaining                            
unset .can.data.frame.11 // if you need to remove source CAN data from the message

What does the plugin code do?

If the can.data.frame.11 parameter is present in the original device message (operator optional), the plugin takes the value of this parameter (string "47544B0100014AE1")  and converts it into an 8-byte integer number (0x47544B0100014AE1), treating the value as a string that describes a hexadecimal number (operator %hex).

After that the plugin extracts certain bits of the resulting integer (operator bits) and does the following:

• first 8 bits (counting from right to left, i.e. bit 0 - bit 7, 0xE1 in the example above) — are skipped

• the next 16 bits (bit 8 - bit 23, 0x014A in the example above)  — are stored in the resulting message as a bms.capacity.remaining parameter with a decimal value 330

• and the rest of the bits are skipped

Finally, the plugin removes the processed parameter can.data.frame.11 from the device message (operator unset).

Assign the plugin to the device(s) (Plugin tab on the Device card) and the code will be applied to every message of the assigned devices. The resulting message will look like this:

{
  "battery.current": 0,
  "battery.level": 100,
  "battery.voltage": 4.129,
  "bms.capacity.remaining": 330,
  "din": 0,
  "din.1": false,
  "din.2": false,
  "engine.ignition.status": false,
  "event.priority.enum": 0,
  "external.powersource.voltage": 48.601,
  "ident": "555444333111222",
  "movement.status": false,
  "position.altitude": 567,
  "position.direction": 344,
  "position.hdop": 0.6,
  "position.latitude": 17.833562,
  "position.longitude": 87.149355,
  "position.pdop": 1.2,
  "position.satellites": 15,
  "position.speed": 0,
  "position.valid": true,
  "server.timestamp": 1649927515.468316,
  "timestamp": 1649922834
}

Here bms.capacity.remaining=330 parameter is a decimal number that carries converted bits 8 - 23 of the initial parameter can.data.frame.11.

Skipping empty values

One more thing. If a device can’t read CAN data frame bytes from the CAN reader, it may send an empty (zero) value for can.data.frame.11 parameter:

{
  ...
  "can.data.frame.11": "0000000000000000",
  ...
}

In this case, in order to skip the empty values you may use the if operator as a filter. The modified code is below:

optional .can.data.frame.11 ==> if this != "0000000000000000" ==> %hex ==> bits:
	8 ==> skip    
	16 ==> #bms.capacity.remaining                            
unset .can.data.frame.11

In this case, the resulting message will not contain the "bms.capacity.remaining": 0 parameter with zero value and the can.data.frame.11 parameter with zeros will still be removed.

Advanced binary parsing

If your CAN data contains a more complex binary structure, like multi-byte integers in Big Endian, you need to use a base ability of PVM to parse arbitrary binary data. Here is how to use it:

optional .can.data.frame.11 ==> %hexstr ==> input:
	%uint16[BIG] ==> #bms.capacity.remaining
	[size=3] ==> skip
	%int16 ==> this / 1000.0 ==> #bms.capacity.voltage

In this example, the source 16-char string  "47544B0100014AE1" will be decoded with the %hexstr operator as a Hexadecimal string into an 8-byte binary data buffer containing bytes 0x47, 0x54, 0x4B, 0x01, 0x00, 0x01, 0x4A, 0xE1. Then the buffer will be parsed with an input: section as a binary stream in such a manner:

1. The %uint16[BIG] type will consume the first two bytes (16 bits) 0x47, 0x54 as a two-byte Big Endian (note the [BIG] attribute) unsigned integer into value 0x4754 (which is  18260 in decimal), which will be stored in the message parameter "bms.capacity.remaining": 18260.
2. The next 3 bytes - 0x4B, 0x01, 0x00 - will be skipped with line [size=3] ==> skip.
3. Then the %int16 type will consume the next two bytes (16 bits) 0x01, 0x4A as a two-byte Little Endian signed integer into value 0x4A01 (which is 18945 in decimal). Then such value will pass through the simple math division by 1000.0 and become 18.945 (note that without floating point zero suffix .0 the integer division will take place). And finally, the value will be stored in the message parameter "bms.capacity.voltage": 18.945.
4. The last byte 0xE1 of the binary buffer will be left unparsed.