This article is left for history, however, the most up-to-date guide to PVM is located here.
flespi gateway is designed to interact with various telematics devices and convert all the data diversity coming through different protocols into a unified message format.
You already know from our blog that protocols are defined by our own “pvm” language, so you should be curious about the details. Let me unveil the power of pvm language for you.
PVM is a domain-specific language for Parsing Virtual Machine. PVM instructions are compiled into a special assembler code (defined by us) by a proprietary compiler and then executed on the embedded virtual machine. We make pvm language so simple that it could describe complex binary protocols in 50-100 lines of code.
Wialon IPS
Let's start with Wialon IPS communication protocol – the protocol with an open specification developed by Gurtam and used by lots of communication devices.
Since pvm is a descriptive language, all we need to do is to describe the data input stream. The input stream is a sequence of bytes received from a device, and it can be treated both as text and as binary data. And we’ll describe this sequence moving upside down. According to the protocol specification, Wialon IPS requires a login packet to be sent as soon as a connection is established, and data packets to be sent afterward. To tell the machine how to deal with these packets we will describe each packet under the corresponding @label.
A login packet is a text of the following format:
#L#imei;password\r\n
So, the login packet description looks like this:
@login: // this is a login packet
[until="\r\n"]: // it consists of all characters until "\r\n", and these characters are:
{"#L#"} // first three characters are a predefined login packet header
[until=";"] ==> %text ==> #ident // the next characters until ';' are an identifier
[null="NA"] ==> %text ==> #password // the rest of the characters are a password
// a special value "NA" means that the password is not set
So we've taught the machine how to parse the login packet. It appears to be quite simple, isn't it? Now we should tell the machine what to do after the login packet is successfully parsed (actions block) or if an error occurred (error block):
actions: // a special word with and an evident meaning
^login // save the ident and password in all messages of this connection
send {"#AL#1\r\n"} // send a login packet response
error:
send {"#AL#0\r\n"} // send an erroneous response
That’s all for the login packet! And after the login packet has been successfully parsed we`ll have to handle the data packet. According to the protocol, there is a number of data packet types. For example, we can get a text message in the following format:
#M#msg\r\n
or a simple data packet with the format:
#SD#date;time;lat1;lat2;lon1;lon2;speed;course;height;sats\r\n
or just an empty ping packet:
#P#\r\n
It's not a problem to cover them all. Let's see!
@data: // this is a data packet
[until="\r\n"]: // and again: everything until "\r\n" is the data packet content
{"#"} // the first character of the packet is an octothorpe
[until="#"] ==> %text ==> $packet_type // the packet type is specified between two octothorpes
if $packet_type == "P": // the rest of the data is parsed according to the type of a packet
pass // nothing to do here: this is just a ping
elif $packet_type == "M":
%text ==> #message // save message as text
elif $packet_type == "SD": // a simple data packet is not a problem:
split[";", items=%text, null="NA"] ==> items: // split the content by ';' and treat split items as text
@short_data // if the text is "NA", then a parameter is absent
// what to do with each item will be described separately
// elif $packet_type == ... // and so on ...
// parsing of other types of packets can be described here
else: // unexpected type of packet
^error // rise an error
The main data packet structure is covered. But what is missing? Yeah, now we need to send an answer and teach the machine what to do in case of error. And even for such protocol as Wialon IPS using a complicated scheme of server answers, it is possible to clearly define how to compose an answer.
Go ahead!
actions:
if $packet_type == "P":
send {"#AP#\r\n"} // send special response for a ping packet
// certainly, the developers of wialon ips protocol made their best to ravel us :)
else:
^store // it's high time we stored all parsed parameters in a unified channel message!
send: // and send a response to inform the device
{"#A"} // that a packet of a certain type was successfully parsed
$packet_type ==> %text
{"#1\r\n"}
error:
if is_set $packet_type: // we've parsed the packet type, but failed to parse the packet content accordingly
send: // so tell the device that the packet was incorrect:
{"#A"} // send the erroneous response for the corresponding packet type
$packet_type ==> %text
{"#0\r\n"}
Almost done! There’s one more thing left to do: as you remember we've left @short_data label undefined. So, here it is:
// a simple data packet example:
// #SD#DDMMYY;HHMMSS;lat_ggmm;lat_sign_NS;lon_gggmm;lon_sign_EW;speed;course;altitude;sats\r\n
@short_data: // Yeh, it is so simple:
#date["DDMMYY"] // you
#time["HHMMSS"] // just
%double ==> #lat_ggmm // describe
#lat_sign_NS // the parameters
%double ==> #lon_gggmm // one
#lon_sign_EW // by
%uint16 ==> #speed // one
%int16 ==> #course // as
%double ==> #altitude // they
%uint8 ==> #sats // go
There are some other data packet types supported by Wialon IPS, but, not to be boring, I won't describe all of them. Once armed with this knowledge, you will be able to easily implement all other packet types. An attentive reader has certainly noticed some common rules of the pvm language:
colon:
and indentation (used to describe block contents)
@labels (used for abstraction and keeping things separately)
magic ==> sign (which shows data transformation flow)
%types (that specify the data type, i.e. text, double, uint32, etc.)
$variables (to temporarily save values)
#parameters (params of parsed packet to store in a channel message)
^functions (some common predefined actions)
The meaning of these and other syntax constructions can be clearly seen in the examples above.
Complete PVM definition of Wialon IPS protocol
And I bet now you want to know how to apply the pvm language to binary protocol?
Wialon Retranslator
Stay tuned! It's time to tell you about Wialon Retranslator protocol implementation in pvm language.
Wialon Retranslator is a binary protocol and it can be used to transfer data from Wialon (you’ve probably heard about Wialon ;) ). Despite the fact that it is a binary protocol, pvm copes with it easily. According to the protocol specification, data is transferred as TCP packets and we will describe its structure under the @packet label. What else we need to do is to describe actions in case of either successful packet acquisition or error. Let's do it right now!
In order to define the packet structure, we need to look at the protocol documentation. It says that each packet starts with a common heading comprising size, device id, timestamp, and flags followed by payload with an arbitrary number of data blocks. So, the packet definition looks like the following:
@packet: // a packet consists of:
%uint32 ==> $size // the first 4-byte unsigned value is the packet size saved into a variable of the same name
[size=$size]: // the next byte sequence of this size is parsed as follows:
[until=0x00] ==> %text ==> #ident // device identifier, null-terminated string
%uint32[BIG] ==> #timestamp // timestamp
[size=4] ==> $flags // 4 bytes of flags
repeat: // repeated blocks of data
@data_block
Here we defined that after the first 4 bytes containing the packet size goes the following sequence of data:
- null-terminated string - device identifier
- timestamp value
- 4 bytes of message flags
- and repeated blocks of data, which we will define later.
Also, this example shows how to switch from little-endian byte order (that is used by default) to big-endian for parsing integer values when it is needed by a protocol. Let's proceed. Now tell the machine what to do after the packet is parsed:
actions:
if $flags & 0x10:
^param["SOS", 1] // save "SOS" parameter if alert bit is set
^login // perform login (i.e. store ident into channel message)
^store // store parsed parameters
send {11} // send 0x11 - packet acquisition byte
Should we tell the machine what to do in case of a parsing error? Not in that case - Wialon Retranslator protocol doesn't require any special answer in case of error, so default actions will be performed: the error will be printed into logs to warn the developers about it and the connection will be closed.
Now let’s define @data_block label. Again look into the protocol specification and see that each data block has a common structure.
@data_block:
[size=2] ==> skip // block type, it is always \x0B\xBB so we skip it
%uint32[BIG] ==> $block_size // get the size of a data block
[size=$block_size]: // parse all bytes of the block, they are:
[size=1] ==> skip // security attribute - skip it
%uint8 ==> $data_type // data type
[until=0x00] ==> %text ==> $param_name // parameter name
// then parse the data according to data type
if $data_type == 0x1: // text data
[until=0x00] ==> %text ==> #param[$param_name]
elif $data_type == 0x2: // binary block with position information
%double ==> #lon
%double ==> #lat
%double ==> #altitude
%uint16[BIG] ==> #speed
%int16[BIG] ==> #course
%uint8 ==> #sats
elif $data_type == 0x3: // integer, 4 bytes
%int32[BIG] ==> #param[$param_name]
elif $data_type == 0x4: // double
%double ==> #param[$param_name]
elif $data_type == 0x5: // integer, 8 bytes
%int64[BIG] ==> #param[$param_name]
else: // unknown data type - rise an error
^format["unsupported data type 0x%02X", $data_type] ==> ^error
That’s all!
Besides the syntax constructions that you already know from the previous example here you can see yet another interesting case:
#param[$param_name]
It means "save parameter with the name param_name" and is useful when you don't know the name of a parameter in advance.
Complete PVM definition of Wialon Retranslator protocol
This is what makes our product unique: purely in-house development, focus on simplicity, and years of experience in protocol integrations.
So let’s see what you get with flespi. The platform providing unified data communication, the technology allowing the description of even the most sophisticated protocols, high-quality support throughout the process of project implementation, and articles on the flespi blog making complicated processes seem much simpler ;).
We continuously develop the pvm language guided by the ambitious goals we set and would like to share our achievements with you on our blog, so follow the news and stay tuned!