5 March, 2019

flespi technologies — the power of the hidden gems

Where flespi performance and stability originate from.

Leveraging inimitability

Flespi team is often referred to as the R&D lab. 

And what R&D labs do is come up with ideas, check hypotheses, and develop technologies. Not the user-facing products, but developer-facing tools.

From the economic perspective, for a company to be more competition-resistant on the market, it has to possess inimitable assets. 

Competition can beat you on price, can do better marketing, and can deliver better customer service. So, it's important to find the inimitable core of your business that competitors will struggle to mimic.

In flespi we crystallized three things hard to imitate (at least in the short term):

  • Team — it’s not only the expertise mix acquired by the flespi crew but also the mutually complementing approaches, views, and personalities.

  • Experience — 50+ years in telematics combined.

  • Technologies — we create new concepts and methodologies in telematics from square one (see below ↓).

What’s more, these three form an inseparable whole: the team uses its experience to create cutting-edge technologies.

Bringing value to the industry

When you are creating a product, you have a list of problems it’s intended to solve. With technologies, it’s slightly different — we simplify processes, give tools, and reinvent approaches, but it’s the developer of a particular system who decides in what ways the offered options will be used.

That is why we sometimes fail to clearly outline where our services can apply — because oftentimes the range of applications is so wide that we might not realize all of them ourselves.

Let’s get to the specifics and look at the cornerstone technologies shaping the flespi platform.

Protocols parsing technology (PVM)

Flespi welcomes telemetry messages from 250+ GPS tracker models. Handling all this diversity transparently for the user is the PVM’s calling.

In essence, PVM is a programming language that makes it easy to describe devices communication protocols with a set of standard instructions.

What PVM does is unscrambles each message according to the protocol specification, applies universal parameter naming, and saves the message in a JSON format.

pvm scheme


The common naming scheme makes the two-sided communication with the tracking devices so much easier by relieving you from any concerns about protocols specificity. 

A very detailed (yet slightly obsolete) article about PVM is available on our blog.

Telematics storage (MDB)

Dozens of database engines from the well-known brands are available on the market? Then why reinvent the wheel? Because we need a special wheel that will run fast when pulling a cart full of telematics data.

Telemetry messages have their peculiarities in terms of volume and frequency. Usually, a GPS tracker sends messages with a timestamp and 100-900 bytes of useful payload. The frequency varies from a few seconds to a few minutes, depending on the task and required precision. The messages need to be sorted by device, then sorted again by time for this device, and finally stored in the database and read upon request.

None of the available non-SQL and SQL databases available on the market could deliver the desired level of performance and stability when dealing with hundreds of terabytes of telematics data.

MDB architecture accounts for the specifics of the telemetry messages to guarantee fetching 500,000 messages per second in a block mode and 100,000 messages per second in a random access mode.

A special share mechanism distributes the storage capacity and load over the MDB nodes, thus ensuring superior redundancy similar to RAID-10 solutions.

Once the telematics data from your tracking devices get into the flespi platform, it will be stored safe and sound and will be spit back to you lightning-fast once needed.

MQTT-based hierarchical database (HASD)

This concept is one of the freshest inventions and one of the most outside-of-the-box ideas.

Inspired by the hierarchical nature of the databases of the 60s, HASD represents the new generation of the hierarchical state database wired using the cutting-edge MQTT 5.0 protocol.

By design, MQTT is a communication protocol — a lightweight alternative to HTTP gaining its popularity due to the development of the Internet of Things (IoT) and machine-to-machine (M2M) interaction. 

HASD, however, uses another important feature of MQTT — retained messages — to store data inside MQTT Broker.

Each service or app that relies on HASD as storage maintains an MQTT session for communication with other services and at the same time for data actualization purposes. 

HASD is

  • Hierarchical — since it relies on the MQTT topic structure

  • Asynchronous — because it’s in the MQTT protocol’s nature

  • State — the data is updated automatically via the established subscription

  • Database — no-one’s ever positioned MQTT broker as a database, however, it’s capabilities perfectly fit the database definition in Wikipedia.

HASD fits perfectly to some of the most demanded use cases on the market:

  • Secure access to the cloud state database from remote services applicable to both frontend and backend.
    Here HASD solves the most pressing concern with relational databases — the issue of ensuring secure remote access to it. Usually, security is delegated to a proxy, but creating one is not often a trivial task. With HASD, the security is provided by the SSL-ed MQTT connection based on access tokens and flexible read/write ACLs.

  • Asynchronous performance beyond the capabilities of a relational database — generate hundreds of thousands of update requests per second and MQTT broker together with its storage system will easily handle this load sorting out the latest actual data state, thus giving you the opportunity to go beyond the transactions-per-second rate of relational databases.

  • Session access from a mobile network — when the client IP address is changing all the time. Setting an expiration period for MQTT sessions allows resuming the session easily even after reconnect with zero data loss. 

HASD technology accelerates and simplifies the development process thanks to the following principles:

  • Standardized access to data regardless of whether backend or frontend

  • The same methods for initializing state data and keeping it updated

  • One MQTT session required for the application to communicate with the database and/or with other services

  • A plentitude of MQTT client libraries make the choice of the language a matter of your personal taste

As a result, you get high-performance, high-availability, and high-reliability storage for frontend and backend services while saving development time and efforts.

***

The three gems above are the major inventions that we utilize in our infrastructure and offer to our clients. We also have a number of internal developments that are not visible on the surface but that ensure flespi platform reliability and performance, e.g. automated administration system, platform health checker, the core of the flespi MQTT broker, etc. 

We in flespi enjoy finding room for improvement but not by doing the same things better, but by doing things differently.

Our mission is to serve as a technology incubator for the industry with a strong focus on quality and added value to the end user.

UPD: our website now features a dedicated page about our major technologies.