13 November, 2017

Streams: now pushing as well as pulling [UPD: Jul 10, 2019]

New independent streams in flespi can forward data from channels, devices, or both, and enable the flespi platform to work in push and pull modes.

Think of flespi as a warehouse where your devices deliver data. flespi stores it and sends it out to the addressee(s) in response to a REST API call. This is a PULL mode because data sits in the warehouse until requested (pulled).

An alternative is the PUSH mode. And this one caused us sleepless nights since it is another opportunity to help our customers which we haven't tapped. Until very recently. Advantages of PUSHing are plenty and obvious:

  • one-time setup: tune your app to receive data, enable flespi in PUSH mode, and do nothing else;
  • no REST API: no extra links between flespi and your application. The shorter the chain, the simpler the deployment. And flespi strives for maximum simplicity while ensuring the top performance!
  • no auxiliary modules to tune and host;
  • minimal storage fee: your data sends to you instantly, so you don't need space of flespi servers.

Now we are all set to present you the new streams interface. Here are some quick facts about flespi streams:

  • stream is an instance responsible for delivering messages from flespi to another platform;
  • stream is an independent instance — it does not belong to any channel or device;
  • stream can serve many channels and devices simultaneously;
  • stream is unidirectional — it delivers messages from channels and devices to other platforms (or flespi channels);
  • to start sending messages from flespi devices or channels to a third-party platform through a stream, subscribe it to these devices or channels and set the outbound protocol suitable for the target software.

Below is the illustration of what streams are up to:

how new flespi streams work

Next, we consider three main implementations of flespi streams: MQTT, Wialon Retranslator, and HTTP.

MQTT

MQTT is one of the most widely used protocols for IoT. Currently, flespi streams support three MQTT-based implementations: Amazon Web Services, Thingsboard, and general MQTT. Here are some tips for each of them:

AWS:

  • designed to subscribe for flespi devices. “mqtt_topic” parameter in the stream configuration is used to deliver new messages from flespi devices
  • you can find some advice on AWS IoT configuration here
  • a single certificate supports multiple AWS things
  • “thing_name” parameter in stream configuration applies to update thing’s shadow.

Thingsboard:

  • default URI for demo platform is demo.thingsboard.io:1883
  • no need to manually create devices on the thingsboard platform: just select “is gateway” checkbox in one new device configuration and set up a stream to this device with “access token.” Thingsboard will automatically create a new device if the message with unknown “ident” arrives.

General MQTT:

  • may work with your specific MQTT broker implementation. We have tested it with an open-source Mosquitto.
  • we have implemented our own MQTT broker: efficient, distributed and scalable over multiple servers, and featuring a wide range of settings. Give it a try and let us know what you think.

Wialon Retranslator

HTTP

  • performs POST request with a JSON list of messages as an HTTP “data” parameter
  • optional custom HTTP headers are possible
  • URI must include the protocol: HTTP or HTTPS

Upon completion of the stream config, you should subscribe it to receive data from devices or channels. Do so by selecting the desired stream in the Streams tab on a channel or device card:

assign device to flespi stream

Or use the Devices and Channels tab on the stream card to assign the required instances:

subscribe stream to device or channel

Note: the current stream implementation is just sort of a wrapper for our internal MQTT message bus that processes all messages from all channels and devices. Now you also have direct access to this technology. It gives impressive possibilities to developers, so don't forget to check it out!

We also have examples of the HTTP stream receiver in different programming languages that you can deploy on your server to effectively consume the data from the HTTP stream.

Get flespi news on Telegram!