How to stream IoT & telematics data to Google IoT Cloud?

Ingest telematics events from flespi to the Google IoT Cloud using Google Pub/Sub.

Google Cloud announced that it’s shutting down its IoT Core service on August 16, 2023. The alternative for exchanging the data with the Google ecosystem is the Google Pub/Sub service destined to establish asynchronous communication between applications and solutions. With flespi you can publish telemetry data to Google Pub/Sub and then subscribe to the appropriate topics to get that data in the desired destinations.

  1. Create a project in Google Console. You will need to paste the project ID into the stream configuration later.
    google cloud console project id

  2. Create appropriate topics.

  3. Create a flespi stream to publish messages to these topics. Specify your Google project ID (from step 1), service email (learn how to create a service account), and the private key for IAM authorization in the stream configuration:
    create flespi stream to google pub sub
    Note: you can also specify a list of parameters (or all parameters) to be added as message attributes into the message.
    Find more details on access control, authorization, and required permissions here.

  4. Subscribe the stream to devices to forward data from:
    subscribe stream to devices

  5. Make sure the stream started sending messages:
    check logs flespi stream

  6. Subscribe to the topics from your target applications to get the updates instantly.

Troubleshooting

Stream will skip messages that do not confirm to the Google Pub/Sub service message limits. You can detect such messages under the non-zero skipped column in stream Logs. Currently we skip messages which:

  • size is above 10MB;
  • contain 101 or more attributes per message;
  • size of any attribute key exceeds 256 bytes;
  • size of any attribute value exceeds 256 bytes;

To confirm to this limitations you may pass to Google only certain messages using validate_message option with some expression or leave in message certain parameters only using convert_message option.

Change log

To stay in sync with all improvements and changes to the Google Pub/Sub stream, subscribe to the change log on the forum.


See also
Sending telemetry data from flespi to SENT-GEO
Sending telemetry data from flespi to Pulsar Broker