I am attempting to create an external database (Postgres) that would hold tables with time-series sensor data from devices defined in Exosite.
Upon brief review of Developer Docs, I understand that there are APIs to perform RPCs to read data (PULL). Alternatively, option exists to create Web Sockets for subscription to any new sensor values (PUSH) as they arrive in Exosite. To use Exosite as the data pipeline for time-series data, what method would suit best to transport the sensor readings from Exosite to an external database? What transport mechanism is preferred – pushing from Exosite to external database, OR polling Exosite from external database?
If you can share pertinent examples, and help identify the steps to send data to an external database like Postgres or similar, that would be very helpful.
This is a great question, so thanks for posting @dkamboj!
Remember that pulling data out of Exosite Portals is client initiated, so you will need an application that talks to Exosite over our APIs, and then writes new information into your database. Exosite doesn’t have a preference as to which API you use, but you may prefer web sockets or long-polling in order to have the quickest data resolution.
Documentation on the possible API endpoints to pull data:
Pulling is the most common approach, as data will be made available on client demand.
Closer to true push, you could send data directly to your server via HTTP, provided your application would be configured to accept communication from our platform. Using Portals scripts or dispatches you can send a message to your server including the newest data written to the platform.
They subscribe to a single resource in the system, and can be configured to send a message to a phone/inbox/endpoint based on new writes to another resource. This is the fastest and most light-weight way to actively send information out of Exosite.
Code written in Lua that can execute based on new writes to a resource. You can detect, read, and parse data, then later send it to your server. The documentation on our scripting service can be read here: http://docs.exosite.com/scripting, and an basic example using HTTP can be read below:
We are also using Exosite as an IoT data ingestion service, where we know it will be acquired and stored for us, then we flow the data to an enterprise database (at present MS SQL Server). The response here is a great outline of how to approach the process.