Plan out and possibly prototype what a data pipeline would look like to generate the heatmaps in realtime

[keenanjohnson]

We should figure out how to generate emissions heatmaps in real time on our data dashboard.

[abhinavtripathy]

@keenanjohnson real time is actually an interesting challenge, as we look to scale this I wonder if we should consider doing refresh every few seconds first and then pure real time. I wonder what the cost/benefit for the approaches are. I am not sure what you were thinking for this.

[nivashini10]

I am exploring d3 gallery for heatmaps examples - https://observablehq.com/@d3/gallery. D3.js is a JavaScript library for visualizing data using web standards. The particular example that would be great fit for ribbit data would be Hexbin Map and Bubble Map.

I am currently working to create a mock heat map in D3.js and introduce real time by writing a script for refreshing the data. It would be a great way to start. In JavaScript, there is a timer class that will fetch an update using ajax. I am not familiar with JavaScript and learning as I go, so would love to take any suggestions if someone else has worked on this before!

I also looked into tableau and Matplotlib package in python for heatmap generation and it can be used for real time updates when connected to the server or cloud. Tableau may require license key later on.

[fosteman]

@nivashini10 , did you get much further into d3? I have some practical experience, we can hack something together. Lmk

[fosteman]

Thinking that it'll be great to go ahead with Firebase - offers bunch of services (including serverless functions, firestore DB, and CI pipelines) for free under a certain measure of use - would like to tap into our analytics to figure out current usage stats.