Predicting power generation for the day

[jcomand]

Has anyone worked on a way to predict how much power will be generated during the next day, based on past performance and the weather and time of year? And similarly to predict how much power will be needed for the home (more random, I'm sure, especially with car charges for those who have EVs)?

One use case: While I am waiting for my inspections and permission to export to grid, I am use "self-powered mode". On a sunny day, the powerwall charge tops out at 100% and solar generation is lost during the second half of the day. Should I charge my car to use up some of yesterday's stored power, or wait for another day since it will be cloudy and the home will use all of the solar? Probably could find similar cases for off grid or TOU users.

[jasonacox]

Hi @jcomand - Great question. There was a similar discussion here: #143

Just for giggles, I wonder if one approach that we could take is to leverage the current AI generative transformer excitement. If we structure all the historical data into a sentence, something like this:

On June 10, at 08:00, it was 25% cloudy and there was 650W of solar and 829W of power use.
On June 10, at 09:00, it was 15% cloudy and there was 1650W of solar and 913W of power use.
On June 10, at 10:00, it was 0% cloudy and there was 31650W of solar and 1024W of power use.

We could then train the model on that data and push a prompt to it like, "On January 3, at 09:00..." and see if it is able to generate, "it was 23% cloudy and there was 500W of solar and 812W of power use." I think the issue is that many of us only have 2 years of data so the month/date may not be enough signal. Perhaps we just drop the date and let the the model learn averages for the month and time.

[jcomand]

Sorry for the duplicate topic, but glad to revive interest. Fun idea to use AI. You would not necessarily need a language model and could use a model designed to directly suck in columns of numbers. https://orangedatamining.com/ has a GUI that can make this approachable to see what works; underlying models are actually in python. Then there is also the more traditional approach of using a regular statistical model (run a curve fit on insolation/time versus solar production). Either way you would create an input training set with time of day, date of year (or better, some sun angle/insolation variable proportional to expected power for a given lat/long), clouds, rain, snow, and solar production, home use. When permission to export is off (is this recorded?), you would want to exclude data points where the powerwall charge is 100%. You might have to explicitly add up each's days data. You could implement daily predictions or with finer time scale.

The graphable variables would be predicted solar production, predicted home use, predicted battery SOC based on those. If predicted SOC >100% during today and you don't export to grid or want to save the 10% round trip losses, it's time to charge you car, set your AC to run in the afternoon before you get home (could be automatic), or do all the laundry. (Another idea only partially related would be to control car charging directly through the tesla API to match the solar production minus home use from minute to minute.)

I do not have time to do it but maybe someone else would. Happy to give input if someone makes a data set. (No pun intended.)

[jasonacox]

Brilliant idea! I also love the idea of taking actions based on predictions. And thanks for the link to Orange. I'll check that out.

Probably the best first action would be to create a tool to export data from InfluxDB into a relevant format and place for everyone to upload/share. I like the idea of not using month/day but using solar insolation and time. The tool could have the user input something like solar insolation by month (kWh/m²/day) using online tools like Fabhabs and just inputting your Lat/Lon. - better yet is to find a table/equation/API to use.

What are the key data elements for the training set that would abstract it away from location or system size?

{system_size} = Size of solar panel system production (e.g. 8.5kW)

• Current solar insolation (kWh/m²/day) - instead of day of year and location
• Current solar production (kW) / {system_size}
• Current home usage (kW) / {system_size}
• Current weather cloudiness (%)
• Current grid usage - optional? (kW) / {system_size}
• Current battery SOC (State of Charge) - optional (%)

[jcomand]

Yes you got it! The "system size" could be used to give a prediction without looking at any user data (e.g. from the online tools/formulas), and this would be a good first step that gets most of the utility out of this concept. One level deeper would be to use the user data estimate the efficiency of the local system compared to what is expected based on system size and location. (panel orientation, shade, etc would matter and you could see how you are doing comapred to ideal.) Two levels deeper would be to train a model / regression to predict production just for your house, like if you have shade on one side and your curves are different than others. The last option is what we I thought of first with the AI, but maybe not where to start since most complicated.

[jcomand]

Or whatever seems fun for whoever is coding it :) Just my first thoughts

[BJReplay]

There's a very easy way to accurately predict what the insolation will be, and that's using Solcast that has a free account for personal use.

See https://solcast.com/free-rooftop-solar-forecasting to sign up for free. The free account allows a limited number of API calls per day, so you would need a way of calling it a few times per day, storing the results, and then including the results in a dashboard.

I'm reusing something I've done previously where I'm storing the forecasts in a database, so I just query that database from grafana (energy estimate is half hourly power, which is why it is multiplied by two to turn it into hourly energy).

SELECT $__timeEpoch(EstimateDateTimeUTC), EnergyEstimate * 2 as value, 'Forecast Generation' as metric FROM [dbo].[forecasts] WHERE $__timeFilter(EstimateDateTimeUTC) ORDER BY EstimateDateTimeUTC ASC

The simplest - and, to be honest, best - way of predicting load, without going over the top, is same time last year. For my purposes (controlling charging / discharging my powerwalls) I use the four weeks around the same day last year, and the most recent two weeks from this year and average those.

If you want to get more sophisticated you can add in heating degree days and cooling degree days to adjust for expected heating and cooling load - especially if you have a lot of heating or cooling dependent load, but for most, that's probably overkill.

[jgleigh]

I'm not even sure AI can predict future solar production. What's interesting is theoretically there's a specific cloud pattern that could increase your solar production beyond what is possible with a clear blue sky. You'll see this as power spikes when clouds are moving across the sky due to fringing effects.

[youzer-name]

I'm also using SolCast, but not to the level of details that @BJReplay is using it. I'm just curious to see how the predictions match up to reality, and I expect it would be useful information during an extended power outage.

I set up two systems in SolCast (one for each side of my roof) and pull the data once a day, before the sun comes up. I'm using Node-Red to make the API request, parse the response, and store it in InfluxDB.

On my Current Status dashboard, I display the values like this:

(there are scattered thunderstorms predicted for tomorrow)

Just yesterday I set up a panel to show the last 7 days predictions vs. actual production.

Green/Yellow/Blue is the forecasted low/medium/high. The orange bar is the actual production. It looks like most days I produce between the low and mid values, so I'll probably tweak the system parameters on SolCast after collecting data a bit longer.

[BJReplay]

I'm just curious to see how the predictions match up to reality

I find in summer, the mid values are pretty good. In winter, they are a little more optimistic about the cloud clearing, and the low values are a little pessimistic.

I set up two systems in SolCast (one for each side of my roof) and pull the data once a day,

I pull the forecast data hourly and find that the updated forecasts are useful; between about now (11:30am) and peak (3pm) my charge algorithm in winter gets quite busy as it uses the mid-range forecast, and if the cloud isn't clearing, the updated forecast is for less insolation and that means that my expected shortfall between insolation and expected load through to end of peak increases. That, in turn, means that my required charge at any point increases, and so the algorithm bumps up the reserve setting needed.

This shows it doing its thing yesterday and today.

As soon as the white dashed line goes above the red charge line, the PW starts charging, including a little top up before peak yesterday. Some of the up/down movement in the reserve setting is due to the lumpiness of the load forecast (it is only hourly) and as an hour ticks over, it can cause the algorithm to think that (depending on the actual load in the last hour) that a top up is required or less reserve is required. I haven't bothered refining, because it doesn't result in too much unnecessary battery cycling.

It looks like most days I produce between the low and mid values, so I'll probably tweak the system parameters on SolCast after collecting data a bit longer.

Good idea!