Perfect Foresight vs. Uncertainty: Optimising your Energy Modelling Projects

At Gridcog we don’t claim to see the future, but we do give our users the tools to plan for it effectively.

In the evolving energy landscape, optimising highly steerable assets like batteries, EV fleets, or flexible loads in the real world hinges on understanding what’s about to happen, both in terms of price signals and patterns of consumption or generation. 

Maybe there’s a great opportunity for your battery to sell into the intraday market in a few hours time, but first you need to make sure the battery is fully charged otherwise that opportunity cannot be fully realised. Or perhaps you want to top up the batteries of your EV fleet when prices are lowest, but do you know exactly when that will be? 

In modelling software like Gridcog, the optimisation process can be vulnerable to what’s known in the trade as “perfect foresight”. Essentially, the assets in the model will operate flawlessly, buying the lows, selling the highs, charging when there’s solar to charge from, discharging into a peak demand event to reduce network tariff costs. You get the idea. 

Modelling with perfect foresight is a useful and legitimate approach as it provides a top end for revenue forecasts for a project, but it doesn’t reflect “real life”, where short term forecasts can be murky at best and will deliver sub-optimal commercial returns. 

That’s why we include some specific features in the Gridcog software that allow our clients to introduce uncertainty into their models and generate more robust financial forecasts. .

In this context, within Gridcog, uncertainty involves perturbing forecast data that is fed to the optimisation engine, such that the optimisation makes mistakes, just as an optimiser would, operating real assets in the market. The precise approach can be tuned by the user to reflect their internal forecasting capabilities, or that of their control software. 

For the rest of this blog, we'll focus on market price uncertainty to illustrate our approach, though very similar principles apply to load or generation uncertainty.

Managing Short-Term Market Price Uncertainty

We enable users to apply well known mathematical forecasting techniques used by traders and optimisers to make decisions with imperfect information. These techniques help smooth out short-term price fluctuations, simulating more realistic trading conditions. The techniques we use are:

📉 EMA (Exponential Moving Average) – Reduces short-term price fluctuations. Reacts faster to price changes than a conventional moving average by weighting more recent prices more heavily.

📉 DEMA (Double Exponential Moving Average) – More responsive to more recent price changes than EMA but still lags behind real-time price shifts, mimicking real-world trading constraints.

📉 Shaped Moving Average – Combines DEMA with the principles of CAISO ‘10 of 10’. We first select some historic days to average prices over and then apply DEMA to adjust scaling based on the most recent trends. 

These methods simulate how traders struggle to anticipate short-term price swings, leading to more realistic optimisation models compared to ‘perfect foresight.’

To illustrate what this all means, let’s look at an example, say January 8, 2025, in GB, when energy prices soared due to high demand along with supply shortages. We compared actual prices with forecasted prices using the techniques introduced above:

• Actual prices were highly volatile.
• DEMA reacted but didn’t capture the full spread, mirroring real-world trader limitations.
• EMA performed worse, missing much of the volatility due to lower emphasis on recent data.
• Shaped Moving Average (SMA) best captured the price curve’s shape but underestimated the magnitude of the event.

Applying these techniques to historical data illustrates how traders may miss optimal trades due to uncertainty about the future. Whilst this isn’t a perfect reflection of real-life trading conditions, it allows our clients to better understand the sensitivity of their project revenues from different value streams.

Uncertainty in Gridcog

So, how might applying these short-term uncertainty methods impact the outcome of a battery storage investment? 

To answer this, we spun up a basic project in Gridcog that has a 50MW/1h battery trading in the three GB wholesale energy markets (Day-Ahead Hourly, Day-Ahead Half hourly and Intraday Continuous). The table summarises results of the different uncertainty methods, zooming in on January 8 2025 in line with Figure 1.

As shown in Figure 2, all scenarios we analysed had the BESS actively importing and exporting energy during periods of high price volatility—most notably between 1 PM and 7 PM, shown in Figure 1. However, when the different uncertainty settings were applied, the battery behaviour varied resulting in different trading outcomes.

The added forecast uncertainty caused the battery to miss out on the most optimal trading opportunities to varying degrees, in some cases trading too much at suboptimal periods (EMA and DEMA) or trading too little during optimal periods (SMA). In essence, while the BESS still responded to market fluctuations, the extent of its effectiveness depended on how much uncertainty was factored into the trading strategy.

By adjusting uncertainty settings such as the method and the hours included in the rolling average,  it is possible to create a range of projected earnings, enabling users to make robust investment decisions under uncertainty. This highlights an important takeaway: real-world asset dispatch and trading is based on realistic data-driven decision-making, not perfection, and project modelling should be able to reflect that.

Conclusion

As renewable energy penetration increases, price volatility in wholesale electricity markets is expected to rise. Additionally, policy changes such as P415 are enabling behind-the-meter (BTM) storage to participate in these markets, opening up new revenue opportunities.

For energy developers and investors, understanding how battery storage or flexible assets interact with market volatility is crucial for financial success. Rarely, does a battery investment, be it behind-the-meter or front-of-meter, make sense without leveraging price volatility.

At Gridcog, we provide the modelling tools necessary to evaluate how different market strategies impact financial outcomes for renewable energy investments. Whether you’re an investor, energy trader, or project developer, our platform helps you make data-driven decisions in an increasingly complex energy market.

‍

‍

‍