Week Twenty CalTRACK Update
Over the past three weeks, CalTRACK methods testing has revolved around issues that need resolution to facilitate pay-for-performance using hourly savings. In particular, the focus has been on (i) testing and validating the Time-Of-Week and Temperature model for residential buildings and (ii) scenario analysis of different valuation methods for hourly savings. Other working group members (particularly Home Energy Analytics) contributed significant empirical results that will help in improving the robustness of the CalTRACK methods. This type of participation is the foundation for improving CalTRACK methods. Thank you for the great work!
Hourly methods improvements
The default Time-Of-Week and Temperature model allows for extended baseline periods when fitting baseline models. When the model adaptation function is not used, a single model can be fit to the entire baseline period, which could be up to 12 months long. The single, yearly regression approach assumes that base load and weather sensitivity of energy consumption is constant throughout the year.
Empirical evidence shows that baseline and weather-related energy use varies during different months of the year. This variation is not represented when a single regression is estimated for the entire baseline period. Below are two potential modeling approaches:
One potential problem that appears when models are fit with data from limited time periods is that without many data points, they tend to overfit the data. We can see evidence of overfitting by looking at the relationship of model error from within-sample to the model error when applied to out-of-sample data. Large discrepancies between the two values indicate potential overfitting. This relationship is evident in the figure below.
After reviewing the results of the empirical testing, we recommend applying a three-month weighted regression model for residential hourly methods. Twelve models will be fit for each month of the year, with months before and after the month of interest weighted down by 50%. For example, when predicting the counterfactual energy usage for the month of July, the corresponding baseline model will be fit using data from June, July and August of the previous year. The data points from June and August will be assigned a 50% weight compared to the data points from July. This approach accounts for varying energy consumption patterns across months of the reporting period without overfitting the model to limited data.
Keep an eye out for next week’s blog post where we’ll summarize the testing of valuation methods for hourly savings.
The purpose of this blog is to provide a high-level overview of CalTrack progress.
For a deeper understanding or to provide input on technical aspects of CalTrack, refer to the GitHub issues page (https://github.com/CalTRACK-2/caltrack/issues).
2019 CalTRACK Kick Off:
July 19, 2018
June 28, 2018
June 7, 2018
May 24, 2018
May 3, 2018
April 12, 2018
March 29, 2018
March 15, 2018
March 1, 2018
February 15, 2018
February 1, 2018