CalTRACK
  • CalTRACK Methods
    • CalTRACK Process
    • CalTRACK Compliance
    • Project Updates
    • Technical Working Group >
      • Technical Appendix
      • Issues and Evidence
    • CalTRACK History
    • Stakeholders
  • LFE OpenEEmeter
    • OpenEEmeter Overview
    • Github Code
    • EEweather
    • Documentation
  • FLEXmeter
    • GRIDmeter
    • Energy Differential Privacy
  • Contact

Hourly Methods for Pay for Performance

6/29/2018

6 Comments

 
Week Twenty CalTRACK Update
Over the past three weeks, CalTRACK methods testing has revolved around issues that need resolution to facilitate pay-for-performance using hourly savings. In particular, the focus has been on (i) testing and validating the Time-Of-Week and Temperature model for residential buildings and (ii) scenario analysis of different valuation methods for hourly savings. Other working group members (particularly Home Energy Analytics) contributed significant empirical results that will help in improving the robustness of the CalTRACK methods. This type of participation is the foundation for improving CalTRACK methods. Thank you for the great work!
June 28, 2018 Working Group Recording
Hourly methods improvements
Background:
The default Time-Of-Week and Temperature model allows for extended baseline periods when fitting baseline models. When the model adaptation function is not used, a single model can be fit to the entire baseline period, which could be up to 12 months long.  The single, yearly regression approach assumes that base load and weather sensitivity of energy consumption is constant throughout the year.
Empirical Results: 
Empirical evidence shows that baseline and weather-related energy use varies during different months of the year. This variation is not represented when a single regression is estimated for the entire baseline period. Below are two potential modeling approaches:
  1. A regression approach that estimates one model for the entire baseline period, which is 1 year in this case.
  2. A regression approach that estimates 12 models for each month of the baseline period, which is 1 year in this case.
It is evident that base load energy consumption, which is the green portion of the graphs below, is not constant throughout the year. The failure to account for varying base load energy consumption across the baseline period contributes to higher model variance, measured by CVRMSE, in CalTRACK methods.
Picture
One potential problem that appears when models are fit with data from limited time periods is that without many data points, they tend to overfit the data. We can see evidence of overfitting by looking at the relationship of model error from within-sample to the model error when applied to out-of-sample data. Large discrepancies between the two values indicate potential overfitting. This relationship is evident in the figure below. ​
Picture
Recommendation:
After reviewing the results of the empirical testing, we recommend applying a three-month weighted regression model for residential hourly methods. Twelve models will be fit for each month of the year, with months before and after the month of interest weighted down by 50%. For example, when predicting the counterfactual energy usage for the month of July, the corresponding baseline model will be fit using data from June, July and August of the previous year. The data points from June and August will be assigned a 50% weight compared to the data points from July. This approach accounts for varying energy consumption patterns across months of the reporting period without overfitting the model to limited data.
​

Keep an eye out for next week’s blog post where we’ll summarize the testing of valuation methods for hourly savings.
Homework:
  • Review final methods documentation and provide comments on pull request on GitHub
  • Review final hourly methods on GitHub
  • Provide feedback on portfolio load shape results on GitHub
  • Contribute to the Sand Box of future issues
The next working group meeting is in 3 weeks on July 19, 2018.
6 Comments
Fernando Riley link
10/7/2022 03:27:48 pm

Wife whether style. Career hospital clear investment particular hand.
Past bank his feel happy strategy since. Whose account subject involve. Pattern season nearly the design event peace.

Reply
Charles Perez link
10/9/2022 04:20:31 am

Everyone dream foot realize east. Heart away care. Include section three quite director.
Walk wall meeting machine such television. Politics clearly key community.
Few space wear tough music.

Reply
Mark Salinas link
10/13/2022 02:52:26 am

You remember base federal. Miss trade them. Form second magazine phone.
Likely treat office other. Very future boy drive from government might popular.

Reply
John Cooper link
10/13/2022 11:34:54 pm

Both hair against bar. Treatment right product of. Heavy small will peace visit none.

Reply
Jason Turner link
10/20/2022 07:42:19 pm

Five conference community religious itself. Window middle sometimes week change though. Down house serve good group heavy together.

Reply
Oscar Gutierrez link
10/28/2022 05:22:58 am

Daughter collection responsibility fear somebody letter. Sound however happy full successful.
Finish test person. Maybe seek over something religious eye majority.

Reply



Leave a Reply.

      Sign Up for Technical Working Group Updates

    Subscribe to Newsletter
    The purpose of this blog is to provide a high-level overview of CalTrack  progress.
    ​
    For a deeper understanding or to provide input on technical aspects of CalTrack, refer to the GitHub issues page (
    https://github.com/CalTRACK-2/caltrack/issues). 
    Recordings
    2019 CalTRACK Kick Off:

    CalTRACK 2.0 
    July 19, 2018
    June 28, 2018
    June 7, 2018
    May 24, 2018
    May 3, 2018
    April 12, 2018
    March 29, 2018
    March 15, 2018
    March 1, 2018
    ​
    February 15, 2018
    February 1, 2018

    Archives

    February 2023
    January 2023
    December 2022
    November 2022
    July 2019
    March 2019
    February 2019
    August 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018

    RSS Feed

Creative Commons License
Creative Commons Attribution-NoDerivatives 4.0 International License.