|
Week Twenty CalTRACK Update Over the past three weeks, CalTRACK methods testing has revolved around issues that need resolution to facilitate pay-for-performance using hourly savings. In particular, the focus has been on (i) testing and validating the Time-Of-Week and Temperature model for residential buildings and (ii) scenario analysis of different valuation methods for hourly savings. Other working group members (particularly Home Energy Analytics) contributed significant empirical results that will help in improving the robustness of the CalTRACK methods. This type of participation is the foundation for improving CalTRACK methods. Thank you for the great work! Hourly methods improvements Background: The default Time-Of-Week and Temperature model allows for extended baseline periods when fitting baseline models. When the model adaptation function is not used, a single model can be fit to the entire baseline period, which could be up to 12 months long. The single, yearly regression approach assumes that base load and weather sensitivity of energy consumption is constant throughout the year. Empirical Results: Empirical evidence shows that baseline and weather-related energy use varies during different months of the year. This variation is not represented when a single regression is estimated for the entire baseline period. Below are two potential modeling approaches:
 One potential problem that appears when models are fit with data from limited time periods is that without many data points, they tend to overfit the data. We can see evidence of overfitting by looking at the relationship of model error from within-sample to the model error when applied to out-of-sample data. Large discrepancies between the two values indicate potential overfitting. This relationship is evident in the figure below.  Recommendation:
After reviewing the results of the empirical testing, we recommend applying a three-month weighted regression model for residential hourly methods. Twelve models will be fit for each month of the year, with months before and after the month of interest weighted down by 50%. For example, when predicting the counterfactual energy usage for the month of July, the corresponding baseline model will be fit using data from June, July and August of the previous year. The data points from June and August will be assigned a 50% weight compared to the data points from July. This approach accounts for varying energy consumption patterns across months of the reporting period without overfitting the model to limited data. Keep an eye out for next week’s blog post where we’ll summarize the testing of valuation methods for hourly savings.
6 Comments
Week Eighteen CalTRACK Update Our second week of discussion on hourly portfolios included a working group meeting. You can watch the full meeting here: The value of hourly energy savings varies by use case. As such, the CalTRACK framework should be flexible to accomodate for the priorities of different use cases. A load impact factor assigns more value to energy savings during certain hours of the reporting period. For example, if a utility values energy savings 3 times more from 4-8 PM, then hourly energy savings will be multiplied by a load impact factor of three during the hours of 4-8 PM for each day of the reporting period. Each use case may need to define the load impact factors according to their desired outcomes. The portfolio-level savings are calculated by multiplying estimated hourly energy savings by the hourly load impact factor and aggregating across all hours in the reporting period. In the working group meeting we reviewed three basic methods calculating portfolio-level energy demand savings: Constant Marginal Pricing: A constant marginal pricing scheme uniformly values energy savings at each hour of the year. In this case, the load impact factor for each hour of the year is 1. This method is the simplest to compute and does not require additional data, but it fails to account for load impacts of energy savings. Static Peak Pricing: A static peak pricing scheme accounts for increased value of energy savings during peak consumption periods. A certain range of hours (ie-4-8 PM) is defined as the daily peak consumption period. The designated peak consumption period is imposed on all days of the reporting period. Static peak pricing assigns hourly load impact factors of 1 to all hours except for hours during the peak consumption period. Peak consumption period hours are given load impact factors that are greater than 1, which reflects the higher value of energy savings during these hours. Although Static Peak Pricing accounts for load impacts, its price signals are imperfect because they assume a constant peak consumption period. In reality, the peak consumption period varies throughout days of the year due to seasonality and other energy consumption trends. Real-Time Pricing: A Real-Time Pricing scheme assigns a unique load impact multiplier to each hour of the year. The load impact multipliers can be calculated according to locational marginal energy prices (LMPs). The real-time pricing scheme provides the most accurate price signals for energy savings with respect to grid impacts. The Real-Time Pricing scheme generates the most accurate price signals, but requires LMP data and there is uncertainty in hourly prices.  Empirical Testing:
In the coming weeks, we would like to empirically test methods for calculating portfolio-level energy savings for common use cases. Some use cases include:
Week Seventeen CalTRACK Update Establishing guidelines for aggregating building-level hourly energy savings into portfolio loadshapes requires careful consideration of information and uncertainty-level preferences for various use cases. The issues with aggregating energy savings for hourly methods and potential solutions are outlined below:  Aggregation Method for Billing Period and Daily Methods: In daily and billing period methods, building-level savings are generated by summing the building’s estimated energy savings for each day or billing period of the reporting period. The portfolio savings are then calculated by aggregating total savings for each building in the portfolio. Why are daily and billing period aggregation methods problematic with hourly models? For hourly models, portfolio uncertainty is difficult to calculate when savings are aggregated for each hour due to correlation in the error term. Suggested Hourly Aggregation Methods: To begin our discussion of hourly aggregation methods, consider two types of roll-ups: Vertical Roll-Ups: In a vertical roll-up, hours within a day are grouped together for each building before aggregation. For example, one may choose to aggregate hourly energy savings in three-hour intervals throughout the day instead of each hour individually. Although vertical roll-ups can reduce portfolio uncertainty, larger time intervals provide less information in portfolio loadshapes. Hourly methods are created to provide granular information about energy load impacts during each time-of-day. Less information is available if hours are “rolled-up” into larger time intervals. Horizontal Roll-Ups: Horizontal roll-ups aggregate each hourly estimate with estimates of the same hour across weeks. A horizontal roll-up can aggregate individual hours or time intervals, such as the three-hour interval discussed above.  Other Considerations: There were a few additional suggestions from the working meeting (5/24) that could help create guidelines for aggregating portfolio loadshapes:
Homework:
Week Sixteen CalTRACK Update  During the standing meeting on 5/24, the working group finalized hourly methods for calculating hourly energy savings and commenced discussion on aggregating hourly savings into portfolio loadshapes. The finalized hourly methods and an introduction on aggregating portfolio loadshapes are outlined below. Finalized Hourly Methods:
Aggregating Hourly Savings into Portfolio Loadshapes
To provide an accurate valuation of energy efficiency as a grid resource, energy savings must be quantified at specified time intervals and geographic locations. To create portfolio loadshapes, building-level savings must be aggregated. The method of aggregation has implications on the portfolio uncertainty and provides different granularity of information for aggregators, utilities, and customers. Different use cases may prefer different aggregation methods based on priorities specific to their use case. To accommodate different use cases, flexible methods for aggregating hourly savings into portfolio loadshapes may be preferred. As we explore this topic further, some potential use cases to consider are: Pay-for-Performance Programs In the PG&E pay-for-performance program, the utility provides incentives for peak savings. This requires estimates of portfolio savings at the hourly level. Non-Wires-Alternative Procurement Non-Wireless-Alternative procurements require estimates of portfolio savings for buildings connected to specific grid nodes in order to measure grid impacts and potentially avoid infrastructure investments. Cap and Trade, Greenhouse Gases, or Carbon Tracking or Trading Initiatives Initiatives attempting to accurately quantify carbon offsets from energy efficiency investments require savings estimates at specified time and geographic locations because generation portfolios utilize resources with different carbon intensity at different times and locations. We discussed a few options for aggregation methods, and look forward to input from the working group in the coming week. Week Fourteen & Fifteen CalTRACK Update Review of hourly method proposals continued in week fourteen of CalTRACK 2.0 and will be finalized at the 5/24 working group meeting. Lawrence Berkeley National Lab’s Time-of-Week Temperature (TOWT) model is the specification to be used in CalTRACK 2.0. Mathieu et al. describe the application of TOWT models in Quantifying Changes in Building Electricity Use, with Application to Demand Response. Overview of TOWT models: As energy efficiency finds its legs as a grid resource, time dependent savings will be essential to the value proposition. Pay-for-performance programs can leverage this value with accurate building-level energy savings calculations at granular time intervals. TOWT models are one method for calculating energy savings at the hourly level. Strengths:
 Weaknesses:
Week Thirteen Update for CalTRACK  During week thirteen, the CalTRACK working group discussed proposals for hourly methods in the standing meeting. The discussions included helpful suggestions of other reference materials as well as variations that may be appropriate for different applications of hourly methods and suggested improvements in CalTRACK 2.0’s documentation. The video from May 3, 2018 is provided at the end of this post. Hourly Methods In the development of hourly methods, the goal is to establish guidelines that mirror the methodology in billing period and daily methods. However, hourly methods have unique complexities that require departures from billing period and daily methods. These complexities are identified and discussed below: Data Management When compared to hourly data, daily and billing period savings calculations have higher data sufficiency requirements because hourly data contains more information per time period. This characteristic of hourly data supports the two adjustments to hourly data sufficiency requirements listed below: 1. Usage data sufficiency will be specified in terms of data coverage (or common support) instead of a minimum time period: Daily and billing period data sufficiency requirements impose a minimum quantity of time observed from a year of data. In hourly methods, usage data sufficiency will be specified in terms of data coverage in the independent variables. In Time of Week and Temperature (TOWT) models, the independent variables are temperature and occupancy. Data sufficiency requirements will be based on LBNL recommendations for data coverage. 2. Missing Data Temperature has less variation between hours than days or billing periods. Smaller temperature variation between hours increases the likelihood that interpolated temperature values are accurate. For this reason, interpolated temperature values will be allowed in the reporting period for hourly methods. The threshold of allowable interpolated hours will be determined through empirical testing. Recommendation for Pay-for-Performance Use Case:
Time Of Week Temperature (TOWT) Modeling Approach The TOWT model, originally by Lawrence Berkeley Lab, contains two covariates: Occupancy: Occupancy is an indicator variable that takes the value of 1 if the building is occupied in the hour and 0 otherwise. In LBNL’s model, occupancy of a building is defined by:
The TOWT model allows user-defined temperature bins for modeling a building’s weather dependence. We are recommending setting 7 fixed bins with endpoints at 30, 45, 55, 65, 75, 90, in order to cover a wide variety of climate conditions. Use Case and Uncertainty Time-aggregated Uncertainty: In the program evaluation use case, an analyst may be interested in obtaining time-aggregated savings and uncertainty. Due to residual autocorrelation at the hourly level, aggregating hourly uncertainty for larger time intervals creates imprecise standard errors and uncertainty calculations. Instead, we recommend using daily methods with improved ASHRAE or Ordinary Least Squares (OLS) formulations of Fractional Savings Uncertainty (see Koran 2017) for aggregating uncertainty over time periods. Hour-level Uncertainty Estimates: For the procurement and pay-for-performance use cases, regression analysis is an effective tool for acquiring point estimates of savings and uncertainty at each hour. If each building is assumed to have independent errors, the uncertainty at each hour for all buildings in the portfolio can be aggregated without an autocorrelation problem. Methods Documentation Currently, the documentation for CalTRACK 2.0 is being updated. The first half of CalTRACK 2.0’s documentation will be posted on GitHub to allow the working group to review and comment on the changes in documentation. Similar to the methods, the development of effective documentation is an iterative process. The documentation for CalTRACK 2.0 will improve by dividing into three distinct documents: 1. Methods This document outlines the methodology for quantifying billing period, daily, and hourly energy savings while maintaining CalTRACK-compliancy. In CalTRACK 2.0, the Methods will be organized with a numbering system that corresponds to the Methodological Appendix. This will make referencing and accessing the appendix easier. 2. Methodological Appendix The Methodological Appendix summarizes discussions and empirical testing that justify methodological decisions. The Methods will reference sections in the Methodological Appendix for readers to easily access empirical support for methodological decisions. 3. Field Guide A document with minimum requirements for an implementation to maintain CalTRACK-compliancy. This is designed to be a practical and accessible checklist for analysts and other implementers of CalTRACK. Ideas for Future CalTRACK Work A sandbox has been added to the GitHub site to document proposals for participants to add ideas for future CalTRACK iterations. If you have an idea for CalTRACK 3.0, or beyond that cannot be addressed this year, please add it the the sandbox. Additional Hourly Methods Resources
Homework Week Twelve CalTRACK Update During week twelve, we continued our discussion of hourly methods. In the upcoming week, we will analyze test results for hourly methods on GitHub. We will also be talking about how to log ideas for future improvements. The standing meeting to discuss hourly methods will be on Thursday, May 3rd at 12:00 (PST).  Homework:
Week Eleven CalTRACK Update Week eleven was the first week of hourly methods discussion. Developing hourly methods will require discussion and empirical testing of topics unique to hourly methods before we can make final specifications. Topics that must be addressed include:
Time Of Week Temperature Models (TOWT): A proposed model for hourly methods is the TOWT model from Lawrence Berkeley National Labs (LBNL) is shown below.   Notes:
Homework:
Week Ten CalTRACK Update The CalTRACK working group finalized discussions on Building Qualifications during the first half of Thursday’s (4/12) meeting and dove into the hourly methods during the second half. Bill Koran from SBW consulting provided a helpful overview of hourly models and the ECAM energy data analysis tool. The major findings of this meeting are summarized below: Building Qualification Observations and Recommendations: Main observations that were driving the recommendations:
 Goals for Hourly Methods: Hourly models are necessary for estimating the load impact of energy efficiency. This makes hourly energy savings important information for aggregators and utilities in an energy efficiency marketplace. Our goal is to establish suitable methods for calculating whole building hourly energy savings for residential and commercial buildings. Additionally, the methods will include guidelines for aggregating site-level savings. Discussion Topics: We have allotted three weeks to test and discuss hourly methods. Below are some important topics that will need to be addressed:
Homework:
Week Nine CalTRACK Update During week nine of CalTRACK, there were continued discussions on building qualification criteria and some introductory comments on hourly methods. The working group meeting will be held on Thursday, April 12th at 12:00 (PST). We will discuss:
Hourly Methods Overview: Hourly methods are a new addition in CalTRACK 2.0 and were not in the first version of CalTRACK. This task involves testing various hourly modeling methods and recommending a standardized approach to hourly modeling, which can reveal the time value of energy efficiency. Importance of Hourly Methods:
 Time Of Week and Temperature Models: To start the discussion of hourly methods, it is helpful to consider to existing open source tools. Lawrence Berkeley National Lab has developed an open source time of the week and temperature (TOWT) model to calculate hourly energy savings and is part of the RMV2.0 - LBNL M&V2.0 Tool. A TOWT model predicts hourly energy savings by utilizing hourly temperature data instead of daily or billing period HDD and CDD. Another is ECAM (ENERGY CHARTING & METRICS) developed by Bill Koran at SBW consulting. We look forward to discussion on the working group call about these approaches, and more to frame the approach to empirical tests for the CalTRACK 2.0 hourly methods. Final Note: It is important to remember that CalTRACK methods development is an iterative process. The finalized methods for CalTRACK 2.0, just like v. 1.0, will benefit from field deployment and may need to be revised in future years. We expect this process to result in improved and refined methods with successive iterations. Homework:
|
The purpose of this blog is to provide a high-level overview of CalTrack progress.
For a deeper understanding or to provide input on technical aspects of CalTrack, refer to the GitHub issues page (https://github.com/CalTRACK-2/caltrack/issues). Recordings
2019 CalTRACK Kick Off:
CalTRACK 2.0 July 19, 2018 June 28, 2018 June 7, 2018 May 24, 2018 May 3, 2018 April 12, 2018 March 29, 2018 March 15, 2018 March 1, 2018 February 15, 2018 February 1, 2018 Archives
April 2024
|
RSS Feed
