CalTRACK
  • CalTRACK Methods
    • CalTRACK Process
    • CalTRACK Compliance
    • Project Updates
    • Technical Working Group >
      • Technical Appendix
      • Issues and Evidence
    • CalTRACK History
    • Stakeholders
  • LFE OpenEEmeter
    • OpenEEmeter Overview
    • Github Code
    • EEweather
    • Documentation
  • FLEXmeter
    • GRIDmeter
    • Energy Differential Privacy
  • Contact

December 6, 2022 OpenEEmeter | Technical Working Group Meeting Recap

12/26/2022

0 Comments

 
Watch the 12/06/2022 technical working group meeting on demand or below. If you haven't signed up yet, click here to register for the next meeting. 
0 Comments

CalTRACK 2.1 | OpenEEmeter Kick-Off Recap

11/29/2022

0 Comments

 
Thank you to everyone who was able to make it to our CalTRACK 2.1 | OpenEEmeter kick off on 11/22. If you were unable to attend check out the recording available on demand or below. We look forward to continuing this work at next week's meeting on Tuesday, 12/6/2022.
Sign up to get involved and join the next Technical Working Group meeting.
Sign Up
0 Comments

CalTRACK 2.1 Working Group Kick-Off 2022

11/2/2022

0 Comments

 
LF Energy and the OpenEEmeter project team are pleased to re-launch the CalTRACK working group to incorporate updates to the methods and code and formally enfold CalTRACK under the OpenEEmeter project umbrella.
Get Involved
Picture

​What Is CalTRACK
The CalTRACK methods and OpenEEmeter codebase are a standard and transparent set of procedures designed to calculate savings based on metered energy consumption. Since their development these procedures have been used to measure the impacts of dozens of energy efficiency programs across the United States and as a settlement tool for performance-based demand flexibility markets. 
​​
​The original CalTRACK working group members included the California Energy Commission, the California Public Utilities Commission, Pacific Gas and Electric Company, DNV GL, Energy Savvy, and a host of other contributing members (CalTRACK history). The OpenEEmeter codebase was donated to Linux Foundation Energy (LFE) as an open-source project in 2019 to culminate the group's efforts.

What's Next
In the last few years, the need for accurate and standardized meter-based methods has grown, along with the market for scalable and integrated demand-side programs. In 2020-2021, COVID-19 disruptions heightened the importance of identifying and removing the impacts of external (non-program) factors on energy consumption. In partnership with the Department of Energy, Recurve developed the GRIDmeter comparison group methods to address this challenge. Similarly, California's 2020 heat wave and rolling power outages exposed the necessity for more reliable demand response methods. In response, Recurve developed the FLEXmeter methods on behalf of the California Independent System Operator. CalTRACK and GRIDmeter are the foundations of FLEXmeter, which can accurately measure the impact of demand response events, even during extreme heat waves, and, importantly, support integrated EE and DR measurements. 

While CalTRACK 2.0 is foundational to all of these approaches, we believe that further updates to the methods and the OpenEEmeter code will promote greater accuracy and confidence in the impacts of behind-the-meter resources and could help bring a wider variety of demand-side programs to the table. We look forward to working with other stakeholders and partners who have used CalTRACK and the 
OpenEEmeter for their own initiatives and would like to contribute to this process.
​
On November 22, 2022 Recurve will host the CalTRACK 2.1 kickoff meeting. The CalTRACK 2.1-OpenEEmeter development process will follow the LFE Charter and Code of Conduct.  


Methods development is an experimental process that can benefit from the input and experiences of many. The collaborative feedback loop is an exciting aspect of open-source development. The successes and failures at every stage represent progress for all to see and provide many points for collaboration and for new ideas to emerge. We can't wait to get the band back together.
Get Involved
0 Comments

LF Energy Webinar on the OpenEEmeter

7/11/2019

0 Comments

 
Picture
Phil Ngo, Director of Engineering at Recurve, provides insights into the OpenEEmeter and EEweather, which is an open source implementation of the CalTRACK Methods that resides in LF Energy.

0 Comments

Webinar: Getting Going with CalTRACK

3/13/2019

0 Comments

 
Picture
This one hour training will cover the origin of CalTRACK, the scope and appropriate application of the methods, and how to validate that a tool or approach is a verifiable execution of the standard method.

​Learn how CalTRACK can be used to standardize M&V to enable greater confidence in savings for regulators, buildings owners, utilities, and third party finance.


This training will be useful for efficiency regulators, implementers, evaluators, administrators and anyone who is interested in learning more about how to become part of the CalTRACK process and contribute to its ongoing development.

The next training is on March 20, at 10:00 AM Pacific, and will be hosted by McGee Young.  Click here a few minutes before the meeting to join.
EM2 Google Events Calendar

​Monthly CalTRACK Training
March 20, 2019
10:00 AM (Pacific) 
Hosted by McGee Young
Learn the basics about CalTRACK’s origins and methods in this one-hour introductory class. 

Check out the full EM2 Calendar for trainings and working groups.

CalTRACK March Training: https://zoom.us/j/185257351

One tap mobile
+16699006833,,185257351# US (San Jose)
+16465588656,,185257351# US (New York)

Dial by your location
       +1 669 900 6833 US (San Jose)
       +1 646 558 8656 US (New York)
Meeting ID: 185 257 351
Find your local number: https://zoom.us/u/aefkrPrmr3

0 Comments

Energy Market Methods Consortium (EM2) Kickoff

2/23/2019

2 Comments

 
Picture
The next phase of collaboration on CalTRACK has commenced!  CalTRACK and the OpenEEMeter are migrating to new governance structures and “homes” aligned with the Linux Foundation, one of the largest supporters of open source projects in the world. 

CalTRACK methods will continue to be developed under the umbrella of a new group called the Energy Market Methods Consortium (EM2). The CalTRACK methods working group will continue to address updates to the avoided energy use tool and two other working groups will tackle the related topics of adjustments for grid integration (GRID) and secure data transfer (SEAT). The governance of this the project and the three working groups will be under the umbrella of charters approved by the Joint Development Fund. 

Bruce Mast of Ardenna Energy will serving as an Interim Executive Director to administer the processes outlined in the EM2 charter. Those who would like to join a working group or the technical steering committee can contact Bruce for more information on membership.  All meetings are open to the public.
A video of the kick off meeting, held on February 19th, provides more detail on the structure of the EM2. Unfortunately the recording picked up the video boxes which may impair viewing of some of bullets; so the slide deck is available below the video link.
em2_feb_19_2019.pdf
File Size: 437 kb
File Type: pdf
Download File

2019 Calendar For Kick Off, Working Group and Trainings in EM2
A public Google Calendar (Shared EM2 Calendar) includes all EM2 events (kick off, training, and working group meetings).  You can copy specific events to your calendar or link to the public calendar.
Sign up on Github to track progress on the OpenEEMeter.
Thank you to all who have contributed in the past. We look forward to continued collaboration and progress in this new phase of development. 
2 Comments

Quick Links to the 2018 Project Updates

8/10/2018

3 Comments

 
​From February to July 2018 the CalTRACK working group covered four key topic areas and reported back on progress roughly each week.   The list below provides quick access to the summaries by topic.
Updates to Daily and Billing Period Methods:
  • Proposed Testing
  • Updates to  Documentation
  • Results from Methods Testing​
Building Qualifications: ​
  • ​Metrics & Model Selection
  • Considering Metrics 
  • Test Reveals Wide Applicability of CalTRACK Method for Portfolio Analysis
  • Criteria Discussions
  • Results & Recommendations
Site Specific Hourly Methods: ​​​
  •  Approach & Testing Considerations
  • Discussions Continue
  • Data & Use Cases
  • Time of Week and Temperature Model CalTRACK Application for Site Specific Hourly Savings
  • Site-Specific Hourly Methods Finalized




​Portfolio Hourly​​ Methods:
  • Aggregation Approaches and Uncertainties
  • Tackling Hourly Savings for a Portfolio
  • Hourly Pay for Performance
  • Valuation Approaches in the CalTRACK Methods
  • Finalized Method
3 Comments

Working Group Finalizes Hourly Method

7/23/2018

0 Comments

 
​Week Twenty-Two CalTRACK Update
Last week marked the final CalTRACK 2.0 working group meeting.  In this meeting we discussed an update to hourly methods, an overview of our progress during CalTRACK 2.0, and suggestions for CalTRACK 3.0.  You can view the final meeting at following link:
July 19, 2018 Working Group Meeting
Hourly Methods Update:
Empirical testing has shown correlation between energy consumption and season in residential buildings, that can have an effect on the savings error. The CalTRACK 2.0 working group has proposed accounting for this seasonal effect by shortening the 12-month baseline period to 3-month weighted baseline periods. The figure below shows the effect of shortening baseline periods to 3-month on Normalized Mean Bias Error (NMBE) for residential buildings.
Picture
However, shortening an annual baseline to 3-month weighted baselines may not be necessary for all building types. Notably, commercial buildings tend to have a smaller seasonal effect than residential buildings, and may not experience increased NMBE from using a 12-month baseline period.
The working group has established the following NMBE thresholds to define buildings that require 3-month weighted baselines and those where an annual baseline period is acceptable.
  1. If there are 2 or more months that have an NMBE greater than 0.01, then 3-month weighted baselines are required.
  2. If there are less than 2 months that have an NMBE greater than 0.01, then 3-month weighted baselines are optional.
CalTRACK 2.0 Recap:
Since February, the CalTRACK 2.0 process has tackled several major issues.  Below is a quick synopsis of the major tasks addressed and outcomes for these topics.
Picture
Task 1: Updates to CalTRACK daily and billing methods based on feedback from CalTRACK 1.0 users.  Some updates include:
  • Improved weather station mapping.
  • Weighted regression for months in billing period methods.
  • Expanded grid search range for variable balance points.
  • Maximum lengths for baseline and reporting periods in billing period and daily methods.

Picture
Task 2: Assess the feasibility of a portfolio aggregation approach for calculating savings as well as any effects on savings uncertainty.
  • ​For portfolio-based cases, buildings with a high uncertainty metric can still be included in a portfolio as long as the defined portfolio-level uncertainty threshold is not exceeded.
  • For site-based applications, ASHRAE Guideline 14 thresholds were recommended.

Task 3: Develop a prototype method for calculating hourly savings.
  • Lawrence Berkeley National Lab’s Time-of-Week and Temperature model was used as a template for hourly methods.
  • The aggregated hourly energy savings estimates were stable when considered at the portfolio-level for residential houses, which was an encouraging finding.
Task 4: Demonstrate how price signals can adjust the value of hourly load shapes to match procurement needs.
  • When price signals are applied to hourly savings estimates, the temporal and locational value of energy efficiency projects can be calculated.
  • Various price signals were analyzed and it was shown that they can provide different values to different types of load shapes based on priorities of the procurer.
Picture
CalTRACK 3.0:
The direction of CalTRACK 2.0 methods development was guided by feedback from use cases that required “payable savings”. For example, PG&E’s pay-for-performance energy efficiency program decided to increase compensation for energy savings during peak hours during the second iteration of their program. This required CalTRACK 2.0 to develop methods that generate savings estimates at the hourly level. Similarly, we expect CalTRACK 3.0’s tasks will be guided by the demands of stakeholders that implement programs using CalTRACK 2.0 methods.
​

In addition, we have designed a CalTRACK 3.0 sandbox on GitHub to document issues that require further investigation. We encourage working group members to continue adding ideas to the CalTRACK 3.0 sandbox as they arise.
Homework:
  • Use the CalTRACK 2.0 methods​
  • Contribute to the CalTRACK 3.0 sandbox
  • Keep an eye out for the open source CalTRACK-based engine release (eemeter 2.0)

Next Working Group Meeting is in 2019!

0 Comments

Valuation Approaches in the CalTRACK Methods

7/9/2018

3 Comments

 
Week Twenty One CalTRACK Update
Picture
​Energy efficiency and other distributed energy resources have the potential to bring value to the grid. The value of energy efficiency in particular has been calculated with averaged assumptions about avoided costs of producing, procuring and distributing energy from fossil fuel infrastructure, as well as other factors in some cases (e.g. avoided emissions, societal benefits etc.).

​This valuation is estimated at the regulatory level, when assessing the cost-effectiveness of energy efficiency, but it is usually sequestered from market actors, who have the most control over actual generated energy savings. While pay-for-performance is a key step to aligning the value of energy efficiency with the market actors responsible for it, using total annual savings as the performance metric conceals the time- and location-based value of energy efficiency which can vary significantly.  
 
This current disconnect of energy efficiency savings to its actual value to the grid or emissions avoided is an area ripe for improvements in modeling and understanding the real value of delivered energy savings.  Furthermore, it is critical if energy efficiency wants to have a place at the integrated resource planning table. The availability of both hourly savings AND hourly valuations for those savings (e.g. avoided costs or emissions) allows program administrators to estimate, to a much better degree of accuracy, the value of energy efficiency to the grid.
 
In last week’s working group discussion we covered the topic of valuation in the context of hourly load shape analysis conducted through CalTRACK 2.0.

Introduction
Portfolio load shapes  can be used by programs that compensate for hourly energy savings and whose intention is to align the incentives with savings that are valuable to the grid when it is needed. The construction of portfolio load shapes requires:
  1. A model for calculating hourly energy savings
  2. A method for valuing energy savings

​To calculate hourly energy savings, CalTRACK methods utilize a time-of-week and temperature model.
 
To value the energy savings, a valuation method must be applied to the hourly savings calculations.
Overview of Valuation Methods 
There are a range of valuation strategies one can consider; which may have different effects on the outcomes.

1. Constant Valuation           
 Energy savings are valued at a constant price across all hours of the year.

2. Step Valuation During Peak
Energy savings are valued higher during peak hours of day. This valuation method assumes the peak period is the same across all days. For example, a step valuation may value energy savings from 5-8 PM 3 times more than non-peak hours of the day.

3. Avoided Cost Valuation 
Energy savings are valued based on their hourly avoided costs, which provides a unique value for each hour of the year. The total avoided costs include costs associated with transmission, distribution, resource portfolios, carbon and more.
 
4. Avoided Energy Valuation 
Energy savings are valued based on the cost of generating a unit of energy at the given time and location. This is similar to Avoided Cost Valuation, but only costs of generating energy are considered. 
Overview of Examined Scenarios
Different programs and even different measures within a program will deliver different types of time-based value even if their annual savings seem similar.  Here are three fairly common cases to consider for this example:
 
1. Home Performance Scenario 
Home performance improvements, which are primarily focused on weatherization and HVAC, have concentrated energy savings during periods of high and low temperatures, with little to no savings in the shoulder seasons. For example, insulated windows will generate energy savings from reduced heating during the winter and air conditioning during the summer.
 
2. Lighting Scenario 
Lighting improvements provide consistent savings year round, during hours when lighting is required.
 
3. Load Shifting Scenario 
Load shifting equipment moves energy load from high demand periods to lower demand periods. An example is an air conditioning demand response program that reduces electricity demand during peak hours. Net energy savings from load shifting programs are typically low or neutral because they cause increased energy consumption before and after the high demand period. 
Scenario Load Shape Analysis
In the table below, the home performance, lighting, and load shifting scenarios were compared with different valuation methods. The numbers in this table represent relative value across programs (i.e. they are unit-less), making the comparison of the different methods for a particular measure/program not very meaningful. However, when each row in the table is examined separately, the type of savings that are encouraged by each valuation scheme becomes apparent. 
Picture
​The valuation scheme will usually be a policy, market or procurement decision that should consider the ultimate outcomes intended - to ensure incentives and motivation can be aligned.

​For example, if a program aims to reduce electricity grid operating costs, then an avoided cost is the most appropriate valuation method. Under this valuation method, the load shifting program is most effective.
 
However, if a program aims to reduce net energy use, avoided energy valuation may be the most appropriate method. In this context, a market actor may decide that a lighting program is their best option.
Homework:
  • Review final methods documentation and provide comments on pull request on GitHub
  • Review final hourly methods on GitHub
  • Provide feedback on portfolio load shape results on GitHub
  • Contribute to the Sand Box of future issues
 
The next working group meeting is in 2 weeks on July 19, 2018.
3 Comments

Hourly Methods for Pay for Performance

6/29/2018

6 Comments

 
Week Twenty CalTRACK Update
Over the past three weeks, CalTRACK methods testing has revolved around issues that need resolution to facilitate pay-for-performance using hourly savings. In particular, the focus has been on (i) testing and validating the Time-Of-Week and Temperature model for residential buildings and (ii) scenario analysis of different valuation methods for hourly savings. Other working group members (particularly Home Energy Analytics) contributed significant empirical results that will help in improving the robustness of the CalTRACK methods. This type of participation is the foundation for improving CalTRACK methods. Thank you for the great work!
June 28, 2018 Working Group Recording
Hourly methods improvements
Background:
The default Time-Of-Week and Temperature model allows for extended baseline periods when fitting baseline models. When the model adaptation function is not used, a single model can be fit to the entire baseline period, which could be up to 12 months long.  The single, yearly regression approach assumes that base load and weather sensitivity of energy consumption is constant throughout the year.
Empirical Results: 
Empirical evidence shows that baseline and weather-related energy use varies during different months of the year. This variation is not represented when a single regression is estimated for the entire baseline period. Below are two potential modeling approaches:
  1. A regression approach that estimates one model for the entire baseline period, which is 1 year in this case.
  2. A regression approach that estimates 12 models for each month of the baseline period, which is 1 year in this case.
It is evident that base load energy consumption, which is the green portion of the graphs below, is not constant throughout the year. The failure to account for varying base load energy consumption across the baseline period contributes to higher model variance, measured by CVRMSE, in CalTRACK methods.
Picture
One potential problem that appears when models are fit with data from limited time periods is that without many data points, they tend to overfit the data. We can see evidence of overfitting by looking at the relationship of model error from within-sample to the model error when applied to out-of-sample data. Large discrepancies between the two values indicate potential overfitting. This relationship is evident in the figure below. ​
Picture
Recommendation:
After reviewing the results of the empirical testing, we recommend applying a three-month weighted regression model for residential hourly methods. Twelve models will be fit for each month of the year, with months before and after the month of interest weighted down by 50%. For example, when predicting the counterfactual energy usage for the month of July, the corresponding baseline model will be fit using data from June, July and August of the previous year. The data points from June and August will be assigned a 50% weight compared to the data points from July. This approach accounts for varying energy consumption patterns across months of the reporting period without overfitting the model to limited data.
​

Keep an eye out for next week’s blog post where we’ll summarize the testing of valuation methods for hourly savings.
Homework:
  • Review final methods documentation and provide comments on pull request on GitHub
  • Review final hourly methods on GitHub
  • Provide feedback on portfolio load shape results on GitHub
  • Contribute to the Sand Box of future issues
The next working group meeting is in 3 weeks on July 19, 2018.
6 Comments
<<Previous

      Sign Up for Technical Working Group Updates

    Subscribe to Newsletter
    The purpose of this blog is to provide a high-level overview of CalTrack  progress.
    ​
    For a deeper understanding or to provide input on technical aspects of CalTrack, refer to the GitHub issues page (
    https://github.com/CalTRACK-2/caltrack/issues). 
    Recordings
    2019 CalTRACK Kick Off:

    CalTRACK 2.0 
    July 19, 2018
    June 28, 2018
    June 7, 2018
    May 24, 2018
    May 3, 2018
    April 12, 2018
    March 29, 2018
    March 15, 2018
    March 1, 2018
    ​
    February 15, 2018
    February 1, 2018

    Archives

    December 2022
    November 2022
    July 2019
    March 2019
    February 2019
    August 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018

    RSS Feed

Creative Commons License
Creative Commons Attribution-NoDerivatives 4.0 International License.