Thanks to everyone who attended the most recent OpenEEmeter working group meeting.

The meeting began with a discussion by Jason Chulock of coming improvements in the OpenEEmeter 4.0 API, including consolidating usage between all three methods — hourly, daily, and billing — and making certain common configurations the default. Jason then laid out improvements around data sufficiency and methods compliance. The goal of these changes is to make the API more user-friendly and efficient.

Travis Sikes then led a recap and discussion of the current issues and progress on the CalTRACK 2.0 model. Key concerns of CalTRACK 2.0 include its tendency to be overfit, its incompleteness for solar PV customers, and the inflexibility in handling input data. 

Travis explained that the team would be using AMI measurements combined with weather, solar, and categorical data to enhance prediction accuracy. He then discussed evolving the cross-validation methodology from a static 24-hour window to a dynamic rolling test/train approach. There was a consensus on the need for a more robust error metric, suggesting a shift from CVRMSE to PNRMSE. Travis emphasized the need for commercial data to complete the test data sets.
 
Looking ahead, next steps include the continued exploration of advanced modeling techniques like neural networks and the use of larger datasets for a more thorough population analysis.

Next Meeting Scheduled: Tuesday December 5th, 2023.  
  
Watch the full presentation below.

The working group is pleased to announce that they have wrapped up the OpenEEmeter 4.0 Daily Model, and that the Alpha version is now available to the public and can be installed using: pip install eemeter==4.0.0a2.

The discussion then moved to progress on the hourly model. Armin began by reviewing some of the issues with the current hourly model, including that it seems to be overfit, and that it is incomplete when it comes to solar photovoltaic customers, where it produces high error rates. The question is how to make the model flexible and allow more inputs, including solar data.

Armin then discussed two broad options for addressing this issue: disaggregating solar data, and including weather and solar data to improve AMI prediction. Solar disaggregation would be more complex, which suggests that improving AMI prediction might be preferable for now.

The discussion then moved to the question of what approach to take when adding complexity to the model. There was discussion of the pros and cons of various machine learning approaches, including elastic nets and neural nets, with elastic nets as a simpler and more "interpretable" option.

There was also discussion in the group about the potential downside of neural nets and losing the interpretability of the model; however it was pointed out when you add a lot of coefficients, even in linear model, you lose physical interpretability.

The discussion ended with next steps, including implementing rolling train/test cross validation, using larger datasets for population analysis, doing a deep dive on the impact of GHI in model improvement, and developing neural net models for AMI prediction.

Next Meeting Scheduled: Tuesday November 7th, 2023.

Watch the full presentation below.

Thanks to everyone who joined the last OpenEEmeter working group.

Adam Scheer kicked off the meeting with the announcement that the name CalTRACK was being deprecated; going forward, both methods and code will be referred to as the OpenEEmeter under a single umbrella to emphasize the tool's global relevance. This decision was influenced by feedback from clients and users that the name CalTRACK gave an impression of regional specificity, leading to concerns about the model's applicability outside California.

In addition, while it is appropriate to continue to have a detailed description of methods, the OpenEEmeter code is now sophisticated enough that it should be considered the source of truth as to what is actually happening with model calculations. 

Adam then announced that the 2.1 daily model is at the point where it will soon be ready to be merged and available to everyone. In addition, the team will soon release comprehensive R&D results to support the decisions made in the final formulation of the model.

Next, Adam gave a high level overview of the 2.0 hourly methods to set the stage for discussing improvements in the performance of the 3.0 methods. Adam explained that 2.0 is founded on a Time of Week and Temperature (TOWT) model, which is primarily based on two variables: the hour of the week and the temperature. With 168 hours in a week, this results in a unique energy consumption prediction for each hour. To capture seasonal variations, the model is designed to create an independent prediction for each month of the year, considering the unique energy consumption characteristics of every month.

Issues with the 2.0 model include potential overfitting and that the model is incomplete when it comes to solar PV customers, whose consumption is heavily dependent on the amount of sunlight (solar irradiance). Without an awareness of solar irradiance the model will perform poorly when predicting consumption patterns of solar customers.

The goal of 3.0, is to reduce over- or under-fitting, introduce solar irradiance and other weather variables that may have an impact on consumption, and allow the model to take advantage of the patterns that it recognizes in the data.

The group then discussed in detail various aspects of these challenges and potential solutions. The meeting concluded with a discussion of laying the groundwork for next steps.

Next Meeting Scheduled: Tuesday, October 3rd, 1pm PT.

Watch the full presentation below.

Thanks to everyone who joined us for this week's OpenEEmeter working group.

Adam Scheer led off this week with a discussion of wrapping up the CalTRAck 2.1 daily model and beginning work updating the hourly model in preparation for CalTRACK 3.0. CalTRACK 2.1 makes significant improvements on accuracy and computational efficiency over CalTRACK 2.0 daily (speeds are up to 100 times faster). This is a huge improvement and will make the methods evne more valuable across a variety of use cases. Next steps for CalTRACK 2.1 include releasing detailed model specifications and updating the OpenEEmeter to incorporate CalTRACK 2.1.

Adam then explained how the CalTRACK 2.0 hourly model works, what some of its limitations are, and why we need more adaptable approaches to address modern demand-side programs including electric vehicles, heat pumps, solar + storage, and other load shifting approaches.

To begin work on the CalTRACK 3.0 hourly methods, the team has initiated a  literature review aimed at understanding the current state of the art and the latest technological advancements and modeling techniques that have emerged in the past few years. Armin Aligholian explained some of the approaches explored, including load simulation software, statistical learning, machine learning, and Baysean methods. 

Next steps:

• Focus on low bias model for AMI baseline prediction
• Models should be implementable at scale
• Explore and implement optimization and machine learning models
• Explore and implement probabilistic models
• Explore and implement solar weather data
• Combine solar disaggregation with AMI modeling
• Focus on individual level solar disaggregation
• Explore and implement solar PV physical model

Next Meeting Scheduled: Tuesday, September 5th, 1pm PT.

Watch the full presentation and download the slides below.

Thanks to everyone who attended the most recent working group meeting.

Adam Scheer led off the meeting with results of testing on how CalTRACK 2.1 running in "2.0 mode" gives almost identical model results to CalTRACK 2.0, while benefitting from the enormous improvements in computational efficiency of CalTRACK 2.1. This is important, because in many use cases (such as when only monthly data is available), users will be running the methods with the older model; they can now have confidence that results will almost identical. Adam then discussed the testing results for CalTRACK 2.1, noting that while not every problem has been solved, many big-ticket items have been addressed. He mentioned the model's improved reliability and more efficient handling of complex data. He pointed out that CalTRACK 2.1 has improved seasonal and weekend/weekday bias dramatically over CalTRACK 2.0.

• Wintertime bias across 4,000 residential gas meters went from -7% to -1%.  
• Summertime bias went from 11% to 5%  
• Weekday/weekend bias for electric meters went from -3% to 1%.  
• For commercial buildings, the weekend/weekday bias went from 14% to under 1%.

Adam and Travis then discussed the introduction of an adaptive loss function, a step beyond using mean squared error. Introducing this function yields similar results to the CalTRACK 2.1 model, but is slightly better for some uses cases and does not introduce significant bias. The discussion moved towards the final steps in the development of CalTRACK 2.1. Adam highlighted the importance of good software hygiene and maintaining updated versions. He also talked about the possibility of incorporating new features, such as more efficient data handling and storage capabilities provided by the new version of Panda's library.

The discussion on the future of the working group centered around what comes next, including the potential development of CalTRACK 3.0 and its associated features, such as improved daily modeling and potential incorporation of thermal lag and other factors into the model. The conversation also included a brief mention of possible code consolidation under the OEEM umbrella. Adam noted the increasing use of OpenEEmeter, indicating that it might be beneficial to bring all these tools and methods together for a more streamlined approach.

The team is aiming to get the updates into the OpenEEmeter to be available for others to test within the month.

Next Meeting Scheduled: Tuesday, July 11th, 1pm PT.

Watch the full presentation below.

Thanks to everyone who joined for this month's OpenEEmeter working group. We're excited at the huge progress we're making on CalTRACK 2.1, and greatly appreciate all who have taken time out of their schedule to participate in this important discussion.

Tim Guiterman from Sealed led off yesterday's discussion with a follow-up conversation on revising CalTRACK's requirements to permit its usage with delivered fuels such as propane and heating oil. Because these fuels are delivered and manually refilled, data for them are not as consistent nor as abundant as metered energy is. Tim and the Sealed team proposed making an exception to the 70 day data limit for delivered fuels.

The group discussed what the minimum number of data points should be and various factors that could alter this. There was also discussion of relaxing the 365 day baseline period to allow for more data when fitting this period. Most delivered fuel customers can provide data in the form of bills that cover years of time. Adam pointed out that while there might not be a perfect answer to the delivered fuel problem, using the CalTRACK approach is still much better than deemed approaches which are much less accurate.

The conversation concluded with Tim agreeing to analyze Sealed data so that the working group can make a data-driven decision on if rule changes should be made for delivered fuel customers and if so what should the minimum requirements be, with proof that backs up these revisions.

Adam and Travis then summarized CalTRACK 2.0's deficiencies and how the nearly final model of CalTRACK 2.1 rectifies them. The main problem the team was attempting to solve was seasonal bias in the CalTRACK 2.0 model found primarily in gas meters. They showed that CalTRACK 2.1 has significantly reduced seasonal bias. At the same time it has been necessary to revisit how fast the model can be fit in. Travis has implemented several clever means of speeding up the model from worst-case scenarios of 1 minute to fit down to 10 seconds. He also went into detail about how CalTRACK 2.1 differs from CalTRACK 2.0 in order to solve the issues laid out prior. Additionally, Travis briefly mentions that they have implemented a CalTRACK 2.0 mode, a legacy mode, which can replicate CalTRACK 2.0 results with all of the speed improvements developed recently.

Overall, they show that CalTRACK 2.1 reduces seasonal bias by 74% and is between 2 and 100 times faster than CalTRACK 2.0, depending on how it's used (legacy mode or not).

Adam and Travis anticipate that by the next meeting, the final CalTRACK 2.1 model will have completed its R&D phase. Next steps involve code cleanup, polishing, and documentation so that CalTRACK 2.1 can be integrated into the OpenEEmeter.
​
Next Meeting Scheduled: Tuesday, June 6th, 1pm PT.

Watch the full presentation below.

Thanks to everyone who joined the most recent OpenEEmeter working group meeting. 

Tim Guiterman from Sealed led off the conversation with a discussion of the need to address delivered fuels such as propane or heating oil in the CalTRACK specifications. CalTRACK's current data sufficiency requirements often limit the ability to model homes that using delivered fuels, which are common in the Northeast. Because these fuels are not necessarily delivered on a consistent cadence, CalTRACK's monthly data requirements are barrier to using it for these kinds of customers. Making CalTRACK more accessible and avoiding unnecessary disqualifications is particularly important given the new funding from the IRA and the opportunities it presents for adopting a measured pathway for incentives. 

Adam Sheer and Travis Sikes then shared updates and results related to the CalTRACK Daily 2.1 model, which is nearing completion. Adam reviewed the issues CalTRACK 2.1 process is attempting to address around model bias, and the tradeoffs between adding parameters to correct bias and the need to avoid overfitting. Preliminary results on the new 2.1 model showed a dramatic improvement, reducing summer, winter and weekend bias. 

Getting these kinds of improvements requires additional computations. CalTRACK 2.0 performance prohibits any additional complexity due to the time it takes to fit a model (20 - 60 seconds). This is particularly prohibitive to the cross-validation that is the lynchpin of our current improvement efforts.

To address this, Travis gave an extended presentation of how he has dramatically improved the computational efficiency of the OpenEEmeter, reducing the per-meter fitting time from 20 to 60 seconds to approximately 0.5 seconds (20-120 times faster) for an equivalent fit. For those that need or want to run the original CalTRACK 2.0, a CalTRACK 2.0 legacy mode will be included that has all of these computational efficiency improvements. 

The team discussed the remaining steps needed to finalize the model formulation and are hopeful that it will be wrapped up in the next one to two months. We are excited about the progress made so far and the potential impact of the updated model.

Watch the full meeting below.
​
Next Meeting Scheduled: Tuesday, May 2nd, 1pm PT

Thanks to everyone who joined us for yesterday's OpenEEMeter technical working group meeting.

Adam Scheer led off the meeting by reiterating the high-level goals for the CalTRACK 2.1 Daily model which are to improve specific and known areas of model bias and gain computational efficiency in the process. He explained the need to solve seasonal and weekend/weekday bias by adding parameters and optimization steps without creating an overfitting problem or making the model so complex that it can't be scaled to hundreds of thousands or millions of meters.

For the CalTRACK Hourly model, Adam laid out issues to discuss in future meetings, including remedying known small bugs, specifying demand response baselines, and incorporating solar PV modeling.

Adam then introduced James Fenna from Carbon Co-op to discuss some challenges and modifications needed to implement CalTRACK (specifically eeweather) in the U.K. and internationally. In particular, James discussed the problems of calibrating for metric temperatures and the need for different sources of weather data internationally.

Adam then led a more detailed discussion of the specific tradeoffs of adding parameters to address seasonal bias in the model.

The group discussed how seasonal bias is a problem that has been known for decades, and how much effort should be put into improving the Daily model versus Hourly.

Watch the full meeting below.

Next Meeting Scheduled: Tuesday, April 4th, 10am ET, 1pm PT

Join the working group to get access by clicking this link:

Thanks to all who attended the most recent CalTRACK working group meeting.

McGee Young opened this meeting with some words for Shuli Goodman of LFEnergy Foundation who recently passed away. McGee made the point that Shuli was instrumental in helping to give the OpenEEmeter credibility and a home. Shuli had a vision that open source/open standards could do for energy what it's done for other industries, like the Internet. Under her leadership LFEnergy has grown as an umbrella to many innovative projects. She will be missed.

The technical discussion explained the current goal of examining sources of bias or error in CalTRACK 2.0 and potential solutions. The group discussed the need to balance adding additional parameters against the risk of overfitting.

Watch the full meeting below.

Next Meeting Scheduled: Tuesday, March 7th, 10am ET, 1pm PT

Join the working group to get access by clicking this link:

Meeting topics will include

Join the working group to get access by clicking this link: