During the past week, we started discussions to establish an empirical method to determine a building’s suitability for CalTRACK methods. Listed below are two topics related to building qualifications that require testing:
- Preferred Metric: The two main metrics being discussed to evaluate a building’s suitability for CalTRACK methods are Coefficient of Variance Root-Mean Standard Error (CVRMSE) and Mean Absolute Percent Error (MAPE). The ideal metric will effectively determine a building’s suitability for CalTRACK methods, while also being convenient for other stakeholders in pay-for-performance programs.
- Model Selection: Due to the tendency of buildings that are unqualified for CalTRACK methods to be intercept-only models, it is proposed that the quality of intercept-only models is evaluated based on the variance of mean usage for daily or billing period data instead of the R-Squared criteria.
- How do our metrics and methods align with pay-for-performance programs? Do they reflect performance risk and uncertainty?
- What should we do with disqualified buildings? What is the best way to accommodate them in pay-for-performance programs?
Homework for Participants: