Forecasting
Looking Backward to Move Forward
July 12, 2017
For most things in life, I look forward to go forward. I keep my eyes on the road ahead. I look before I leap. I aim at my target. But this summer while we were on our family vacation, we rented a row boat and it reminded me that rowing a boat is different. When rowing, I face backward to maximize my power going forward. Forecasting electricity is like rowing a boat, we need to look backward to go forward.
Intuitively, we already do this by using regression models. Regression models look backward, find relationships and project them forward. Although too often, we leave the looking to models instead of our own eyes.
Below is annual historical average use for residential customers over an 11-year period. Leaving the forecast to a regression model with HDD, CDD, and some binary variables, the outlook is close to 11,600 kWh/year (blue dashed line) and flat because the model does not include any trend variables.
But, the last three years show dramatic decline in usage and raises the question of whether the forecast should be closer to 11,000 kWh/year (yellow-dashed line).
Instead of leaving the forecast to the model, we should use our own eyes to identify the underlying pattern in the history. Is average use declining?
The only way to answer this question is to weather normalize historic average use removing the impact of weather variation. Weather normalization will answer the question of why 2016 was so low, why 2014 was so high, and what happened in 2012.
After weather normalizing historic average use, we can answer what happened in 2012, 2014, and 2016.
Additionally, we can see that average usage has consistently declined from 2009 to 2016.
If we assume the trend stops in 2016, the forecast should be close to 11,400 kWh (purple solid line). If we assume the trend continues, the forecast will be close to 11,000 kWh/year in 2020 (purple dashed line).
Weather normalization is an essential part of the forecaster’s work process. The process allows us to look backwards to ensure that we move forward in the right direction.
Intuitively, we already do this by using regression models. Regression models look backward, find relationships and project them forward. Although too often, we leave the looking to models instead of our own eyes.
Below is annual historical average use for residential customers over an 11-year period. Leaving the forecast to a regression model with HDD, CDD, and some binary variables, the outlook is close to 11,600 kWh/year (blue dashed line) and flat because the model does not include any trend variables.
But, the last three years show dramatic decline in usage and raises the question of whether the forecast should be closer to 11,000 kWh/year (yellow-dashed line).
Instead of leaving the forecast to the model, we should use our own eyes to identify the underlying pattern in the history. Is average use declining?
The only way to answer this question is to weather normalize historic average use removing the impact of weather variation. Weather normalization will answer the question of why 2016 was so low, why 2014 was so high, and what happened in 2012.
After weather normalizing historic average use, we can answer what happened in 2012, 2014, and 2016.
- 2012 was a mild weather year
- 2014 was an extreme weather year
- 2016 was a mild weather year
Additionally, we can see that average usage has consistently declined from 2009 to 2016.
If we assume the trend stops in 2016, the forecast should be close to 11,400 kWh (purple solid line). If we assume the trend continues, the forecast will be close to 11,000 kWh/year in 2020 (purple dashed line).
Weather normalization is an essential part of the forecaster’s work process. The process allows us to look backwards to ensure that we move forward in the right direction.
Si è verificato un errore nell'elaborarazione del modello.
The following has evaluated to null or missing:
==> authorContent.contentFields [in template "44616#44647#114455" at line 9, column 17]
----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: contentFields = authorContent.content... [in template "44616#44647#114455" at line 9, column 1]
----
1<#assign
2 webContentData = jsonFactoryUtil.createJSONObject(author.getData())
3 classPK = webContentData.classPK
4/>
5
6<#assign
7authorContent = restClient.get("/headless-delivery/v1.0/structured-contents/" + classPK + "?fields=contentFields%2CfriendlyUrlPath%2CtaxonomyCategoryBriefs")
8contentFields = authorContent.contentFields
9categories=authorContent.taxonomyCategoryBriefs
10authorContentData = jsonFactoryUtil.createJSONObject(authorContent)
11friendlyURL = authorContentData.friendlyUrlPath
12authorCategoryId = "0"
13/>
14
15<#list contentFields as contentField >
16 <#assign
17 contentFieldData = jsonFactoryUtil.createJSONObject(contentField)
18 name = contentField.name
19 />
20 <#if name == 'authorImage'>
21 <#if (contentField.contentFieldValue.image)??>
22 <#assign authorImageURL = contentField.contentFieldValue.image.contentUrl />
23 </#if>
24 </#if>
25 <#if name == 'authorName'>
26 <#assign authorName = contentField.contentFieldValue.data />
27 <#list categories as category >
28 <#if authorName == category.taxonomyCategoryName>
29 <#assign authorCategoryId = category.taxonomyCategoryId />
30 </#if>
31 </#list>
32 </#if>
33 <#if name == 'authorDescription'>
34 <#assign authorDescription = contentField.contentFieldValue.data />
35
36 </#if>
37
38 <#if name == 'authorJobTitle'>
39 <#assign authorJobTitle = contentField.contentFieldValue.data />
40
41 </#if>
42
43</#list>
44
45<div class="blog-author-info">
46 <#if authorImageURL??>
47 <img class="blog-author-img" id="author-image" src="${authorImageURL}" alt="" />
48 </#if>
49 <#if authorName??>
50 <#if authorName != "">
51 <p class="blog-author-name">By <a id="author-detail-page" href="/w/${friendlyURL}?filter_category_552298=${authorCategoryId}"><span id="author-full-name">${authorName}</span></a></p>
52 <hr />
53 </#if>
54 </#if>
55 <#if authorJobTitle??>
56 <#if authorJobTitle != "">
57 <p class="blog-author-title" id="author-job-title" >${authorJobTitle}</p>
58 <hr />
59 </#if>
60 </#if>
61 <#if authorDescription??>
62 <#if authorDescription != "" && authorDescription != "null" >
63 <p class="blog-author-desc" id="author-job-desc">${authorDescription}</p>
64 <hr />
65 </#if>
66 </#if>
67</div>
The following has evaluated to null or missing:
==> authorContent.contentFields [in template "44616#44647#114455" at line 9, column 17]
----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: contentFields = authorContent.content... [in template "44616#44647#114455" at line 9, column 1]
----
1<#assign
2 webContentData = jsonFactoryUtil.createJSONObject(author.getData())
3 classPK = webContentData.classPK
4/>
5
6<#assign
7authorContent = restClient.get("/headless-delivery/v1.0/structured-contents/" + classPK + "?fields=contentFields%2CfriendlyUrlPath%2CtaxonomyCategoryBriefs")
8contentFields = authorContent.contentFields
9categories=authorContent.taxonomyCategoryBriefs
10authorContentData = jsonFactoryUtil.createJSONObject(authorContent)
11friendlyURL = authorContentData.friendlyUrlPath
12authorCategoryId = "0"
13/>
14
15<#list contentFields as contentField >
16 <#assign
17 contentFieldData = jsonFactoryUtil.createJSONObject(contentField)
18 name = contentField.name
19 />
20 <#if name == 'authorImage'>
21 <#if (contentField.contentFieldValue.image)??>
22 <#assign authorImageURL = contentField.contentFieldValue.image.contentUrl />
23 </#if>
24 </#if>
25 <#if name == 'authorName'>
26 <#assign authorName = contentField.contentFieldValue.data />
27 <#list categories as category >
28 <#if authorName == category.taxonomyCategoryName>
29 <#assign authorCategoryId = category.taxonomyCategoryId />
30 </#if>
31 </#list>
32 </#if>
33 <#if name == 'authorDescription'>
34 <#assign authorDescription = contentField.contentFieldValue.data />
35
36 </#if>
37
38 <#if name == 'authorJobTitle'>
39 <#assign authorJobTitle = contentField.contentFieldValue.data />
40
41 </#if>
42
43</#list>
44
45<div class="blog-author-info">
46 <#if authorImageURL??>
47 <img class="blog-author-img" id="author-image" src="${authorImageURL}" alt="" />
48 </#if>
49 <#if authorName??>
50 <#if authorName != "">
51 <p class="blog-author-name">By <a id="author-detail-page" href="/w/${friendlyURL}?filter_category_552298=${authorCategoryId}"><span id="author-full-name">${authorName}</span></a></p>
52 <hr />
53 </#if>
54 </#if>
55 <#if authorJobTitle??>
56 <#if authorJobTitle != "">
57 <p class="blog-author-title" id="author-job-title" >${authorJobTitle}</p>
58 <hr />
59 </#if>
60 </#if>
61 <#if authorDescription??>
62 <#if authorDescription != "" && authorDescription != "null" >
63 <p class="blog-author-desc" id="author-job-desc">${authorDescription}</p>
64 <hr />
65 </#if>
66 </#if>
67</div>