Apache: Big Data 2016

Automated Metering Infrastructure (AMI) is gaining traction within the utilities sector and has brought with it numerous improvements in all related fields. Specifically in tariff setting and demand response models, classification of smart meter readings into load profiles helps find the right segments to target. The methodology explained in this tutorial combines commercial, government and open data with the internal company data to accurately predict the load profile of a new customer using high performing classification models in both R and PySpark. Load profiles are generated using a clustering algorithm and are subsequently used as the dependent variable in our classification model. The results of this model are then scored and interpreted in a business context. During the entire process, possible business hurdles will be identified and solutions will be offered.