Out earlier this year! A new paper with my colleagues at Carnegie Institution for Science that provides algorithms to clean U.S. Energy Information Administration hourly electricity demand data. We make available a documented method for data screening and imputation that we hope will save energy researchers time.
As compilers of this demand data, EIA does a great job of consistently making it widely available. And while EIA cleans some data, missing and suspicious data remain in the hourly time-series for many balancing authorities.
Our paper – authored by Tyler Ruggles, me, Dan Tong (UC Irvine), and Ken Caldeira – in Scientific Data uses the raw EIA demand data to develop and implement screening and imputation algorithms to produce continuous hourly demand data for the balancing authorities of the US.
Please check out the paper Developing reliable hourly electricity demand data through screening and imputation for the details. Hopefully others will find this cleaned data useful!
You can find the cleaned data here And you can find the code used to clean and analyze the data here
Please email me if you have issues accessing the paper.