A (hopefully eventually) complete listing of the most popular electric LF datasets
We found it difficult to find suitable datasets in the flood of information. So we came up with the idea of doing a proper search and making the results available to the public.
Based on a sample set of representative publications, relevant, publicly accessible data sets were extracted, structured and analyzed. The details of the search can be found in the scientific publication: https://doi.org/10.15488/17659
We are happy about any kind of cooperation, feedback or extension to make the list even more valuable for other scientists. So feel free to expand the list and initiate a pull request.
| ID | Abbrev | Name | Domain1 | Resolution2 | Features3 | Duration4 | Spanned years | Horizons5 | Regions6 | Type7 | Links | Access8 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ISO-NE | New England Independent System Operator | S | 60 | E | 108 | 2003-2014 | ❌✔️✔️❌ | ✔️ | 📦 | 🔗Link | 🔓 |
| 2 | NYISO | New York Independent System Operator | S | 5 | E | 264 | 2001-2023 | ✔️✔️✔️❌ | ✔️ | 📦 | 🔗Link | 🔓 |
| 3 | PJM | PJM Hourly Energy Consumption | S | 60 | E | 240 | 1998-2018 | ❌✔️✔️✔️ | ✔️ | 📦 | 🔗Link | 🔓 |
| 4 | CIF | CIF 2016 competition dataset | ? | d,m,y | Undef. | 8-909 | unknown | ❌❌✔️✔️ | ❌ | 📦 | 🔗Link | 🔓 |
| 5 | GEFCOM14 | GEFCom 2014 | S | 60 | E, W, T, PV | 10 | 2021 | ❌✔️❌❌ | ❌ | 📦 | 🔗Link | 🔓 |
| 6 | EUNITE | EUNITE 2001 | S | 30 | E, T, H | 24 | 1997-1999 | ❌✔️✔️❌ | ❌ | 📦 | 🔗Link | 🔓 |
| 7 | ENTSO-E | ENTSO-E electric load dataset | S | 60 | E | <=288 | till 2015 | ❌✔️✔️✔️ | ✔️ | 📦 | 🔗Link | 🔓 |
| 299 | EWELD | Large-Scale Industrial and Commercial Load Dataset in Extreme Weather Events | I | 15 | E, W, xW | <=74 | 2016-2022 | ✔️✔️✔️✔️ | ✔️ (386) | 📦 | 🔗Link | 🔓 |
| 289 | WPuQ | Electrical single-family house and heat pump load | R | <1 | E | 30 | 2018-2020 | ❌✔️✔️❌ | ✔️ (38) | 📦 | 🔗Link | 🔓 |
| 329 | PanETESA | Panama ETESA | S | 60 | E, W, H | 66 | 2015-2020 | ❌✔️✔️✔️ | ❌ | 📦 | 🔗Link | 🔓 |
| 389 | REFIT | REFIT: Electrical Load Measurements | H | 8sec | E | 20 | 2013-2015 | ✔️✔️✔️❌ | ✔️(20) | 📦 | 🔗Link1 🔗Link2 | 🔓 |
| 399 | ECD-UY | household electricity consumption dataset of Uruguay | S, R | 1-15 | E | 11-23 | 2019-2020 | ✔️✔️❌❌ | ✔️(9) | 📦 | 🔗Link1 🔗Link2 | 🔓 |
| 409 | IDEAL | IDEAL UK Household Energy Dataset 255 | R | 1-12sec | E, W, T | 23 | 2019-2020 | ✔️✔️✔️❌ | ✔️(255) | 📦 | 🔗Link1 🔗Link2 | 🔓 |
| 419 | HANOI-Res | Residential Apartments Dataset Hanoi, Vietnam (CAMaRSEC Project) | R | 15 | E, W, T | 12 | 2020-2021 | ✔️✔️❌❌ | ✔️(49) | 📦 | 🔗Link1 🔗Link2 | 🔓 |
| 429 | UK-DALE | UK Domestic Appliance Level Electricity (UKERC EDC), Disaggregated (6s) and aggregated (1s) | R | 1-6sec | E | 5-53 | 2012-2017 | ✔️✔️✔️✔️ | ✔️(5) | 📦 | 🔗Link1 🔗Link2 🔗Link3 | 🔓 |
| 8 | LCL | LCL Load Dataset (London Households) | H | 30 | E | 12 | 2013 | ❌✔️❌❌ | ❌ | 📁 | 🔗Link | 🔓 |
| 9 | SET | Energy Consumption Dataset for Milano/Trento | S | 10 | E | <1 | 2013 | ✔️❌❌❌ | ❌ | 📁 | 🔗Link | 🔓 |
| 10 | BDG-Proj | Building Data Genome Project | S | 60 | E | 12 | unknown | ❌✔️❌❌ | ✔️ | 📁 | 🔗Link | 🔓 |
| 349 | BDG-Proj2 | Building Data Genome Project 2 (BDG2) | R | 60 | E | 24 | 2016-2017 | ❌✔️✔️❌ | ✔️ (1636) | 📁 | 🔗Link | 🔓 |
| 11 | IHPC | Individual Household power consumption | S | 1 | E | 48 | 2006-2010 | ✔️✔️✔️✔️ | ❌ | 📁 | 🔗Link | 🔓 |
| 12 | GEFCOM12 | GEFCom 2012 | S | 60 | E, W, T | 42 | 2004-2008 | ❌✔️✔️❌ | ❌ | 📁 | 🔗Link | 🔓 |
| 13 | OPSD-TS | Open Power System Data TS | S | 15-60 | E, PV, W | 148 | 2005-2019 | ✔️✔️✔️✔️ | ✔️ | 📁 | 🔗Link | 🔓 |
| 279 | OPSD-HH | Open Power System Data Household Data | R, I | 1-60 | E, PV | diff | 2012-2019 | ✔️✔️✔️✔️ | ✔️ | 📁 | 🔗Link | 🔓 |
| 14 | ELD | ElectricityLoadDiagrams20112014 | S | 15 | E | 36 | 2011-2014 | ✔️✔️✔️✔️ | ❌ | 📁 | 🔗Link1 🔗Link2 | 🔓 |
| 15 | ENERTALK | ENERTALK Dataset Korea (household) | S | 15 hz | E | 12 | 2016 | ✔️✔️❌❌ | ❌ | 📁 | 🔗Link | 🔓 |
| 16 | S-TSO | Spanish Transmission Service operator (TSO) | H | 60 | >25 | 24 | 2017-2018 | ❌✔️✔️❌ | ❌ | 📁 | 🔗Link | 🔓 |
| 269 | CER | CER Smart Metering Project | R,I | 30 | E | 18 | 2009-2010 | ❌✔️✔️❌ | ✔️(5237) | 📁 | 🔗Link | 📧 |
| 309 | DEDDIAG | domestic electricity demand dataset (individual appliances in Germany) | R | 1Hz | E | 2-44 | 2011-2014 | ✔️✔️✔️❌ | ✔️(14) | 📁 | 🔗Link1 🔗Link2 | 🔓 |
| 319 | AusSmartGrid | Electricity Use Interval Reading | R | 60 | E | ? | 2010-2014 | ❌✔️✔️❌ | ✔️ | 📁 | 🔗Link | 🔓 |
| 359 | UK-GRID | Electricity consumption UK 2009-2024 | S | 30 | E | 180 | 2009-2024 | ❌✔️✔️✔️ | ❌ | 📁 | 🔗Link | 🔓 |
| 369 | HoustonRes | Houston Residential power usage (one house) | R | 60 | E, W | 49 | 2016-2020 | ❌✔️✔️❌ | ❌ | 📁 | 🔗Link | 🔓 |
| 379 | CU-BEMS-Bangkok | Bangkok CU-BEMS, smart building energy and IAQ data | R | 1 | E, W | 18 | 2018-2019 | ✔️✔️✔️❌ | ❌ | 📁 | 🔗Link | 🔓 |
| 17 | RTE-France | RTE France | S | 30 | E | 12 | 2012-2020 | ❌✔️❌❌ | ✔️ | 🌐 | 🔗Link | 🔓 |
| 18 | AEMO | Australian Energy market operator | H | 60 | E | 12 | 2013 | ❌✔️❌❌ | ✔️ | 🌐 | 🔗Link | 🔓 |
| 19 | IESO-O | IESO Ontario | H | 60 | E, P | 20+ | 2022-2023 | ❌✔️✔️❌ | ❌ | 🌐 | 🔗Link | 🔓 |
| 20 | AESO | Alberta Electric Sys. Op. Electrical Load Dataset | S | 60 | E | 132 | 2005-2016 | ❌✔️✔️✔️ | ❌ | 🌐 | 🔗Link | 🔓 |
| 21 | PPS | Polish power system | S | 15-60 | E | 120+ | 2013- now | ✔️✔️✔️✔️ | ❌ | 🌐 | 🔗Link | 🔓 |
| 22 | AUSGRID | Ausgrid: Distribution zone substation | S | 15 | E | 204 | 2005-2022 | ✔️✔️✔️✔️ | ✔️(>100) | 🌐 | 🔗Link | 🔓 |
| 23 | KPX | KPX Korea | H | 5 | E | 240 | 2003-now | ✔️✔️✔️✔️ | ❌ | 🌐 | 🔗Link | 🔓 |
| 24 | ADMIE | Independent Electricity Transmission Operator | S | 60 | E | 120+ | 2011-now | ❌✔️✔️✔️ | ✔️ | 🌐 | 🔗Link | 🔓 |
| 25 | Pecan | Pecan Street dataset | S | 15 | E, W | 24 | 2017-2018 | ✔️✔️✔️❌ | ✔️ | 🌐 | 🔗Link | 🔓 |
| 339 | Cal-ISO | California ISO Hourly Load Data | S | 60 | E | 100+ | 2014-now | ❌✔️✔️✔️ | ✔️ | 🌐 | 🔗Link1 🔗Link2 | 🔓 |
1Domain: Either system level load (S), residential load (R) or Industry (I)
2Resolution: In minutes, if not other stated (d=day, m=month, y=year, hz=1sec)
3Features: Electricity (E), Weather (W), Extreme Weather Events, e.g. heat periods and taifune (xW), Temperature (T), Photovoltaic production (PV), Holiday features (H), Price (P)
4Duration: in number of months
5Forecasting-Horizons for modeling applicable: Very Short Term (VST), Short Term (ST), Medium Long Term (MT), Long Term (LT)
6Dataset records multiple regions / consumers separately (e.g. buildings, cities, countries) or disaggregated single loads available. Numbers in brackets indicate the number of regions / consumers / loads
7Type: Either 📦 = a collection (accumulation of datasets), 📁=a file or achive or 🌐=a data platform / API
8Access: Either 🔓 = can be accessed directly (no login, no request), 📧 = written application / request has to be sent first
9 not part of the original Paper, added later (only here)
for further details take a look at the publication below
If this work has helped you with your scientific work, we would appreciate a proper mention. ❤️
Our citation recommendation is:
Baur, L.; Chandramouli, V.; Sauer, A.: Publicly Available Datasets For Electric Load Forecasting – An Overview. In: Herberger, D.; Hübner, M. (Eds.): Proceedings of the CPSL 2024. Hannover : publish-Ing., 2024, S. 1-12. DOI: https://doi.org/10.15488/17659
BibTeX entry
@inproceedings{baur2024datasets,
author = {Baur, Lukas and Chandramouli, Vignesh and Sauer, Alexander},
title = {Publicly Available Datasets For Electric Load Forecasting – An Overview},
booktitle = {Proceedings of the CPSL 2024},
editor = {Herberger, D. and Hübner, M.},
location = {Hannover},
publisher = {publish-Ing.},
year = {2024},
pages = {1--12},
doi = {10.15488/17659}
}
💰 We'd like to thank the German Federal Ministry of Economic Affairs and Climate Action (BMWK) and the project supervision of the Project Management Jülich (PtJ) for the project „FlexGUIde“ which allowed for the work.
💡 We would also like to thank an anonymous reviewer who suggested publishing the datasets not only in the above-mentioned publication but also as a repository.
👨🎓 We would like to thank K. Kunkel, whose master's thesis contributed greatly to the expansion of the initial dataset collection.