dlaj commited on
Commit
84b14d7
·
verified ·
1 Parent(s): 4539157

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -295
README.md CHANGED
@@ -1,296 +1,12 @@
1
- # Energy Forecasting with Transformer and LightGBM
2
-
3
- This project focuses on forecasting urban energy consumption based solely on historical usage and temperature data from Chicago (2011–2018). Two model architectures are compared: a LightGBM ensemble model and a Transformer-based neural network (based on the Moments Time Series Transformer). The goal is to predict hourly electricity demand and analyze model performance, interpretability, and generalizability.
4
-
5
- The project also simulates a real-time setting, where hourly predictions are made sequentially to mirror operational deployment. The modular design allows for adaptation to other urban contexts, assuming a compatible data structure.
6
-
7
  ---
8
-
9
- ## Overview
10
-
11
- * **Goal**: Predict hourly energy consumption using timestamp, temperature, and historical consumption features.
12
- * **Models**: LightGBM and Time Series Transformer Model (moements).
13
- * **Results**: Both models perform well; LightGBM achieves the best overall performance.
14
- * **Dashboard**: Live forecast simulation via Streamlit interface.
15
- * **Usage Context**: Developed as a prototype for real-time hourly forecasting, with a modular structure that supports adaptation to similar operational settings.
16
-
17
- ---
18
-
19
- ## Results
20
-
21
- ### Evaluation Metrics
22
-
23
- | Model | RMSE | R² | MAPE |
24
- | ----------- | ------- | ----- | ------ |
25
- | Transformer | 3933.57 | 0.972 | 2.32 % |
26
- | LightGBM | 1383.68 | 0.996 | 0.84 % |
27
-
28
- > **Note:** All values are in megawatts (MW). Hourly consumption typically ranges from 100,000 to 200,000 MW.
29
-
30
- * LightGBM achieves the best trade-off between performance and resource efficiency.
31
- * The Transformer model generalizes well to temporal patterns and may scale better in more complex or multi-network scenarios.
32
- * Both models show no signs of overfitting, supported by learning curves, consistent evaluation metrics, and additional diagnostics such as residual distribution analysis and noise-feature validation.
33
-
34
- ---
35
-
36
- ### Forecast Plots
37
-
38
- | LightGBM Prediction Plot | Transformer Prediction Plot |
39
- | :----------------------: | :--------------------------: |
40
- | ![LightGBM Prediction](assets/lightgbm_prediction_with_timestamp.png) | ![Transformer Prediction](assets/comparison_plot_1month.png) |
41
-
42
- > **Note:** Example forecast windows are shown (LightGBM: 3 months, Transformer: 1 month).
43
- > LightGBM maintains highly consistent performance over time, while the Transformer shows occasional over- or underestimation on special peak days.
44
-
45
- ---
46
-
47
- ### Learning Curves
48
-
49
- These plots visualize training dynamics and help detect overfitting.
50
-
51
- | LightGBM Learning Curve | Transformer Learning Curve |
52
- | :----------------------: | :------------------------: |
53
- | ![LightGBM LC](assets/lightgbm_learning_curve.png) | ![Transformer LC](assets/training_plot.png) |
54
-
55
- * The LightGBM curve shows a stable gap between training and validation RMSE, indicating low overfitting.
56
- * The Transformer learning curve also converges smoothly without divergence, supporting generalizability.
57
- * In addition to visual inspection, further checks like residual analysis and a noise feature test confirmed robustness.
58
-
59
- > **Note:** The LightGBM curve shows boosting rounds with validation RMSE,
60
- > while the Transformer plot tracks training loss and test metrics per epoch.
61
-
62
- More plots are available in the respective `/results` directories.
63
-
64
- ---
65
-
66
- ## Streamlit Simulation Dashboard
67
-
68
- * Live hourly forecast simulation
69
- * Uses the trained models
70
- * Repeats predictions sequentially for each hour to simulate real-time data flow
71
- * Hosted on Hugging Face (CPU only, slower prediction speed)
72
-
73
- You can try the model predictions interactively in the Streamlit dashboard:
74
-
75
- **Try it here:**
76
- **[Launch Streamlit App](https://huggingface.co/spaces/dlaj/energy-forecasting-demo)**
77
-
78
- **Preview:**
79
-
80
- ![Streamlit Dashboard Preview](assets/streamlit_preview.gif)
81
-
82
- ---
83
-
84
- ## Data
85
-
86
- * **Source**:
87
-
88
- * [COMED Hourly Consumption Data](https://www.kaggle.com/datasets/robikscube/hourly-energy-consumption)
89
- * [NOAA Temperature Data](https://www.ncei.noaa.gov/)
90
- * **Time range**: January 2011 – August 2018
91
- * **Merged file**: `data/processed/energy_consumption_aggregated_cleaned.csv`
92
-
93
- ---
94
-
95
- ## Feature Engineering
96
-
97
- The models rely on timestamp and temperature data, enriched with derived time-based and lag-based features:
98
-
99
- * hour\_sin, hour\_cos
100
- * weekday\_sin, weekday\_cos
101
- * month\_sin, month\_cos
102
- * rolling\_mean\_6h
103
- * temperature\_c
104
- * consumption\_last\_hour
105
- * consumption\_yesterday
106
- * consumption\_last\_week
107
-
108
- Feature selection was guided by LightGBM feature importance analysis. Weak features with nearly no impact like "is_weekend" were deleted.
109
-
110
- ### Final LightGBM Feature Importance
111
-
112
- <img src="assets/lightgbm_feature_importance.png" alt="Feature Importance" style="width: 80%;"/>
113
-
114
- ---
115
-
116
- ## Model Development
117
-
118
- ### LightGBM
119
-
120
- * Custom grid search with over 50 parameter combinations
121
- * Parameters tested:
122
-
123
- * num\_leaves, max\_depth, learning\_rate, lambda\_l1, lambda\_l2, min\_split\_gain
124
- * Final Parameters:
125
-
126
- * learning\_rate: 0.05
127
- * num\_leaves: 15
128
- * max\_depth: 5
129
- * lambda\_l1: 1.0
130
- * lambda\_l2: 0.0
131
- * min\_split\_gain: 0.0
132
- * n\_estimators: 1000
133
- * objective: regression
134
-
135
- Overfitting was monitored using a noise feature and RMSE gaps. See grid search results:
136
- `notebooks/lightgbm/lightgbm_gridsearch_results.csv`
137
-
138
- ### Transformer (Moments Time Series Transformer)
139
-
140
- * Based on pretrained Moments model
141
- * Fine-tuned only the forecasting head for regular training
142
- * Also tested variants with unfrozen encoder layers and dropout
143
- * Final config:
144
-
145
- * task\_name: forecasting
146
- * forecast\_horizon: 24
147
- * head\_dropout: 0.1
148
- * weight\_decay: 0
149
- * freeze\_encoder: True
150
- * freeze\_embedder: True
151
- * freeze\_head: False
152
-
153
- ---
154
-
155
- ## Project Structure
156
-
157
- ```
158
- energy-forecasting-transformer-lightgbm/
159
- ├── data/ # Raw, external, processed datasets
160
- ├── notebooks/ # EDA, lightgbm and transformer prototypes, including hyperparameter tuning and model selection
161
- ├── scripts/ # Data preprocessing scripts
162
- ├── lightgbm_model/ # LightGBM model, scripts, results
163
- ├── transformer_model/ # Transformer model, scripts, results
164
- ├── streamlit_simulation/ # Streamlit dashboard
165
- ├── requirements.txt # Main environment
166
- ├── requirements_lgbm.txt # Optional for LightGBM
167
- ├── setup.py
168
- └── README.md
169
- ```
170
-
171
- ---
172
-
173
- ## Reproducibility
174
-
175
- You can reuse this pipeline with any dataset, as long as it contains the following key columns:
176
-
177
- ```csv
178
- timestamp, # hourly timestamp (e.g., "2018-01-01 14:00")
179
- consumption, # energy usage (aggregated; for individual users, consider adding an ID column)
180
- temperature # hourly
181
- ```
182
-
183
- ### Notes:
184
-
185
- * Transformer model training is **very slow on CPU**, also with AMD GPU
186
- * Recommended: use **CUDA or Google Colab + CUDA GPU runtime** for transformer training
187
- * All scripts are modular and can be executed separately
188
-
189
- ---
190
-
191
- ## CI/CD & DevOps Setup
192
-
193
- This project includes a lightweight CI pipeline using GitHub Actions:
194
-
195
- * **CI**:
196
- - Runs `pytest` on every push
197
- - Builds and validates the Docker image
198
-
199
- * **Code quality checks**:
200
- - Uses `pre-commit` hooks with `black`, `isort`, and `ruff`
201
- - Ensures consistent formatting and linting before commits
202
-
203
- To enable pre-commit locally:
204
-
205
- ```bash
206
- pre-commit install
207
- ```
208
-
209
- ---
210
-
211
- ## Run Locally
212
-
213
- ### Prerequisites
214
-
215
- * Python 3.9–3.11 (required for Moments Transformer)
216
-
217
- ### Installation
218
-
219
- ```bash
220
- git clone https://github.com/dlajic/energy-forecasting-transformer-lightgbm.git
221
- cd energy-forecasting-transformer-lightgbm
222
- pip install -r requirements.txt
223
- ```
224
-
225
- ### Preprocess Data
226
-
227
- ```bash
228
- python -m scripts.data_preprocessing.merge_temperature_data # merges raw temperature and energy data (only needed with raw inputs)
229
- python -m scripts.data_preprocessing.preprocess_data # launches full preprocessing pipeline; use if data already matches expected format
230
- ```
231
-
232
- ### Train Models
233
-
234
- ```bash
235
- python -m lightgbm_model.scripts.train.train_lightgbm
236
- python -m transformer_model.scripts.training.train
237
- ```
238
-
239
- ### Evaluate Models
240
-
241
- ```bash
242
- python -m lightgbm_model.scripts.eval.eval_lightgbm
243
- python -m transformer_model.scripts.evaluation.evaluate
244
- python -m transformer_model.scripts.evaluation.plot_learning_curves
245
- ```
246
-
247
- ### Run Streamlit Dashboard (local)
248
-
249
- ```bash
250
- streamlit run streamlit_simulation/app.py
251
- ```
252
-
253
- For editable install:
254
-
255
- ```bash
256
- pip install -e .
257
- ```
258
-
259
- ## Run App with Docker
260
-
261
- This project also supports containerized execution using Docker:
262
-
263
-
264
- ```bash
265
- # Start app with Docker Compose (Linux)
266
- ./start.sh
267
-
268
- # Or on Windows (PowerShell)
269
- ./start.ps1
270
- ```
271
-
272
- Make sure Docker (Docker-Desktop) is running before executing the script.
273
-
274
- This will:
275
-
276
- 1. Build the Docker image
277
- 2. Start the Streamlit app on localhost:8501
278
- 3. Open it automatically in your browser
279
-
280
- ---
281
-
282
- ## Author
283
-
284
- Dean Lajic
285
- GitHub: [dlajic](https://github.com/dlajic)
286
-
287
- ---
288
-
289
- ## References
290
-
291
- - Moments Time Series Transformer
292
- https://github.com/moment-timeseries-foundation-model/moment
293
- - COMED Consumption Dataset
294
- https://www.kaggle.com/datasets/robikscube/hourly-energy-consumption
295
- - NOAA Weather Data
296
- https://www.ncei.noaa.gov
 
 
 
 
 
 
 
1
  ---
2
+ title: Energy Forecasting Demo
3
+ emoji:
4
+ colorFrom: blue
5
+ colorTo: green
6
+ sdk: streamlit
7
+ sdk_version: 1.30.0
8
+ app_file: streamlit_simulation/app.py
9
+ pinned: true
10
+ license: apache-2.0
11
+ short_description: Hourly energy consumption forecasting
12
+ ---