Transcendental-Programmer
commited on
Commit
ยท
b407fad
1
Parent(s):
45309a1
fix : fixed the client simulator
Browse files- README.md +19 -19
- app.py +68 -68
- webapp/streamlit_app.py +68 -68
README.md
CHANGED
@@ -17,13 +17,13 @@ license: mit
|
|
17 |
|
18 |
# Federated Learning for Privacy-Preserving Financial Data Generation with RAG Integration
|
19 |
|
20 |
-
This project implements a
|
21 |
|
22 |
-
##
|
23 |
|
24 |
**Try it now**: [Hugging Face Spaces](https://huggingface.co/spaces/ArchCoder/federated-credit-scoring)
|
25 |
|
26 |
-
##
|
27 |
|
28 |
- **Complete Federated Learning System**: Working server, clients, and web interface
|
29 |
- **Real-time Predictions**: Get credit score predictions from the federated model
|
@@ -33,9 +33,9 @@ This project implements a **complete federated learning framework** with a Retri
|
|
33 |
- **Educational**: Learn about federated learning concepts
|
34 |
- **Production Ready**: Docker and Kubernetes deployment support
|
35 |
|
36 |
-
##
|
37 |
|
38 |
-
### Option 1: Try the Demo
|
39 |
1. Visit the [Live Demo](https://huggingface.co/spaces/ArchCoder/federated-credit-scoring)
|
40 |
2. Enter customer features and get predictions
|
41 |
3. Learn about federated learning
|
@@ -72,7 +72,7 @@ streamlit run webapp/streamlit_app.py
|
|
72 |
python test_complete_system.py
|
73 |
```
|
74 |
|
75 |
-
##
|
76 |
|
77 |
### Web Application Features:
|
78 |
- **Demo Mode**: Works without server (perfect for HF Spaces)
|
@@ -91,7 +91,7 @@ python test_complete_system.py
|
|
91 |
6. **Global Model**: Updated model is distributed to all clients
|
92 |
7. **Prediction**: Users can get predictions from the global model
|
93 |
|
94 |
-
##
|
95 |
|
96 |
```
|
97 |
โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโ
|
@@ -107,7 +107,7 @@ python test_complete_system.py
|
|
107 |
โโโโโโโโโโโโโโโโโโโ
|
108 |
```
|
109 |
|
110 |
-
##
|
111 |
|
112 |
```
|
113 |
FinFedRAG-Financial-Federated-RAG/
|
@@ -129,7 +129,7 @@ FinFedRAG-Financial-Federated-RAG/
|
|
129 |
โโโ test_complete_system.py # End-to-end system test
|
130 |
```
|
131 |
|
132 |
-
##
|
133 |
|
134 |
### Server Configuration (`config/server_config.yaml`)
|
135 |
```yaml
|
@@ -159,7 +159,7 @@ client:
|
|
159 |
input_dim: 32
|
160 |
```
|
161 |
|
162 |
-
##
|
163 |
|
164 |
Run the complete system test:
|
165 |
```bash
|
@@ -167,12 +167,12 @@ python test_complete_system.py
|
|
167 |
```
|
168 |
|
169 |
This will test:
|
170 |
-
-
|
171 |
-
-
|
172 |
-
-
|
173 |
-
-
|
174 |
|
175 |
-
##
|
176 |
|
177 |
### Hugging Face Spaces (Recommended)
|
178 |
1. Fork this repository
|
@@ -196,14 +196,14 @@ streamlit run webapp/streamlit_app.py
|
|
196 |
docker-compose up
|
197 |
```
|
198 |
|
199 |
-
##
|
200 |
|
201 |
- **Model Accuracy**: 85%+ across federated rounds
|
202 |
- **Response Time**: <1 second for predictions
|
203 |
- **Scalability**: Supports 10+ concurrent clients
|
204 |
- **Privacy**: Zero raw data sharing
|
205 |
|
206 |
-
##
|
207 |
|
208 |
1. Fork the repository
|
209 |
2. Create a feature branch
|
@@ -211,11 +211,11 @@ docker-compose up
|
|
211 |
4. Add tests
|
212 |
5. Submit a pull request
|
213 |
|
214 |
-
##
|
215 |
|
216 |
MIT License - see LICENSE file for details.
|
217 |
|
218 |
-
##
|
219 |
|
220 |
- TensorFlow for the ML framework
|
221 |
- Streamlit for the web interface
|
|
|
17 |
|
18 |
# Federated Learning for Privacy-Preserving Financial Data Generation with RAG Integration
|
19 |
|
20 |
+
This project implements a complete federated learning framework with a Retrieval-Augmented Generation (RAG) system for privacy-preserving synthetic financial data generation. The system includes a working server, multiple clients, and an interactive web application.
|
21 |
|
22 |
+
## Live Demo
|
23 |
|
24 |
**Try it now**: [Hugging Face Spaces](https://huggingface.co/spaces/ArchCoder/federated-credit-scoring)
|
25 |
|
26 |
+
## Features
|
27 |
|
28 |
- **Complete Federated Learning System**: Working server, clients, and web interface
|
29 |
- **Real-time Predictions**: Get credit score predictions from the federated model
|
|
|
33 |
- **Educational**: Learn about federated learning concepts
|
34 |
- **Production Ready**: Docker and Kubernetes deployment support
|
35 |
|
36 |
+
## Quick Start
|
37 |
|
38 |
+
### Option 1: Try the Demo
|
39 |
1. Visit the [Live Demo](https://huggingface.co/spaces/ArchCoder/federated-credit-scoring)
|
40 |
2. Enter customer features and get predictions
|
41 |
3. Learn about federated learning
|
|
|
72 |
python test_complete_system.py
|
73 |
```
|
74 |
|
75 |
+
## How to Use
|
76 |
|
77 |
### Web Application Features:
|
78 |
- **Demo Mode**: Works without server (perfect for HF Spaces)
|
|
|
91 |
6. **Global Model**: Updated model is distributed to all clients
|
92 |
7. **Prediction**: Users can get predictions from the global model
|
93 |
|
94 |
+
## System Architecture
|
95 |
|
96 |
```
|
97 |
โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโ
|
|
|
107 |
โโโโโโโโโโโโโโโโโโโ
|
108 |
```
|
109 |
|
110 |
+
## Project Structure
|
111 |
|
112 |
```
|
113 |
FinFedRAG-Financial-Federated-RAG/
|
|
|
129 |
โโโ test_complete_system.py # End-to-end system test
|
130 |
```
|
131 |
|
132 |
+
## Configuration
|
133 |
|
134 |
### Server Configuration (`config/server_config.yaml`)
|
135 |
```yaml
|
|
|
159 |
input_dim: 32
|
160 |
```
|
161 |
|
162 |
+
## Testing
|
163 |
|
164 |
Run the complete system test:
|
165 |
```bash
|
|
|
167 |
```
|
168 |
|
169 |
This will test:
|
170 |
+
- Server health
|
171 |
+
- Client registration
|
172 |
+
- Training status
|
173 |
+
- Prediction functionality
|
174 |
|
175 |
+
## Deployment
|
176 |
|
177 |
### Hugging Face Spaces (Recommended)
|
178 |
1. Fork this repository
|
|
|
196 |
docker-compose up
|
197 |
```
|
198 |
|
199 |
+
## Performance
|
200 |
|
201 |
- **Model Accuracy**: 85%+ across federated rounds
|
202 |
- **Response Time**: <1 second for predictions
|
203 |
- **Scalability**: Supports 10+ concurrent clients
|
204 |
- **Privacy**: Zero raw data sharing
|
205 |
|
206 |
+
## Contributing
|
207 |
|
208 |
1. Fork the repository
|
209 |
2. Create a feature branch
|
|
|
211 |
4. Add tests
|
212 |
5. Submit a pull request
|
213 |
|
214 |
+
## License
|
215 |
|
216 |
MIT License - see LICENSE file for details.
|
217 |
|
218 |
+
## Acknowledgments
|
219 |
|
220 |
- TensorFlow for the ML framework
|
221 |
- Streamlit for the web interface
|
app.py
CHANGED
@@ -6,6 +6,73 @@ import threading
|
|
6 |
import json
|
7 |
from datetime import datetime
|
8 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
9 |
st.set_page_config(page_title="Federated Credit Scoring Demo", layout="centered")
|
10 |
st.title("Federated Credit Scoring Demo (Federated Learning)")
|
11 |
|
@@ -172,71 +239,4 @@ if st.session_state.client_simulator and not DEMO_MODE:
|
|
172 |
st.markdown("---")
|
173 |
st.markdown("""
|
174 |
*This is a demonstration of federated learning concepts. For full functionality, run the federated server and clients locally.*
|
175 |
-
""")
|
176 |
-
|
177 |
-
# Client Simulator Class
|
178 |
-
class ClientSimulator:
|
179 |
-
def __init__(self, server_url):
|
180 |
-
self.server_url = server_url
|
181 |
-
self.client_id = f"web_client_{int(time.time())}"
|
182 |
-
self.is_running = False
|
183 |
-
self.thread = None
|
184 |
-
self.last_update = "Never"
|
185 |
-
|
186 |
-
def start(self):
|
187 |
-
self.is_running = True
|
188 |
-
self.thread = threading.Thread(target=self._run_client, daemon=True)
|
189 |
-
self.thread.start()
|
190 |
-
|
191 |
-
def stop(self):
|
192 |
-
self.is_running = False
|
193 |
-
|
194 |
-
def _run_client(self):
|
195 |
-
try:
|
196 |
-
# Register with server
|
197 |
-
client_info = {
|
198 |
-
'dataset_size': 100,
|
199 |
-
'model_params': 10000,
|
200 |
-
'capabilities': ['training', 'inference']
|
201 |
-
}
|
202 |
-
|
203 |
-
resp = requests.post(f"{self.server_url}/register",
|
204 |
-
json={'client_id': self.client_id, 'client_info': client_info})
|
205 |
-
|
206 |
-
if resp.status_code == 200:
|
207 |
-
st.session_state.training_history.append({
|
208 |
-
'round': 0,
|
209 |
-
'active_clients': 1,
|
210 |
-
'clients_ready': 0,
|
211 |
-
'timestamp': datetime.now()
|
212 |
-
})
|
213 |
-
|
214 |
-
# Simulate client participation
|
215 |
-
while self.is_running:
|
216 |
-
try:
|
217 |
-
# Get training status
|
218 |
-
status = requests.get(f"{self.server_url}/training_status")
|
219 |
-
if status.status_code == 200:
|
220 |
-
data = status.json()
|
221 |
-
|
222 |
-
# Update training history
|
223 |
-
st.session_state.training_history.append({
|
224 |
-
'round': data.get('current_round', 0),
|
225 |
-
'active_clients': data.get('active_clients', 0),
|
226 |
-
'clients_ready': data.get('clients_ready', 0),
|
227 |
-
'timestamp': datetime.now()
|
228 |
-
})
|
229 |
-
|
230 |
-
# Keep only last 50 entries
|
231 |
-
if len(st.session_state.training_history) > 50:
|
232 |
-
st.session_state.training_history = st.session_state.training_history[-50:]
|
233 |
-
|
234 |
-
time.sleep(5) # Check every 5 seconds
|
235 |
-
|
236 |
-
except Exception as e:
|
237 |
-
print(f"Client simulator error: {e}")
|
238 |
-
time.sleep(10)
|
239 |
-
|
240 |
-
except Exception as e:
|
241 |
-
print(f"Failed to start client simulator: {e}")
|
242 |
-
self.is_running = False
|
|
|
6 |
import json
|
7 |
from datetime import datetime
|
8 |
|
9 |
+
# Client Simulator Class (moved to top)
|
10 |
+
class ClientSimulator:
|
11 |
+
def __init__(self, server_url):
|
12 |
+
self.server_url = server_url
|
13 |
+
self.client_id = f"web_client_{int(time.time())}"
|
14 |
+
self.is_running = False
|
15 |
+
self.thread = None
|
16 |
+
self.last_update = "Never"
|
17 |
+
|
18 |
+
def start(self):
|
19 |
+
self.is_running = True
|
20 |
+
self.thread = threading.Thread(target=self._run_client, daemon=True)
|
21 |
+
self.thread.start()
|
22 |
+
|
23 |
+
def stop(self):
|
24 |
+
self.is_running = False
|
25 |
+
|
26 |
+
def _run_client(self):
|
27 |
+
try:
|
28 |
+
# Register with server
|
29 |
+
client_info = {
|
30 |
+
'dataset_size': 100,
|
31 |
+
'model_params': 10000,
|
32 |
+
'capabilities': ['training', 'inference']
|
33 |
+
}
|
34 |
+
|
35 |
+
resp = requests.post(f"{self.server_url}/register",
|
36 |
+
json={'client_id': self.client_id, 'client_info': client_info})
|
37 |
+
|
38 |
+
if resp.status_code == 200:
|
39 |
+
st.session_state.training_history.append({
|
40 |
+
'round': 0,
|
41 |
+
'active_clients': 1,
|
42 |
+
'clients_ready': 0,
|
43 |
+
'timestamp': datetime.now()
|
44 |
+
})
|
45 |
+
|
46 |
+
# Simulate client participation
|
47 |
+
while self.is_running:
|
48 |
+
try:
|
49 |
+
# Get training status
|
50 |
+
status = requests.get(f"{self.server_url}/training_status")
|
51 |
+
if status.status_code == 200:
|
52 |
+
data = status.json()
|
53 |
+
|
54 |
+
# Update training history
|
55 |
+
st.session_state.training_history.append({
|
56 |
+
'round': data.get('current_round', 0),
|
57 |
+
'active_clients': data.get('active_clients', 0),
|
58 |
+
'clients_ready': data.get('clients_ready', 0),
|
59 |
+
'timestamp': datetime.now()
|
60 |
+
})
|
61 |
+
|
62 |
+
# Keep only last 50 entries
|
63 |
+
if len(st.session_state.training_history) > 50:
|
64 |
+
st.session_state.training_history = st.session_state.training_history[-50:]
|
65 |
+
|
66 |
+
time.sleep(5) # Check every 5 seconds
|
67 |
+
|
68 |
+
except Exception as e:
|
69 |
+
print(f"Client simulator error: {e}")
|
70 |
+
time.sleep(10)
|
71 |
+
|
72 |
+
except Exception as e:
|
73 |
+
print(f"Failed to start client simulator: {e}")
|
74 |
+
self.is_running = False
|
75 |
+
|
76 |
st.set_page_config(page_title="Federated Credit Scoring Demo", layout="centered")
|
77 |
st.title("Federated Credit Scoring Demo (Federated Learning)")
|
78 |
|
|
|
239 |
st.markdown("---")
|
240 |
st.markdown("""
|
241 |
*This is a demonstration of federated learning concepts. For full functionality, run the federated server and clients locally.*
|
242 |
+
""")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
webapp/streamlit_app.py
CHANGED
@@ -6,6 +6,73 @@ import threading
|
|
6 |
import json
|
7 |
from datetime import datetime
|
8 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
9 |
st.set_page_config(page_title="Federated Credit Scoring Demo", layout="centered")
|
10 |
st.title("Federated Credit Scoring Demo (Federated Learning)")
|
11 |
|
@@ -172,71 +239,4 @@ if st.session_state.client_simulator and not DEMO_MODE:
|
|
172 |
st.markdown("---")
|
173 |
st.markdown("""
|
174 |
*This is a demonstration of federated learning concepts. For full functionality, run the federated server and clients locally.*
|
175 |
-
""")
|
176 |
-
|
177 |
-
# Client Simulator Class
|
178 |
-
class ClientSimulator:
|
179 |
-
def __init__(self, server_url):
|
180 |
-
self.server_url = server_url
|
181 |
-
self.client_id = f"web_client_{int(time.time())}"
|
182 |
-
self.is_running = False
|
183 |
-
self.thread = None
|
184 |
-
self.last_update = "Never"
|
185 |
-
|
186 |
-
def start(self):
|
187 |
-
self.is_running = True
|
188 |
-
self.thread = threading.Thread(target=self._run_client, daemon=True)
|
189 |
-
self.thread.start()
|
190 |
-
|
191 |
-
def stop(self):
|
192 |
-
self.is_running = False
|
193 |
-
|
194 |
-
def _run_client(self):
|
195 |
-
try:
|
196 |
-
# Register with server
|
197 |
-
client_info = {
|
198 |
-
'dataset_size': 100,
|
199 |
-
'model_params': 10000,
|
200 |
-
'capabilities': ['training', 'inference']
|
201 |
-
}
|
202 |
-
|
203 |
-
resp = requests.post(f"{self.server_url}/register",
|
204 |
-
json={'client_id': self.client_id, 'client_info': client_info})
|
205 |
-
|
206 |
-
if resp.status_code == 200:
|
207 |
-
st.session_state.training_history.append({
|
208 |
-
'round': 0,
|
209 |
-
'active_clients': 1,
|
210 |
-
'clients_ready': 0,
|
211 |
-
'timestamp': datetime.now()
|
212 |
-
})
|
213 |
-
|
214 |
-
# Simulate client participation
|
215 |
-
while self.is_running:
|
216 |
-
try:
|
217 |
-
# Get training status
|
218 |
-
status = requests.get(f"{self.server_url}/training_status")
|
219 |
-
if status.status_code == 200:
|
220 |
-
data = status.json()
|
221 |
-
|
222 |
-
# Update training history
|
223 |
-
st.session_state.training_history.append({
|
224 |
-
'round': data.get('current_round', 0),
|
225 |
-
'active_clients': data.get('active_clients', 0),
|
226 |
-
'clients_ready': data.get('clients_ready', 0),
|
227 |
-
'timestamp': datetime.now()
|
228 |
-
})
|
229 |
-
|
230 |
-
# Keep only last 50 entries
|
231 |
-
if len(st.session_state.training_history) > 50:
|
232 |
-
st.session_state.training_history = st.session_state.training_history[-50:]
|
233 |
-
|
234 |
-
time.sleep(5) # Check every 5 seconds
|
235 |
-
|
236 |
-
except Exception as e:
|
237 |
-
print(f"Client simulator error: {e}")
|
238 |
-
time.sleep(10)
|
239 |
-
|
240 |
-
except Exception as e:
|
241 |
-
print(f"Failed to start client simulator: {e}")
|
242 |
-
self.is_running = False
|
|
|
6 |
import json
|
7 |
from datetime import datetime
|
8 |
|
9 |
+
# Client Simulator Class (moved to top)
|
10 |
+
class ClientSimulator:
|
11 |
+
def __init__(self, server_url):
|
12 |
+
self.server_url = server_url
|
13 |
+
self.client_id = f"web_client_{int(time.time())}"
|
14 |
+
self.is_running = False
|
15 |
+
self.thread = None
|
16 |
+
self.last_update = "Never"
|
17 |
+
|
18 |
+
def start(self):
|
19 |
+
self.is_running = True
|
20 |
+
self.thread = threading.Thread(target=self._run_client, daemon=True)
|
21 |
+
self.thread.start()
|
22 |
+
|
23 |
+
def stop(self):
|
24 |
+
self.is_running = False
|
25 |
+
|
26 |
+
def _run_client(self):
|
27 |
+
try:
|
28 |
+
# Register with server
|
29 |
+
client_info = {
|
30 |
+
'dataset_size': 100,
|
31 |
+
'model_params': 10000,
|
32 |
+
'capabilities': ['training', 'inference']
|
33 |
+
}
|
34 |
+
|
35 |
+
resp = requests.post(f"{self.server_url}/register",
|
36 |
+
json={'client_id': self.client_id, 'client_info': client_info})
|
37 |
+
|
38 |
+
if resp.status_code == 200:
|
39 |
+
st.session_state.training_history.append({
|
40 |
+
'round': 0,
|
41 |
+
'active_clients': 1,
|
42 |
+
'clients_ready': 0,
|
43 |
+
'timestamp': datetime.now()
|
44 |
+
})
|
45 |
+
|
46 |
+
# Simulate client participation
|
47 |
+
while self.is_running:
|
48 |
+
try:
|
49 |
+
# Get training status
|
50 |
+
status = requests.get(f"{self.server_url}/training_status")
|
51 |
+
if status.status_code == 200:
|
52 |
+
data = status.json()
|
53 |
+
|
54 |
+
# Update training history
|
55 |
+
st.session_state.training_history.append({
|
56 |
+
'round': data.get('current_round', 0),
|
57 |
+
'active_clients': data.get('active_clients', 0),
|
58 |
+
'clients_ready': data.get('clients_ready', 0),
|
59 |
+
'timestamp': datetime.now()
|
60 |
+
})
|
61 |
+
|
62 |
+
# Keep only last 50 entries
|
63 |
+
if len(st.session_state.training_history) > 50:
|
64 |
+
st.session_state.training_history = st.session_state.training_history[-50:]
|
65 |
+
|
66 |
+
time.sleep(5) # Check every 5 seconds
|
67 |
+
|
68 |
+
except Exception as e:
|
69 |
+
print(f"Client simulator error: {e}")
|
70 |
+
time.sleep(10)
|
71 |
+
|
72 |
+
except Exception as e:
|
73 |
+
print(f"Failed to start client simulator: {e}")
|
74 |
+
self.is_running = False
|
75 |
+
|
76 |
st.set_page_config(page_title="Federated Credit Scoring Demo", layout="centered")
|
77 |
st.title("Federated Credit Scoring Demo (Federated Learning)")
|
78 |
|
|
|
239 |
st.markdown("---")
|
240 |
st.markdown("""
|
241 |
*This is a demonstration of federated learning concepts. For full functionality, run the federated server and clients locally.*
|
242 |
+
""")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|