Transcendental-Programmer commited on
Commit
135d1e4
Β·
1 Parent(s): e7b58b1

feat: complete demo

Browse files
README.md CHANGED
@@ -4,32 +4,34 @@ This project implements a federated learning framework combined with a Retrieval
4
 
5
  ## Features
6
 
7
- - Federated Learning using TensorFlow Federated
8
  - Privacy-preserving data generation using VAE/GAN
9
  - RAG integration for enhanced data quality
10
  - Secure Multi-Party Computation (SMPC)
11
  - Differential Privacy implementation
12
  - Kubernetes-based deployment
13
  - Comprehensive monitoring and logging
 
14
 
15
- ## Installation
16
-
17
- ```bash
18
- pip install -r requirements.txt
19
- ```
20
-
21
- ## Usage
22
 
 
23
 
24
- ## Project Structure
25
-
 
 
26
 
27
- ## License
28
-
29
- MIT
30
 
31
- ## Contributing
 
 
 
32
 
 
 
 
33
 
34
  ## Federated Credit Scoring Demo (with Web App)
35
 
@@ -65,7 +67,37 @@ streamlit run webapp/streamlit_app.py
65
  - Enter 32 features (dummy values are fine for demo)
66
  - Click "Predict Credit Score" to get a prediction from the federated model
67
  - View training progress in the app
 
68
 
69
  *For best results, keep the server and at least two clients running in parallel.*
70
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
71
  ---
 
 
 
4
 
5
  ## Features
6
 
7
+ - Federated Learning using TensorFlow
8
  - Privacy-preserving data generation using VAE/GAN
9
  - RAG integration for enhanced data quality
10
  - Secure Multi-Party Computation (SMPC)
11
  - Differential Privacy implementation
12
  - Kubernetes-based deployment
13
  - Comprehensive monitoring and logging
14
+ - **NEW: Interactive Web Demo** - Try it out without setup!
15
 
16
+ ## Quick Demo (No Installation Required)
 
 
 
 
 
 
17
 
18
+ πŸš€ **Live Demo**: [Hugging Face Spaces](https://huggingface.co/spaces/ArchCoder/federated-credit-scoring)
19
 
20
+ The web demo allows you to:
21
+ - Enter customer features and get credit score predictions
22
+ - See how federated learning works
23
+ - Understand privacy-preserving ML concepts
24
 
25
+ ## Installation
 
 
26
 
27
+ ```bash
28
+ # Create virtual environment
29
+ python3 -m venv venv
30
+ source venv/bin/activate # On Windows: venv\Scripts\activate
31
 
32
+ # Install dependencies
33
+ pip install -r requirements.txt
34
+ ```
35
 
36
  ## Federated Credit Scoring Demo (with Web App)
37
 
 
67
  - Enter 32 features (dummy values are fine for demo)
68
  - Click "Predict Credit Score" to get a prediction from the federated model
69
  - View training progress in the app
70
+ - Toggle between Demo Mode (no server required) and Real Mode (connects to server)
71
 
72
  *For best results, keep the server and at least two clients running in parallel.*
73
 
74
+ ## Project Structure
75
+
76
+ ```
77
+ FinFedRAG-Financial-Federated-RAG/
78
+ β”œβ”€β”€ src/
79
+ β”‚ β”œβ”€β”€ api/ # REST API for server and client communication
80
+ β”‚ β”œβ”€β”€ client/ # Federated learning client implementation
81
+ β”‚ β”œβ”€β”€ server/ # Federated learning server and coordinator
82
+ β”‚ β”œβ”€β”€ rag/ # Retrieval-Augmented Generation components
83
+ β”‚ β”œβ”€β”€ models/ # VAE/GAN models for data generation
84
+ β”‚ └── utils/ # Privacy, metrics, and utility functions
85
+ β”œβ”€β”€ webapp/ # Streamlit web application
86
+ β”œβ”€β”€ config/ # Configuration files
87
+ β”œβ”€β”€ tests/ # Unit and integration tests
88
+ β”œβ”€β”€ docker/ # Docker configurations
89
+ β”œβ”€β”€ kubernetes/ # Kubernetes deployment files
90
+ └── app.py # Root app.py for Hugging Face Spaces deployment
91
+ ```
92
+
93
+ ## License
94
+
95
+ MIT
96
+
97
+ ## Contributing
98
+
99
+ Please read our contributing guidelines before submitting pull requests.
100
+
101
  ---
102
+
103
+ **Demo URL**: https://huggingface.co/spaces/ArchCoder/federated-credit-scoring
app.py ADDED
@@ -0,0 +1,119 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import streamlit as st
2
+ import requests
3
+ import numpy as np
4
+ import time
5
+
6
+ st.set_page_config(page_title="Federated Credit Scoring Demo", layout="centered")
7
+ st.title("Federated Credit Scoring Demo (Federated Learning)")
8
+
9
+ # Sidebar configuration
10
+ st.sidebar.header("Configuration")
11
+ SERVER_URL = st.sidebar.text_input("Server URL", value="http://localhost:8080")
12
+ DEMO_MODE = st.sidebar.checkbox("Demo Mode (No Server Required)", value=True)
13
+
14
+ st.markdown("""
15
+ This demo shows how multiple banks can collaboratively train a credit scoring model using federated learning, without sharing raw data.
16
+ Enter customer features below to get a credit score prediction from the federated model.
17
+ """)
18
+
19
+ # --- Feature Input Form ---
20
+ st.header("Enter Customer Features")
21
+ with st.form("feature_form"):
22
+ features = []
23
+ cols = st.columns(4)
24
+ for i in range(32):
25
+ with cols[i % 4]:
26
+ val = st.number_input(f"Feature {i+1}", value=0.0, format="%.4f", key=f"f_{i}")
27
+ features.append(val)
28
+ submitted = st.form_submit_button("Predict Credit Score")
29
+
30
+ # --- Prediction ---
31
+ if submitted:
32
+ if DEMO_MODE:
33
+ # Demo mode - simulate prediction
34
+ with st.spinner("Processing prediction..."):
35
+ time.sleep(1) # Simulate processing time
36
+
37
+ # Simple demo prediction based on feature values
38
+ demo_prediction = sum(features) / len(features) * 100 + 500 # Scale to credit score range
39
+ st.success(f"Demo Prediction: Credit Score = {demo_prediction:.2f}")
40
+ st.info("πŸ’‘ This is a demo prediction. In a real federated system, this would come from the trained model.")
41
+
42
+ # Show what would happen in real mode
43
+ st.markdown("---")
44
+ st.markdown("**What happens in real federated learning:**")
45
+ st.markdown("1. Your features are sent to the federated server")
46
+ st.markdown("2. Server uses the global model (trained by multiple banks)")
47
+ st.markdown("3. Prediction is returned without exposing any bank's data")
48
+
49
+ else:
50
+ # Real mode - connect to server
51
+ try:
52
+ with st.spinner("Connecting to federated server..."):
53
+ resp = requests.post(f"{SERVER_URL}/predict", json={"features": features}, timeout=10)
54
+
55
+ if resp.status_code == 200:
56
+ prediction = resp.json().get("prediction")
57
+ st.success(f"Predicted Credit Score: {prediction:.2f}")
58
+ else:
59
+ st.error(f"Prediction failed: {resp.json().get('error', 'Unknown error')}")
60
+ except Exception as e:
61
+ st.error(f"Error connecting to server: {e}")
62
+ st.info("πŸ’‘ Try enabling Demo Mode to see the interface without a server.")
63
+
64
+ # --- Training Progress ---
65
+ st.header("Federated Training Progress")
66
+
67
+ if DEMO_MODE:
68
+ # Demo training progress
69
+ col1, col2, col3, col4 = st.columns(4)
70
+ with col1:
71
+ st.metric("Current Round", "3/10")
72
+ with col2:
73
+ st.metric("Active Clients", "3")
74
+ with col3:
75
+ st.metric("Model Accuracy", "85.2%")
76
+ with col4:
77
+ st.metric("Training Status", "Active")
78
+
79
+ st.info("πŸ’‘ Demo mode showing simulated training progress. In real federated learning, multiple banks would be training collaboratively.")
80
+
81
+ else:
82
+ # Real training progress
83
+ try:
84
+ status = requests.get(f"{SERVER_URL}/training_status", timeout=5)
85
+ if status.status_code == 200:
86
+ data = status.json()
87
+ col1, col2, col3, col4 = st.columns(4)
88
+ with col1:
89
+ st.metric("Current Round", f"{data.get('current_round', 0)}/{data.get('total_rounds', 10)}")
90
+ with col2:
91
+ st.metric("Active Clients", data.get('active_clients', 0))
92
+ with col3:
93
+ st.metric("Clients Ready", data.get('clients_ready', 0))
94
+ with col4:
95
+ st.metric("Training Status", "Active" if data.get('training_active', False) else "Inactive")
96
+ else:
97
+ st.warning("Could not fetch training status.")
98
+ except Exception as e:
99
+ st.warning(f"Could not connect to server for training status: {e}")
100
+
101
+ # --- How it works ---
102
+ st.header("How Federated Learning Works")
103
+ st.markdown("""
104
+ **Traditional ML:** All banks send their data to a central server β†’ Privacy risk ❌
105
+
106
+ **Federated Learning:**
107
+ 1. Each bank keeps their data locally βœ…
108
+ 2. Banks train models on their own data βœ…
109
+ 3. Only model updates (not data) are shared βœ…
110
+ 4. Server aggregates updates to create global model βœ…
111
+ 5. Global model is distributed back to all banks βœ…
112
+
113
+ **Result:** Collaborative learning without data sharing! 🎯
114
+ """)
115
+
116
+ st.markdown("---")
117
+ st.markdown("""
118
+ *This is a demonstration of federated learning concepts. For full functionality, run the federated server and clients locally.*
119
+ """)
config/client_config.yaml CHANGED
@@ -23,6 +23,7 @@ client:
23
  training:
24
  local_epochs: 3
25
  learning_rate: 0.001
 
26
 
27
  # Privacy configuration
28
  privacy:
 
23
  training:
24
  local_epochs: 3
25
  learning_rate: 0.001
26
+ batch_size: 32
27
 
28
  # Privacy configuration
29
  privacy:
config/server_config.yaml CHANGED
@@ -1,26 +1,25 @@
1
  # server_config.yaml configuration
2
 
3
- server:
4
- # API server configuration
5
- api:
6
- host: "0.0.0.0"
7
- port: 8080
8
- debug: false
9
-
10
- # Federated learning configuration
11
- federated:
12
- min_clients: 2
13
- rounds: 10
14
- sample_fraction: 0.8
15
-
16
- # Aggregation configuration
17
- aggregation:
18
- method: "fedavg"
19
- weighted: true
20
-
21
- # Monitoring configuration
22
- monitoring:
23
- log_level: "INFO"
24
 
25
  # Model configuration
26
  model:
 
1
  # server_config.yaml configuration
2
 
3
+ # API server configuration
4
+ api:
5
+ host: "0.0.0.0"
6
+ port: 8080
7
+ debug: false
8
+
9
+ # Federated learning configuration
10
+ federated:
11
+ min_clients: 2
12
+ rounds: 10
13
+ sample_fraction: 0.8
14
+
15
+ # Aggregation configuration
16
+ aggregation:
17
+ method: "fedavg"
18
+ weighted: true
19
+
20
+ # Monitoring configuration
21
+ monitoring:
22
+ log_level: "INFO"
 
23
 
24
  # Model configuration
25
  model:
requirements.txt CHANGED
@@ -1,13 +1,28 @@
1
- # Core ML frameworks
2
- tensorflow
3
- tensorflow-federated
4
- torch
5
- transformers
6
 
7
- # Data processing
8
- pandas
9
- numpy
10
- scikit-learn
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
 
12
  # RAG components
13
  elasticsearch
@@ -18,19 +33,8 @@ tensorflow-privacy
18
  pysyft
19
 
20
  # API and web
21
- flask
22
  fastapi
23
  uvicorn
24
- requests
25
- streamlit
26
-
27
- # Configuration and utilities
28
- pyyaml
29
- # Testing and development
30
- pytest
31
- black
32
- flake8
33
- isort
34
 
35
  # Documentation
36
  sphinx
 
1
+ # Core ML and Deep Learning
2
+ tensorflow>=2.8.0
3
+ numpy>=1.21.0
4
+ pandas>=1.3.0
5
+ scikit-learn>=1.0.0
6
 
7
+ # Web Framework and API
8
+ flask>=2.0.0
9
+ requests>=2.25.0
10
+ streamlit
11
+
12
+ # Configuration and utilities
13
+ pyyaml>=6.0
14
+ pathlib2>=2.3.0
15
+
16
+ # Development and testing
17
+ pytest>=6.0.0
18
+ pytest-cov>=2.0.0
19
+
20
+ # Logging and monitoring
21
+ python-json-logger>=2.0.0
22
+
23
+ # Optional: For advanced features
24
+ # tensorflow-federated>=0.20.0 # Uncomment if using TFF
25
+ # torch>=1.10.0 # Uncomment if using PyTorch
26
 
27
  # RAG components
28
  elasticsearch
 
33
  pysyft
34
 
35
  # API and web
 
36
  fastapi
37
  uvicorn
 
 
 
 
 
 
 
 
 
 
38
 
39
  # Documentation
40
  sphinx
webapp/streamlit_app.py CHANGED
@@ -1,11 +1,15 @@
1
  import streamlit as st
2
  import requests
3
  import numpy as np
 
4
 
5
  st.set_page_config(page_title="Federated Credit Scoring Demo", layout="centered")
6
  st.title("Federated Credit Scoring Demo (Federated Learning)")
7
 
 
 
8
  SERVER_URL = st.sidebar.text_input("Server URL", value="http://localhost:8080")
 
9
 
10
  st.markdown("""
11
  This demo shows how multiple banks can collaboratively train a credit scoring model using federated learning, without sharing raw data.
@@ -24,34 +28,92 @@ with st.form("feature_form"):
24
  submitted = st.form_submit_button("Predict Credit Score")
25
 
26
  # --- Prediction ---
27
- prediction = None
28
  if submitted:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
  try:
30
- resp = requests.post(f"{SERVER_URL}/predict", json={"features": features}, timeout=10)
31
- if resp.status_code == 200:
32
- prediction = resp.json().get("prediction")
33
- st.success(f"Predicted Credit Score: {prediction:.2f}")
 
 
 
 
 
 
 
 
34
  else:
35
- st.error(f"Prediction failed: {resp.json().get('error', 'Unknown error')}")
36
  except Exception as e:
37
- st.error(f"Error connecting to server: {e}")
38
 
39
- # --- Training Progress ---
40
- st.header("Federated Training Progress")
41
- try:
42
- status = requests.get(f"{SERVER_URL}/training_status", timeout=5)
43
- if status.status_code == 200:
44
- data = status.json()
45
- st.write(f"Current Round: {data.get('current_round', 0)} / {data.get('total_rounds', 10)}")
46
- st.write(f"Active Clients: {data.get('active_clients', 0)}")
47
- st.write(f"Clients Ready: {data.get('clients_ready', 0)}")
48
- st.write(f"Training Active: {data.get('training_active', False)}")
49
- else:
50
- st.warning("Could not fetch training status.")
51
- except Exception as e:
52
- st.warning(f"Could not connect to server for training status: {e}")
53
 
54
  st.markdown("---")
55
  st.markdown("""
56
- *This is a demo. All data is synthetic. For best results, run the federated server and at least two clients in parallel.*
57
  """)
 
1
  import streamlit as st
2
  import requests
3
  import numpy as np
4
+ import time
5
 
6
  st.set_page_config(page_title="Federated Credit Scoring Demo", layout="centered")
7
  st.title("Federated Credit Scoring Demo (Federated Learning)")
8
 
9
+ # Sidebar configuration
10
+ st.sidebar.header("Configuration")
11
  SERVER_URL = st.sidebar.text_input("Server URL", value="http://localhost:8080")
12
+ DEMO_MODE = st.sidebar.checkbox("Demo Mode (No Server Required)", value=True)
13
 
14
  st.markdown("""
15
  This demo shows how multiple banks can collaboratively train a credit scoring model using federated learning, without sharing raw data.
 
28
  submitted = st.form_submit_button("Predict Credit Score")
29
 
30
  # --- Prediction ---
 
31
  if submitted:
32
+ if DEMO_MODE:
33
+ # Demo mode - simulate prediction
34
+ with st.spinner("Processing prediction..."):
35
+ time.sleep(1) # Simulate processing time
36
+
37
+ # Simple demo prediction based on feature values
38
+ demo_prediction = sum(features) / len(features) * 100 + 500 # Scale to credit score range
39
+ st.success(f"Demo Prediction: Credit Score = {demo_prediction:.2f}")
40
+ st.info("πŸ’‘ This is a demo prediction. In a real federated system, this would come from the trained model.")
41
+
42
+ # Show what would happen in real mode
43
+ st.markdown("---")
44
+ st.markdown("**What happens in real federated learning:**")
45
+ st.markdown("1. Your features are sent to the federated server")
46
+ st.markdown("2. Server uses the global model (trained by multiple banks)")
47
+ st.markdown("3. Prediction is returned without exposing any bank's data")
48
+
49
+ else:
50
+ # Real mode - connect to server
51
+ try:
52
+ with st.spinner("Connecting to federated server..."):
53
+ resp = requests.post(f"{SERVER_URL}/predict", json={"features": features}, timeout=10)
54
+
55
+ if resp.status_code == 200:
56
+ prediction = resp.json().get("prediction")
57
+ st.success(f"Predicted Credit Score: {prediction:.2f}")
58
+ else:
59
+ st.error(f"Prediction failed: {resp.json().get('error', 'Unknown error')}")
60
+ except Exception as e:
61
+ st.error(f"Error connecting to server: {e}")
62
+ st.info("πŸ’‘ Try enabling Demo Mode to see the interface without a server.")
63
+
64
+ # --- Training Progress ---
65
+ st.header("Federated Training Progress")
66
+
67
+ if DEMO_MODE:
68
+ # Demo training progress
69
+ col1, col2, col3, col4 = st.columns(4)
70
+ with col1:
71
+ st.metric("Current Round", "3/10")
72
+ with col2:
73
+ st.metric("Active Clients", "3")
74
+ with col3:
75
+ st.metric("Model Accuracy", "85.2%")
76
+ with col4:
77
+ st.metric("Training Status", "Active")
78
+
79
+ st.info("πŸ’‘ Demo mode showing simulated training progress. In real federated learning, multiple banks would be training collaboratively.")
80
+
81
+ else:
82
+ # Real training progress
83
  try:
84
+ status = requests.get(f"{SERVER_URL}/training_status", timeout=5)
85
+ if status.status_code == 200:
86
+ data = status.json()
87
+ col1, col2, col3, col4 = st.columns(4)
88
+ with col1:
89
+ st.metric("Current Round", f"{data.get('current_round', 0)}/{data.get('total_rounds', 10)}")
90
+ with col2:
91
+ st.metric("Active Clients", data.get('active_clients', 0))
92
+ with col3:
93
+ st.metric("Clients Ready", data.get('clients_ready', 0))
94
+ with col4:
95
+ st.metric("Training Status", "Active" if data.get('training_active', False) else "Inactive")
96
  else:
97
+ st.warning("Could not fetch training status.")
98
  except Exception as e:
99
+ st.warning(f"Could not connect to server for training status: {e}")
100
 
101
+ # --- How it works ---
102
+ st.header("How Federated Learning Works")
103
+ st.markdown("""
104
+ **Traditional ML:** All banks send their data to a central server β†’ Privacy risk ❌
105
+
106
+ **Federated Learning:**
107
+ 1. Each bank keeps their data locally βœ…
108
+ 2. Banks train models on their own data βœ…
109
+ 3. Only model updates (not data) are shared βœ…
110
+ 4. Server aggregates updates to create global model βœ…
111
+ 5. Global model is distributed back to all banks βœ…
112
+
113
+ **Result:** Collaborative learning without data sharing! 🎯
114
+ """)
115
 
116
  st.markdown("---")
117
  st.markdown("""
118
+ *This is a demonstration of federated learning concepts. For full functionality, run the federated server and clients locally.*
119
  """)