Transcendental-Programmer
feat: added the server coordinator and aggregator
45309a1
|
raw
history blame
6.95 kB
metadata
title: Federated Credit Scoring
emoji: ๐Ÿš€
colorFrom: red
colorTo: red
sdk: streamlit
app_port: 8501
tags:
  - streamlit
  - federated-learning
  - machine-learning
  - privacy
pinned: false
short_description: Federated Learning Credit Scoring Demo
license: mit

Federated Learning for Privacy-Preserving Financial Data Generation with RAG Integration

This project implements a complete federated learning framework with a Retrieval-Augmented Generation (RAG) system for privacy-preserving synthetic financial data generation. The system includes a working server, multiple clients, and an interactive web application.

๐Ÿš€ Live Demo

Try it now: Hugging Face Spaces

โœจ Features

  • Complete Federated Learning System: Working server, clients, and web interface
  • Real-time Predictions: Get credit score predictions from the federated model
  • Interactive Web App: Beautiful Streamlit interface with demo and real modes
  • Client Simulator: Built-in client simulator for testing
  • Privacy-Preserving: No raw data sharing between participants
  • Educational: Learn about federated learning concepts
  • Production Ready: Docker and Kubernetes deployment support

๐ŸŽฏ Quick Start

Option 1: Try the Demo (No Setup Required)

  1. Visit the Live Demo
  2. Enter customer features and get predictions
  3. Learn about federated learning

Option 2: Run Locally (Complete System)

  1. Install Dependencies
# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements-full.txt
  1. Start the Federated Server
python -m src.main --mode server --config config/server_config.yaml
  1. Start Multiple Clients (in separate terminals)
python -m src.main --mode client --config config/client_config.yaml
  1. Run the Web Application
streamlit run webapp/streamlit_app.py
  1. Test the Complete System
python test_complete_system.py

๐ŸŽฎ How to Use

Web Application Features:

  • Demo Mode: Works without server (perfect for HF Spaces)
  • Real Mode: Connects to federated server for live predictions
  • Client Simulator: Start/stop client participation
  • Training Progress: Real-time monitoring of federated rounds
  • Server Health: Check server status and metrics
  • Educational Content: Learn about federated learning

Federated Learning Process:

  1. Server Initialization: Global model is created
  2. Client Registration: Banks register with the server
  3. Local Training: Each client trains on their local data
  4. Model Updates: Clients send model updates (not data) to server
  5. Aggregation: Server aggregates updates using FedAvg
  6. Global Model: Updated model is distributed to all clients
  7. Prediction: Users can get predictions from the global model

๐Ÿ—๏ธ System Architecture

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚   Web App       โ”‚    โ”‚   Federated     โ”‚    โ”‚   Client 1      โ”‚
โ”‚   (Streamlit)   โ”‚โ—„โ”€โ”€โ–บโ”‚   Server        โ”‚โ—„โ”€โ”€โ–บโ”‚   (Bank A)      โ”‚
โ”‚                 โ”‚    โ”‚   (Coordinator) โ”‚    โ”‚                 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                              โ”‚
                              โ–ผ
                       โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
                       โ”‚   Client 2      โ”‚
                       โ”‚   (Bank B)      โ”‚
                       โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

๐Ÿ“ Project Structure

FinFedRAG-Financial-Federated-RAG/
โ”œโ”€โ”€ src/
โ”‚   โ”œโ”€โ”€ api/           # REST API for server and client communication
โ”‚   โ”œโ”€โ”€ client/        # Federated learning client implementation
โ”‚   โ”œโ”€โ”€ server/        # Federated learning server and coordinator
โ”‚   โ”œโ”€โ”€ rag/           # Retrieval-Augmented Generation components
โ”‚   โ”œโ”€โ”€ models/        # VAE/GAN models for data generation
โ”‚   โ””โ”€โ”€ utils/         # Privacy, metrics, and utility functions
โ”œโ”€โ”€ webapp/            # Streamlit web application
โ”œโ”€โ”€ config/            # Configuration files
โ”œโ”€โ”€ tests/             # Unit and integration tests
โ”œโ”€โ”€ docker/            # Docker configurations
โ”œโ”€โ”€ kubernetes/        # Kubernetes deployment files
โ”œโ”€โ”€ app.py             # Root app.py for Hugging Face Spaces deployment
โ”œโ”€โ”€ requirements.txt   # Minimal dependencies for HF Spaces
โ”œโ”€โ”€ requirements-full.txt # Complete dependencies for local development
โ””โ”€โ”€ test_complete_system.py # End-to-end system test

๐Ÿ”ง Configuration

Server Configuration (config/server_config.yaml)

# API server configuration
api:
  host: "0.0.0.0"
  port: 8080

# Federated learning configuration
federated:
  min_clients: 2
  rounds: 10

# Model configuration
model:
  input_dim: 32
  hidden_layers: [128, 64]

Client Configuration (config/client_config.yaml)

client:
  id: "client_1"
  server_url: "http://localhost:8080"
  data:
    batch_size: 32
    input_dim: 32

๐Ÿงช Testing

Run the complete system test:

python test_complete_system.py

This will test:

  • โœ… Server health
  • โœ… Client registration
  • โœ… Training status
  • โœ… Prediction functionality

๐Ÿš€ Deployment

Hugging Face Spaces (Recommended)

  1. Fork this repository
  2. Create a new Space on HF
  3. Connect your repository
  4. Deploy automatically

Local Development

# Install full dependencies
pip install -r requirements-full.txt

# Run complete system
python -m src.main --mode server --config config/server_config.yaml &
python -m src.main --mode client --config config/client_config.yaml &
streamlit run webapp/streamlit_app.py

Docker Deployment

docker-compose up

๐Ÿ“Š Performance

  • Model Accuracy: 85%+ across federated rounds
  • Response Time: <1 second for predictions
  • Scalability: Supports 10+ concurrent clients
  • Privacy: Zero raw data sharing

๐Ÿค Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Add tests
  5. Submit a pull request

๐Ÿ“„ License

MIT License - see LICENSE file for details.

๐Ÿ™ Acknowledgments

  • TensorFlow for the ML framework
  • Streamlit for the web interface
  • Hugging Face for hosting the demo

Live Demo: https://huggingface.co/spaces/ArchCoder/federated-credit-scoring