File size: 6,840 Bytes
80ee9ee
 
 
 
 
 
 
 
 
 
 
 
 
bea3e2c
80ee9ee
 
45309a1
 
 
b407fad
45309a1
b407fad
45309a1
 
 
b407fad
45309a1
 
 
 
 
 
 
 
 
b407fad
45309a1
b407fad
45309a1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
b407fad
45309a1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
b407fad
45309a1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
b407fad
45309a1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
b407fad
45309a1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
b407fad
45309a1
 
 
 
 
 
 
b407fad
 
 
 
45309a1
b407fad
45309a1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
b407fad
45309a1
 
 
 
 
 
b407fad
45309a1
 
 
 
 
 
 
b407fad
45309a1
 
 
b407fad
45309a1
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
---
title: Federated Credit Scoring
emoji: πŸš€
colorFrom: red
colorTo: red
sdk: streamlit
app_port: 8501
tags:
- streamlit
- federated-learning
- machine-learning
- privacy
pinned: false
short_description: Federated Learning Credit Scoring Demo 
license: mit
---

# Federated Learning for Privacy-Preserving Financial Data Generation with RAG Integration

This project implements a complete federated learning framework with a Retrieval-Augmented Generation (RAG) system for privacy-preserving synthetic financial data generation. The system includes a working server, multiple clients, and an interactive web application.

## Live Demo

**Try it now**: [Hugging Face Spaces](https://huggingface.co/spaces/ArchCoder/federated-credit-scoring)

## Features

- **Complete Federated Learning System**: Working server, clients, and web interface
- **Real-time Predictions**: Get credit score predictions from the federated model
- **Interactive Web App**: Beautiful Streamlit interface with demo and real modes
- **Client Simulator**: Built-in client simulator for testing
- **Privacy-Preserving**: No raw data sharing between participants
- **Educational**: Learn about federated learning concepts
- **Production Ready**: Docker and Kubernetes deployment support

## Quick Start

### Option 1: Try the Demo
1. Visit the [Live Demo](https://huggingface.co/spaces/ArchCoder/federated-credit-scoring)
2. Enter customer features and get predictions
3. Learn about federated learning

### Option 2: Run Locally (Complete System)

1. **Install Dependencies**
```bash
# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements-full.txt
```

2. **Start the Federated Server**
```bash
python -m src.main --mode server --config config/server_config.yaml
```

3. **Start Multiple Clients** (in separate terminals)
```bash
python -m src.main --mode client --config config/client_config.yaml
```

4. **Run the Web Application**
```bash
streamlit run webapp/streamlit_app.py
```

5. **Test the Complete System**
```bash
python test_complete_system.py
```

## How to Use

### Web Application Features:
- **Demo Mode**: Works without server (perfect for HF Spaces)
- **Real Mode**: Connects to federated server for live predictions
- **Client Simulator**: Start/stop client participation
- **Training Progress**: Real-time monitoring of federated rounds
- **Server Health**: Check server status and metrics
- **Educational Content**: Learn about federated learning

### Federated Learning Process:
1. **Server Initialization**: Global model is created
2. **Client Registration**: Banks register with the server
3. **Local Training**: Each client trains on their local data
4. **Model Updates**: Clients send model updates (not data) to server
5. **Aggregation**: Server aggregates updates using FedAvg
6. **Global Model**: Updated model is distributed to all clients
7. **Prediction**: Users can get predictions from the global model

## System Architecture

```
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   Web App       β”‚    β”‚   Federated     β”‚    β”‚   Client 1      β”‚
β”‚   (Streamlit)   │◄──►│   Server        │◄──►│   (Bank A)      β”‚
β”‚                 β”‚    β”‚   (Coordinator) β”‚    β”‚                 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                              β”‚
                              β–Ό
                       β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                       β”‚   Client 2      β”‚
                       β”‚   (Bank B)      β”‚
                       β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
```

## Project Structure

```
FinFedRAG-Financial-Federated-RAG/
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ api/           # REST API for server and client communication
β”‚   β”œβ”€β”€ client/        # Federated learning client implementation
β”‚   β”œβ”€β”€ server/        # Federated learning server and coordinator
β”‚   β”œβ”€β”€ rag/           # Retrieval-Augmented Generation components
β”‚   β”œβ”€β”€ models/        # VAE/GAN models for data generation
β”‚   └── utils/         # Privacy, metrics, and utility functions
β”œβ”€β”€ webapp/            # Streamlit web application
β”œβ”€β”€ config/            # Configuration files
β”œβ”€β”€ tests/             # Unit and integration tests
β”œβ”€β”€ docker/            # Docker configurations
β”œβ”€β”€ kubernetes/        # Kubernetes deployment files
β”œβ”€β”€ app.py             # Root app.py for Hugging Face Spaces deployment
β”œβ”€β”€ requirements.txt   # Minimal dependencies for HF Spaces
β”œβ”€β”€ requirements-full.txt # Complete dependencies for local development
└── test_complete_system.py # End-to-end system test
```

## Configuration

### Server Configuration (`config/server_config.yaml`)
```yaml
# API server configuration
api:
  host: "0.0.0.0"
  port: 8080

# Federated learning configuration
federated:
  min_clients: 2
  rounds: 10

# Model configuration
model:
  input_dim: 32
  hidden_layers: [128, 64]
```

### Client Configuration (`config/client_config.yaml`)
```yaml
client:
  id: "client_1"
  server_url: "http://localhost:8080"
  data:
    batch_size: 32
    input_dim: 32
```

## Testing

Run the complete system test:
```bash
python test_complete_system.py
```

This will test:
- Server health
- Client registration
- Training status
- Prediction functionality

## Deployment

### Hugging Face Spaces (Recommended)
1. Fork this repository
2. Create a new Space on HF
3. Connect your repository
4. Deploy automatically

### Local Development
```bash
# Install full dependencies
pip install -r requirements-full.txt

# Run complete system
python -m src.main --mode server --config config/server_config.yaml &
python -m src.main --mode client --config config/client_config.yaml &
streamlit run webapp/streamlit_app.py
```

### Docker Deployment
```bash
docker-compose up
```

## Performance

- **Model Accuracy**: 85%+ across federated rounds
- **Response Time**: <1 second for predictions
- **Scalability**: Supports 10+ concurrent clients
- **Privacy**: Zero raw data sharing

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

## License

MIT License - see LICENSE file for details.

## Acknowledgments

- TensorFlow for the ML framework
- Streamlit for the web interface
- Hugging Face for hosting the demo

---

**Live Demo**: https://huggingface.co/spaces/ArchCoder/federated-credit-scoring