Update README.md
Browse files
README.md
CHANGED
@@ -50,7 +50,7 @@ For example, the following figure demonstrates the advantages of using **LWM-v1.
|
|
50 |
|
51 |
---
|
52 |
|
53 |
-
|
54 |
|
55 |
#### **1️⃣ Breaking Barriers**
|
56 |
🔓 No Channel Size Limitation
|
@@ -72,10 +72,12 @@ For example, the following figure demonstrates the advantages of using **LWM-v1.
|
|
72 |
|
73 |
| Feature | LWM-v1.0 | **LWM-v1.1** |
|
74 |
|-----------------------------|-------------------------|-----------------------|
|
75 |
-
| Channel Size Limitation | Fixed at (32, 32)
|
76 |
-
|
|
77 |
-
|
|
78 |
-
|
|
|
|
|
|
79 |
|
80 |
---
|
81 |
|
@@ -91,7 +93,7 @@ This approach eliminates the rigidity of fixed channel sizes and positions LWM-v
|
|
91 |
### **Larger and More Diverse Pretraining Dataset**
|
92 |
Generalization is a critical aspect of any foundation model. In **LWM-v1.1**, we significantly expanded the training dataset to cover more diverse scenarios and environments. We added **seven new city scenarios**—Charlotte, Denver, Oklahoma, Indianapolis, Fort Worth, Santa Clara, and San Diego—to enrich the model’s exposure to a variety of urban layouts. To enhance the spatial resolution of the training data, we reduced the grid spacing between user locations in the DeepMIMO city scenarios from **2.5m to 1m**, resulting in a higher density of user positions. This adjustment required re-performing ray tracing for all scenarios to generate high-resolution wireless channel data.
|
93 |
|
94 |
-
Additionally, we introduced **channels from multiple base stations** in each scenario, with distinct (N, SC) pairs to ensure the model encounters a broad range of channel characteristics. This diversity mirrors the variability found in real-world deployments, such as urban, suburban, and rural environments. By exposing LWM-v1.1 to this diversity, the model gains the ability to generalize across environments with distinct propagation characteristics, making it more reliable and versatile.
|
95 |
|
96 |
### **Fine-Tuning for Task-Specific Embedding Generation**
|
97 |
While pretraining provides a robust feature extractor, downstream tasks often require tailored embeddings. In **LWM-v1.1**, we introduced **fine-tuning options** that give users the flexibility to customize the model for specific tasks. Users can now **freeze specific layers** of the model, allowing the remaining layers to adapt to task-specific requirements. This feature is particularly valuable for tasks prone to overfitting, such as **LoS/NLoS classification**, where excessive training on all layers can lead to suboptimal generalization.
|
|
|
50 |
|
51 |
---
|
52 |
|
53 |
+
## **🧩 Puzzle Pieces that Redefine LWM-v1.0**
|
54 |
|
55 |
#### **1️⃣ Breaking Barriers**
|
56 |
🔓 No Channel Size Limitation
|
|
|
72 |
|
73 |
| Feature | LWM-v1.0 | **LWM-v1.1** |
|
74 |
|-----------------------------|-------------------------|-----------------------|
|
75 |
+
| Channel Size Limitation | Fixed at (32, 32) | **Dynamic** |
|
76 |
+
| Pre-training Samples | 820K | **1.05M** |
|
77 |
+
| Pre-training Scenarios | 15 | **80** |
|
78 |
+
| Masking Ratio | 15% | **40%** |
|
79 |
+
| Parameters | 600K | **2.5M** |
|
80 |
+
| Sequence Length Support | 128 | **512** |
|
81 |
|
82 |
---
|
83 |
|
|
|
93 |
### **Larger and More Diverse Pretraining Dataset**
|
94 |
Generalization is a critical aspect of any foundation model. In **LWM-v1.1**, we significantly expanded the training dataset to cover more diverse scenarios and environments. We added **seven new city scenarios**—Charlotte, Denver, Oklahoma, Indianapolis, Fort Worth, Santa Clara, and San Diego—to enrich the model’s exposure to a variety of urban layouts. To enhance the spatial resolution of the training data, we reduced the grid spacing between user locations in the DeepMIMO city scenarios from **2.5m to 1m**, resulting in a higher density of user positions. This adjustment required re-performing ray tracing for all scenarios to generate high-resolution wireless channel data.
|
95 |
|
96 |
+
Additionally, we introduced **channels from multiple base stations** in each scenario, with distinct (N, SC) pairs to ensure the model encounters a broad range of channel characteristics. This expansion resulted in a total of **1.3 million pre-training samples**, with 20% allocated for validation. This diversity mirrors the variability found in real-world deployments, such as urban, suburban, and rural environments. By exposing LWM-v1.1 to this diversity, the model gains the ability to generalize across environments with distinct propagation characteristics, making it more reliable and versatile.
|
97 |
|
98 |
### **Fine-Tuning for Task-Specific Embedding Generation**
|
99 |
While pretraining provides a robust feature extractor, downstream tasks often require tailored embeddings. In **LWM-v1.1**, we introduced **fine-tuning options** that give users the flexibility to customize the model for specific tasks. Users can now **freeze specific layers** of the model, allowing the remaining layers to adapt to task-specific requirements. This feature is particularly valuable for tasks prone to overfitting, such as **LoS/NLoS classification**, where excessive training on all layers can lead to suboptimal generalization.
|