wi-lab commited on
Commit
c423d96
Β·
verified Β·
1 Parent(s): 330ebfb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -11
README.md CHANGED
@@ -41,7 +41,7 @@ LWM-v1.1 is designed to be seamlessly integrated into downstream tasks as a sour
41
  For example, the following figure demonstrates the advantages of using **LWM-v1.1-based highly compact CLS embeddings** and **high-dimensional channel embeddings** over raw channels for the LoS/NLoS classification task. The raw dataset is derived from channels of size (128, 32) between BS 3 and 8,299 users in the densified Denver scenario of the DeepMIMO dataset.
42
 
43
  <p align="center">
44
- <img src="https://huggingface.co/wi-lab/lwm-v1.1/resolve/main/images/los_perf.png" alt="LoS/NLoS Classification Performance" width="800"/>
45
  </p>
46
 
47
  <p align="center">
@@ -50,16 +50,32 @@ For example, the following figure demonstrates the advantages of using **LWM-v1.
50
 
51
  ---
52
 
53
- ## **Overview of Main Changes in LWM-v1.1**
54
- 1. **No channel size limitation**
55
- 2. **Larger and more diverse pretraining dataset**
56
- 3. **Fine-tuning capabilities for task-specific embedding generation**
57
- 4. **Increased model capacity**
58
- 5. **2D patch segmentation for realistic learning**
59
- 6. **Challenging MCM task with higher masking ratio**
60
- 7. **Support for larger input sizes**
61
- 8. **Optimized training strategy**
62
- 9. **Improved computational efficiency**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
63
 
64
  ---
65
 
 
41
  For example, the following figure demonstrates the advantages of using **LWM-v1.1-based highly compact CLS embeddings** and **high-dimensional channel embeddings** over raw channels for the LoS/NLoS classification task. The raw dataset is derived from channels of size (128, 32) between BS 3 and 8,299 users in the densified Denver scenario of the DeepMIMO dataset.
42
 
43
  <p align="center">
44
+ <img src="https://huggingface.co/wi-lab/lwm-v1.1/resolve/main/images/los_perf.png" alt="LoS/NLoS Classification Performance" width="600"/>
45
  </p>
46
 
47
  <p align="center">
 
50
 
51
  ---
52
 
53
+ ### **🧩 Puzzle Pieces that Redefine LWM-v1.0**
54
+
55
+ #### **1️⃣ Breaking Barriers**
56
+ πŸ”“ No Channel Size Limitation
57
+ πŸ“ Support for Larger Input Sizes
58
+
59
+ #### **2️⃣ Smarter Foundations**
60
+ 🌍 A More Diverse Dataset
61
+ 🎭 Tougher Masking Challenges with 40% MCM Ratio
62
+
63
+ #### **3️⃣ Amplified Power**
64
+ πŸ”’ Expanded Capacity: 2.5M Parameters
65
+ πŸ“ Realistic 2D Patch Segmentation
66
+
67
+ #### **4️⃣ Efficiency Engineered**
68
+ βš™οΈ Optimized Training with AdamW + Cosine Decay
69
+ ⚑ Faster Computation with Streamlined Attention Heads
70
+
71
+ ### **πŸŒ€ See the Difference at a Glance**
72
+
73
+ | Feature | LWM-v1.0 | **LWM-v1.1** |
74
+ |-----------------------------|-------------------------|-----------------------|
75
+ | Channel Size Limitation | Fixed at (32, 32) | **Dynamic, up to (512)** |
76
+ | Masking Ratio | 15% | **40%** |
77
+ | Parameters | 600K | **2.5M** |
78
+ | Sequence Length Support | 128 | **512** |
79
 
80
  ---
81