Update README.md
Browse files
README.md
CHANGED
@@ -1,89 +1,89 @@
|
|
1 |
-
---
|
2 |
-
tags:
|
3 |
-
- text-classification
|
4 |
-
- transformers
|
5 |
-
- biobert
|
6 |
-
- miRNA
|
7 |
-
- biomedical
|
8 |
-
- LoRA
|
9 |
-
- fine-tuning
|
10 |
-
library_name: transformers
|
11 |
-
datasets:
|
12 |
-
- custom-biomedical-dataset
|
13 |
-
license: apache-2.0
|
14 |
-
---
|
15 |
-
|
16 |
-
# 𧬠miRNA-BioBERT: Fine-Tuned BioBERT for miRNA Sentence Classification
|
17 |
-
**Fine-tuned BioBERT model for classifying miRNA-related sentences in biomedical research papers.**
|
18 |
-
|
19 |
-
<!-- π **Hugging Face Model Link**: [debjit20504/miRNA-biobert](https://huggingface.co/debjit20504/miRNA-biobert) -->
|
20 |
-
|
21 |
-
---
|
22 |
-
|
23 |
-
## π Overview
|
24 |
-
**miRNA-BioBERT** is a fine-tuned version of [BioBERT](https://huggingface.co/dmis-lab/biobert-base-cased-v1.1), trained specifically for **classifying sentences** as **miRNA-related (relevant) or not (irrelevant)**. The model is useful for **automating literature reviews**, **extracting relevant sentences**, and **identifying key insights** in genomic research.
|
25 |
-
|
26 |
-
β **Base Model**: `dmis-lab/biobert-base-cased-v1.1`
|
27 |
-
β **Fine-tuning Method**: **LoRA (Low-Rank Adaptation)**
|
28 |
-
β **Dataset**: **Curated biomedical text corpus containing labeled miRNA-relevant and non-relevant sentences**
|
29 |
-
β **Task**: **Binary classification (1 =
|
30 |
-
β **Trained on**: **RTX A6000 GPU (5 epochs, batch size 32, learning rate 2e-5)**
|
31 |
-
|
32 |
-
## π How to Use the Model
|
33 |
-
### 1οΈβ£ Install Dependencies
|
34 |
-
```bash
|
35 |
-
pip install transformers torch
|
36 |
-
```
|
37 |
-
```python
|
38 |
-
from transformers import AutoModelForSequenceClassification, AutoTokenizer
|
39 |
-
import torch
|
40 |
-
|
41 |
-
# Load the model and tokenizer
|
42 |
-
model_name = "debjit20504/miRNA-biobert"
|
43 |
-
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
44 |
-
model = AutoModelForSequenceClassification.from_pretrained(model_name)
|
45 |
-
|
46 |
-
# Move model to GPU or MPS (for Mac)
|
47 |
-
device = torch.device("mps") if torch.backends.mps.is_available() else torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
48 |
-
model.to(device)
|
49 |
-
model.eval()
|
50 |
-
|
51 |
-
def classify_text(text):
|
52 |
-
inputs = tokenizer(text, return_tensors="pt").to(device)
|
53 |
-
with torch.no_grad():
|
54 |
-
output = model(**inputs)
|
55 |
-
label = torch.argmax(output.logits, dim=1).item()
|
56 |
-
return "
|
57 |
-
|
58 |
-
# Example Test
|
59 |
-
sample_text = "miRNA translation is regulated by miRNAs."
|
60 |
-
print(f"Classification: {classify_text(sample_text)}")
|
61 |
-
```
|
62 |
-
|
63 |
-
## π Training Details
|
64 |
-
- Dataset: Biomedical text dataset with 429,785 relevant sentences and 87,966 irrelevant sentences.
|
65 |
-
- Fine-Tuning Method: LoRA (Low-Rank Adaptation) for efficient training.
|
66 |
-
- Training Hardware: NVIDIA RTX A6000 GPU.
|
67 |
-
- Training Settings:
|
68 |
-
- Batch size: 32
|
69 |
-
- Learning rate: 2e-5
|
70 |
-
- Optimizer: AdamW
|
71 |
-
- Warmup steps: 1000
|
72 |
-
- Epochs: 5
|
73 |
-
- Mixed precision (fp16): β
Enabled for efficiency.
|
74 |
-
|
75 |
-
---
|
76 |
-
|
77 |
-
## π Model Applications
|
78 |
-
β
**Biomedical NLP** β Extracting meaningful information from biomedical literature.
|
79 |
-
β
**miRNA Research** β Identifying sentences discussing miRNA mechanisms.
|
80 |
-
β
**Automated Literature Review** β Filtering relevant studies efficiently.
|
81 |
-
β
**Genomics & Bioinformatics** β Enhancing data retrieval from scientific texts.
|
82 |
-
|
83 |
-
---
|
84 |
-
|
85 |
-
## π¬ Contact
|
86 |
-
For any questions or collaborations, reach out via:
|
87 |
-
|
88 |
-
**π§ Email**: [email protected]
|
89 |
**π LinkedIn**: https://www.linkedin.com/in/debjit-pramanik-88a837171/
|
|
|
1 |
+
---
|
2 |
+
tags:
|
3 |
+
- text-classification
|
4 |
+
- transformers
|
5 |
+
- biobert
|
6 |
+
- miRNA
|
7 |
+
- biomedical
|
8 |
+
- LoRA
|
9 |
+
- fine-tuning
|
10 |
+
library_name: transformers
|
11 |
+
datasets:
|
12 |
+
- custom-biomedical-dataset
|
13 |
+
license: apache-2.0
|
14 |
+
---
|
15 |
+
|
16 |
+
# 𧬠miRNA-BioBERT: Fine-Tuned BioBERT for miRNA Sentence Classification
|
17 |
+
**Fine-tuned BioBERT model for classifying miRNA-related sentences in biomedical research papers.**
|
18 |
+
|
19 |
+
<!-- π **Hugging Face Model Link**: [debjit20504/miRNA-biobert](https://huggingface.co/debjit20504/miRNA-biobert) -->
|
20 |
+
|
21 |
+
---
|
22 |
+
|
23 |
+
## π Overview
|
24 |
+
**miRNA-BioBERT** is a fine-tuned version of [BioBERT](https://huggingface.co/dmis-lab/biobert-base-cased-v1.1), trained specifically for **classifying sentences** as **miRNA-related (relevant) or not (irrelevant)**. The model is useful for **automating literature reviews**, **extracting relevant sentences**, and **identifying key insights** in genomic research.
|
25 |
+
|
26 |
+
β **Base Model**: `dmis-lab/biobert-base-cased-v1.1`
|
27 |
+
β **Fine-tuning Method**: **LoRA (Low-Rank Adaptation)**
|
28 |
+
β **Dataset**: **Curated biomedical text corpus containing labeled miRNA-relevant and non-relevant sentences**
|
29 |
+
β **Task**: **Binary classification (1 = functional, 0 = non-functional)**
|
30 |
+
β **Trained on**: **RTX A6000 GPU (5 epochs, batch size 32, learning rate 2e-5)**
|
31 |
+
|
32 |
+
## π How to Use the Model
|
33 |
+
### 1οΈβ£ Install Dependencies
|
34 |
+
```bash
|
35 |
+
pip install transformers torch
|
36 |
+
```
|
37 |
+
```python
|
38 |
+
from transformers import AutoModelForSequenceClassification, AutoTokenizer
|
39 |
+
import torch
|
40 |
+
|
41 |
+
# Load the model and tokenizer
|
42 |
+
model_name = "debjit20504/miRNA-biobert"
|
43 |
+
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
44 |
+
model = AutoModelForSequenceClassification.from_pretrained(model_name)
|
45 |
+
|
46 |
+
# Move model to GPU or MPS (for Mac)
|
47 |
+
device = torch.device("mps") if torch.backends.mps.is_available() else torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
48 |
+
model.to(device)
|
49 |
+
model.eval()
|
50 |
+
|
51 |
+
def classify_text(text):
|
52 |
+
inputs = tokenizer(text, return_tensors="pt").to(device)
|
53 |
+
with torch.no_grad():
|
54 |
+
output = model(**inputs)
|
55 |
+
label = torch.argmax(output.logits, dim=1).item()
|
56 |
+
return "functional" if label == 1 else "Non-functional"
|
57 |
+
|
58 |
+
# Example Test
|
59 |
+
sample_text = "miRNA translation is regulated by miRNAs."
|
60 |
+
print(f"Classification: {classify_text(sample_text)}")
|
61 |
+
```
|
62 |
+
|
63 |
+
## π Training Details
|
64 |
+
- Dataset: Biomedical text dataset with 429,785 relevant sentences and 87,966 irrelevant sentences.
|
65 |
+
- Fine-Tuning Method: LoRA (Low-Rank Adaptation) for efficient training.
|
66 |
+
- Training Hardware: NVIDIA RTX A6000 GPU.
|
67 |
+
- Training Settings:
|
68 |
+
- Batch size: 32
|
69 |
+
- Learning rate: 2e-5
|
70 |
+
- Optimizer: AdamW
|
71 |
+
- Warmup steps: 1000
|
72 |
+
- Epochs: 5
|
73 |
+
- Mixed precision (fp16): β
Enabled for efficiency.
|
74 |
+
|
75 |
+
---
|
76 |
+
|
77 |
+
## π Model Applications
|
78 |
+
β
**Biomedical NLP** β Extracting meaningful information from biomedical literature.
|
79 |
+
β
**miRNA Research** β Identifying sentences discussing miRNA mechanisms.
|
80 |
+
β
**Automated Literature Review** β Filtering relevant studies efficiently.
|
81 |
+
β
**Genomics & Bioinformatics** β Enhancing data retrieval from scientific texts.
|
82 |
+
|
83 |
+
---
|
84 |
+
|
85 |
+
## π¬ Contact
|
86 |
+
For any questions or collaborations, reach out via:
|
87 |
+
|
88 |
+
**π§ Email**: [email protected]
|
89 |
**π LinkedIn**: https://www.linkedin.com/in/debjit-pramanik-88a837171/
|