Joint NT-ESM2 DNA-Protein Models
This repository contains jointly trained Nucleotide Transformer (NT) and ESM2 models for DNA-protein sequence analysis.
Model Components
DNA Model (dna/)
- Type: Nucleotide Transformer for DNA sequences
- Context: 4096 tokens
- Training: Transcript-specific coding sequences
Protein Model (protein/)
- Type: ESM2 for protein sequences
- Variant: Large model
- Training: Corresponding protein sequences
Usage
from transformers import AutoModel, AutoTokenizer
# Load DNA model
dna_model = AutoModel.from_pretrained("vsubasri/joint-nt-esm2-transcript-coding", subfolder="dna")
dna_tokenizer = AutoTokenizer.from_pretrained("vsubasri/joint-nt-esm2-transcript-coding", subfolder="dna")
# Load protein model
protein_model = AutoModel.from_pretrained("vsubasri/joint-nt-esm2-transcript-coding", subfolder="protein")
protein_tokenizer = AutoTokenizer.from_pretrained("vsubasri/joint-nt-esm2-transcript-coding", subfolder="protein")
# Example joint usage
dna_seq = "ATGAAACGCATTAGCACCACCATTACCACCACCATCACCATTACCACAGGTAACGGTGCGGGCTGA"
protein_seq = "MKRISLHHHHHHHQVTVRWD"
dna_inputs = dna_tokenizer(dna_seq, return_tensors="pt")
protein_inputs = protein_tokenizer(protein_seq, return_tensors="pt")
dna_outputs = dna_model(**dna_inputs)
protein_outputs = protein_model(**protein_inputs)
Training Details
- Joint Training: Models trained together for cross-modal understanding
- Batch Size: 8
- Data: Transcript-specific coding sequences with corresponding proteins
- Architecture: Maintained original NT and ESM2 architectures
Repository Structure
├── dna/ # NT DNA model
│ ├── config.json
│ ├── model.safetensors
│ ├── tokenizer_config.json
│ ├── vocab.txt
│ └── special_tokens_map.json
├── protein/ # ESM2 protein model
│ ├── config.json
│ ├── model.safetensors
│ ├── tokenizer_config.json
│ ├── vocab.txt
│ └── special_tokens_map.json
└── joint_config.json # Joint model configuration
Citation
If you use these models, please cite the original NT and ESM2 papers along with your joint training methodology.