chunked/nltk_chunking/_model_summary/chunk_45.txt

DistilBERT uses knowledge distillation - a compression technique - to create a smaller version of BERT while keeping nearly all of its language understanding capabilities.