import streamlit as st
# Page configuration
st.set_page_config(
layout="wide",
initial_sidebar_state="auto"
)
# Custom CSS for better styling
st.markdown("""
""", unsafe_allow_html=True)
# Title
st.markdown('
Introduction to DistilBERT Annotators in Spark NLP
', unsafe_allow_html=True)
# Subtitle
st.markdown("""
Spark NLP provides a range of DistilBERT-based annotators designed for various natural language processing tasks. DistilBERT offers a more efficient and lightweight alternative to the original BERT model while maintaining competitive performance. Below, we provide an overview of the four key DistilBERT annotators:
""", unsafe_allow_html=True)
tab1, tab2, tab3, tab4 = st.tabs(["DistilBERT for Token Classification", "DistilBERT for Zero-Shot Classification", "DistilBERT for Sequence Classification", "DistilBERT for Question Answering"])
with tab1:
st.markdown("""
DistilBERT for Token Classification
The DistilBertForTokenClassification annotator is designed for Named Entity Recognition (NER) tasks using DistilBERT, a smaller and faster variant of BERT. This model efficiently handles token classification, which involves labeling tokens in a text with tags that correspond to specific entities. The DistilBERT model retains 97% of BERT's language understanding while being lighter and faster, making it suitable for real-time applications.
Token classification with DistilBERT enables:
- Named Entity Recognition (NER): Identifying and classifying entities such as names, organizations, locations, and other predefined categories.
- Information Extraction: Extracting key information from unstructured text for further analysis.
- Text Categorization: Enhancing document retrieval and categorization based on entity recognition.
Here is an example of how DistilBERT token classification works:
Entity |
Label |
Apple |
ORG |
Steve Jobs |
PER |
California |
LOC |
""", unsafe_allow_html=True)
# DistilBERT Token Classification - NER CoNLL
st.markdown('DistilBERT Token Classification - NER CoNLL
', unsafe_allow_html=True)
st.markdown("""
The distilbert_base_token_classifier_conll03 is a fine-tuned DistilBERT model for token classification tasks, specifically adapted for Named Entity Recognition (NER) on the CoNLL-03 dataset. It is designed to recognize four types of entities: location (LOC), organizations (ORG), person (PER), and Miscellaneous (MISC).
""", unsafe_allow_html=True)
# How to Use the Model - Token Classification
st.markdown('How to Use the Model
', unsafe_allow_html=True)
st.code('''
from sparknlp.base import *
from sparknlp.annotator import *
from pyspark.ml import Pipeline
from pyspark.sql.functions import col, expr
document_assembler = DocumentAssembler() \\
.setInputCol('text') \\
.setOutputCol('document')
tokenizer = Tokenizer() \\
.setInputCols(['document']) \\
.setOutputCol('token')
tokenClassifier = DistilBertForTokenClassification \\
.pretrained('distilbert_base_token_classifier_conll03', 'en') \\
.setInputCols(['token', 'document']) \\
.setOutputCol('ner') \\
.setCaseSensitive(True) \\
.setMaxSentenceLength(512)
# Convert NER labels to entities
ner_converter = NerConverter() \\
.setInputCols(['document', 'token', 'ner']) \\
.setOutputCol('entities')
pipeline = Pipeline(stages=[
document_assembler,
tokenizer,
tokenClassifier,
ner_converter
])
example = spark.createDataFrame([["""Apple Inc. is planning to open a new headquarters in Cupertino, California. The CEO, Tim Cook, announced this during the company's annual event on March 25th, 2023. Barack Obama, the 44th President of the United States, was born on August 4th, 1961, in Honolulu, Hawaii. He attended Harvard Law School and later became a community organizer in Chicago. Amazon reported a net revenue of $125.6 billion in Q4 of 2022, an increase of 9% compared to the previous year. Jeff Bezos, the founder of Amazon, mentioned that the company's growth in cloud computing has significantly contributed to this rise. Paris, the capital city of France, is renowned for its art, fashion, and culture. Key attractions include the Eiffel Tower, the Louvre Museum, and the Notre-Dame Cathedral. Visitors often enjoy a stroll along the Seine River and dining at local bistros. The study, conducted at the Mayo Clinic in Rochester, Minnesota, examined the effects of a new drug on patients with Type 2 diabetes. Results showed a significant reduction in blood sugar levels over a 12-month period. Serena Williams won her 24th Grand Slam title at the Wimbledon Championships in London, England. She defeated Naomi Osaka in a thrilling final match on July 13th, 2023. Google's latest smartphone, the Pixel 6, was unveiled at an event in New York City. Sundar Pichai, the CEO of Google, highlighted the phone's advanced AI capabilities and improved camera features. The Declaration of Independence was signed on July 4th, 1776, in Philadelphia, Pennsylvania. Thomas Jefferson, Benjamin Franklin, and John Adams were among the key figures who drafted this historic document."""]]).toDF("text")
result = pipeline.fit(example).transform(example)
result.select(
expr("explode(entities) as ner_chunk")
).select(
col("ner_chunk.result").alias("chunk"),
col("ner_chunk.metadata.entity").alias("ner_label")
).show(truncate=False)
''', language='python')
# Results
st.text("""
+--------------------+---------+
|chunk |ner_label|
+--------------------+---------+
|Apple Inc. |ORG |
|Cupertino |LOC |
|California |LOC |
|Tim Cook |PER |
|Barack Obama |PER |
|President |MISC |
|United States |LOC |
|Honolulu |LOC |
|Hawaii |LOC |
|Harvard Law School |ORG |
|Chicago |LOC |
|Amazon |ORG |
|Jeff Bezos |PER |
|Amazon |ORG |
|Paris |LOC |
|France |LOC |
|Eiffel Tower |LOC |
|Louvre Museum |LOC |
|Notre-Dame Cathedral|LOC |
|Seine River |LOC |
+--------------------+---------+
only showing top 20 rows
""")
# Performance Metrics
st.markdown('Performance Metrics
', unsafe_allow_html=True)
st.markdown("""
Here are the detailed performance metrics for the DistilBERT token classification model:
Entity |
Precision |
Recall |
F1-Score |
Support |
B-LOC |
0.93 |
0.85 |
0.89 |
1668 |
B-MISC |
0.77 |
0.78 |
0.78 |
702 |
B-ORG |
0.81 |
0.89 |
0.85 |
1661 |
B-PER |
0.95 |
0.93 |
0.94 |
1617 |
I-LOC |
0.80 |
0.76 |
0.78 |
257 |
I-MISC |
0.60 |
0.69 |
0.64 |
216 |
I-ORG |
0.80 |
0.92 |
0.86 |
835 |
I-PER |
0.98 |
0.98 |
0.98 |
1156 |
O |
0.99 |
0.99 |
0.99 |
38323 |
Overall |
0.97 |
0.97 |
0.97 |
46435 |
Additional metrics:
- Accuracy (non-O): 88.52%
- Accuracy: 97.24%
- Precision: 84.77%
- Recall: 86.12%
- F1-Score: 85.44
Detailed breakdown for each category:
- LOC: Precision: 91.36%, Recall: 84.29%, F1-Score: 87.68
- MISC: Precision: 70.60%, Recall: 75.93%, F1-Score: 73.16
- ORG: Precision: 77.29%, Recall: 86.27%, F1-Score: 81.54
- PER: Precision: 93.84%, Recall: 92.27%, F1-Score: 93.05
""", unsafe_allow_html=True)
# Model Information - Token Classification
st.markdown('Model Information
', unsafe_allow_html=True)
st.markdown("""
- Model Name: distilbert_base_token_classifier_conll03
- Compatibility: Spark NLP 3.2.0+
- License: Open Source
- Edition: Official
- Input Labels: [token, document]
- Output Labels: [ner]
- Language: English
- Size: 252 MB
- Case Sensitive: Yes
- Max Sentence Length: 512
""", unsafe_allow_html=True)
# References - Token Classification
st.markdown('References
', unsafe_allow_html=True)
st.markdown("""
""", unsafe_allow_html=True)
with tab2:
st.markdown("""
DistilBERT for Zero-Shot Text Classification
The DistilBertForZeroShotClassification annotator offers cutting-edge capabilities for zero-shot text classification, particularly tailored for English. This model utilizes the principles of natural language inference (NLI) to predict labels for text that it has not been explicitly trained on. This adaptability is invaluable for scenarios where predefined labels are either unavailable or may evolve over time.
Key Applications:
- Dynamic Content Tagging: Automatically categorize content without relying on a predefined set of labels, making it ideal for rapidly changing or expanding datasets.
- Sentiment and Topic Analysis: Evaluate sentiment and categorize topics on emerging trends or new content without needing to retrain the model, ensuring up-to-date analysis.
- Contextual Understanding: Adapt the model to understand and classify content based on current events, niche topics, or specialized domains.
This annotator is fine-tuned using the DistilBERT Base Uncased model, offering a balance between efficiency and scalability. Its zero-shot classification capability makes it an excellent choice for dynamic environments where data and categories are constantly evolving.
Text |
Predicted Category |
"I have a problem with my iPhone that needs to be resolved asap!!" |
Urgent |
"The weather today is perfect for a hike in the mountains." |
Weather |
"I just watched an amazing documentary about space exploration." |
Movie |
""", unsafe_allow_html=True)
# DistilBERT Zero-Shot Classification Base - MNLI
st.markdown('DistilBERT Zero-Shot Classification - MNLI Base
', unsafe_allow_html=True)
st.markdown("""
The distilbert_base_zero_shot_classifier_uncased_mnli model is fine-tuned on the MNLI (Multi-Genre Natural Language Inference) dataset, which is well-suited for zero-shot classification tasks. Built on the DistilBERT Base Uncased architecture, this model offers the flexibility to define and apply new labels at runtime, making it adaptable to a wide range of applications without the need for retraining.
Model Highlights:
- Runtime Label Definition: Unlike traditional models that require a fixed set of labels, this model allows users to specify candidate labels during inference, enabling real-time adaptation.
- Scalability: Optimized for performance in production environments, providing fast and scalable text classification.
- Fine-Tuning: Based on the robust MNLI dataset, ensuring high accuracy across various text genres and contexts.
""", unsafe_allow_html=True)
# How to Use the Model - Zero-Shot Classification
st.markdown('How to Use the Model
', unsafe_allow_html=True)
st.code('''
from sparknlp.base import *
from sparknlp.annotator import *
from pyspark.ml import Pipeline
# Document Assembler
document_assembler = DocumentAssembler() \\
.setInputCol('text') \\
.setOutputCol('document')
# Tokenizer
tokenizer = Tokenizer() \\
.setInputCols(['document']) \\
.setOutputCol('token')
# Zero-Shot Classifier
zeroShotClassifier = DistilBertForZeroShotClassification \\
.pretrained('distilbert_base_zero_shot_classifier_uncased_mnli', 'en') \\
.setInputCols(['token', 'document']) \\
.setOutputCol('class') \\
.setCaseSensitive(True) \\
.setMaxSentenceLength(512) \\
.setCandidateLabels(["urgent", "mobile", "travel", "movie", "music", "sport", "weather", "technology"])
# Pipeline Setup
pipeline = Pipeline(stages=[document_assembler, tokenizer, zeroShotClassifier])
# Sample Data for Testing
example = spark.createDataFrame([['I have a problem with my iPhone that needs to be resolved asap!!']]).toDF("text")
# Run the Pipeline
result = pipeline.fit(example).transform(example)
# Show Results
result.select('document.result', 'class.result').show(truncate=False)
''', language='python')
st.text("""
+------------------------------------------------------------------+-------+
|result |result |
+------------------------------------------------------------------+-------+
|[I have a problem with my iPhone that needs to be resolved asap!!]|[music]|
+------------------------------------------------------------------+-------+
""")
# Model Information - Zero-Shot Classification
st.markdown('Model Information
', unsafe_allow_html=True)
st.markdown("""
- Model Name: distilbert_base_zero_shot_classifier_uncased_mnli
- Compatibility: Spark NLP 4.4.1+
- License: Open Source
- Edition: Official
- Input Labels: [token, document]
- Output Labels: [multi_class]
- Language: English (en)
- Model Size: 249.7 MB
""", unsafe_allow_html=True)
# References and Further Reading - Zero-Shot Classification
st.markdown('References and Further Reading
', unsafe_allow_html=True)
st.markdown("""
""", unsafe_allow_html=True)
with tab3:
st.markdown("""
DistilBERT for Emotion Detection and Sequence Classification
The DistilBertForSequenceClassification annotator leverages a fine-tuned version of the DistilBERT model, specifically trained to classify text sequences into predefined categories. This model, distilbert_base_uncased_finetuned_emotion_yoahqiu, is designed for emotion detection in English text, making it a powerful tool for analyzing the emotional tone in various types of content.
This model was originally developed by yoahqiu and adapted from Hugging Face for production environments using Spark NLP. It offers a lightweight yet efficient alternative to BERT, maintaining strong performance while being optimized for faster inference.
Applications:
- Emotion Detection: Automatically identifies and categorizes emotions such as joy, sadness, anger, and surprise from textual data.
- Sentiment Analysis: Determines the overall sentiment (positive, negative, or neutral) expressed in the text, making it useful for customer feedback analysis, social media monitoring, and more.
- Content Personalization: Enhances user experiences by tailoring content based on detected emotions, improving engagement and satisfaction.
- Market Research: Analyzes consumer sentiment and emotional responses to products, services, and campaigns.
By incorporating this model into your text analytics workflow, you can unlock deeper insights into customer emotions and sentiments, enabling more informed decision-making and more effective communication strategies.
""", unsafe_allow_html=True)
# How to Use the Model - Sequence Classification
st.markdown('How to Use the Model
', unsafe_allow_html=True)
st.code('''
from sparknlp.base import *
from sparknlp.annotator import *
from pyspark.ml import Pipeline
# Document Assembler
document_assembler = DocumentAssembler() \\
.setInputCol("text") \\
.setOutputCol("document")
# Tokenizer
tokenizer = Tokenizer() \\
.setInputCols(["document"]) \\
.setOutputCol("token")
# Sequence Classifier
sequenceClassifier = DistilBertForSequenceClassification.pretrained("distilbert_base_uncased_finetuned_emotion_yoahqiu", "en") \\
.setInputCols(["document", "token"]) \\
.setOutputCol("class")
# Pipeline
pipeline = Pipeline().setStages([document_assembler, tokenizer, sequenceClassifier])
# Apply the Pipeline
result = pipeline.fit(data).transform(data)
# Show the Result
result.select("document.result", "class.result").show(truncate=False)
''', language='python')
st.text("""
+------------------------------------------------------------------------------------------------------------------+------+
|result |result|
+------------------------------------------------------------------------------------------------------------------+------+
|[I had a fantastic day at the park with my friends and family, enjoying the beautiful weather and fun activities.]|[joy] |
+------------------------------------------------------------------------------------------------------------------+------+
""")
# Model Information - Sequence Classification
st.markdown('Model Information
', unsafe_allow_html=True)
st.markdown("""
- Model Name: distilbert_base_uncased_finetuned_emotion_yoahqiu
- Compatibility: Spark NLP 5.2.2+
- License: Open Source
- Edition: Official
- Input Labels: [documents, token]
- Output Labels: [class]
- Language: English (en)
- Model Size: 249.5 MB
- Training Data: Fine-tuned on a dataset labeled for various emotions, ensuring robust performance across diverse text inputs.
- Use Case Examples: Sentiment analysis for product reviews, emotional tone detection in social media posts, and more.
- Case Sensitivity: The model is case insensitive, allowing it to handle various text formats effectively.
- Max Sentence Length: Capable of processing sequences up to 512 tokens in length, covering most typical use cases.
""", unsafe_allow_html=True)
# References and Further Reading
st.markdown('References and Further Reading
', unsafe_allow_html=True)
st.markdown("""
""", unsafe_allow_html=True)
with tab4:
st.markdown("""
DistilBERT for Question Answering
The DistilBertForQuestionAnswering model is a state-of-the-art tool for extracting precise answers from text passages based on a given question. This model, based on the distilbert-base-cased-distilled-squad architecture, was originally developed by Hugging Face and is fine-tuned for high performance and scalability using Spark NLP.
This model is highly effective for:
- Information Extraction: Identifying exact spans of text that answer specific questions.
- Automated Customer Support: Enhancing chatbots and support systems by accurately retrieving information from documents.
- Educational Tools: Assisting in creating intelligent systems that can answer questions based on educational materials.
Its capabilities make it an essential tool for applications requiring precise information retrieval from large corpora of text.
""", unsafe_allow_html=True)
# Predicted Entities
st.markdown('Predicted Entities
', unsafe_allow_html=True)
st.markdown("""
The model provides answers by identifying the relevant span of text in the context that best responds to the provided question.
Question |
Context |
Predicted Answer |
What is my name? |
My name is Clara and I live in Berkeley. |
Clara |
Where do I live? |
My name is Clara and I live in Berkeley. |
Berkeley |
What is the capital of France? |
The capital of France is Paris, a beautiful city known for its culture and landmarks. |
Paris |
""", unsafe_allow_html=True)
# How to Use the Model - Question Answering
st.markdown('How to Use the Model
', unsafe_allow_html=True)
st.code('''
from sparknlp.base import *
from sparknlp.annotator import *
from pyspark.ml import Pipeline
# Document Assembler for Questions and Contexts
documentAssembler = MultiDocumentAssembler() \\
.setInputCols(["question", "context"]) \\
.setOutputCols(["document_question", "document_context"])
# DistilBERT Question Answering Model
spanClassifier = DistilBertForQuestionAnswering.pretrained("distilbert_base_cased_qa_squad2", "en") \\
.setInputCols(["document_question", "document_context"]) \\
.setOutputCol("answer") \\
.setCaseSensitive(True)
# Building the Pipeline
pipeline = Pipeline(stages=[documentAssembler, spanClassifier])
# Sample Data
data = spark.createDataFrame([["What is my name?", "My name is Clara and I live in Berkeley."]]).toDF("question", "context")
# Applying the Pipeline
result = pipeline.fit(data).transform(data)
# Showing Results
result.select('document_question.result', 'document_context.result', 'answer.result').show(truncate=False)
''', language='python')
st.text("""
+------------------+------------------------------------------+-------+
|result |result |result |
+------------------+------------------------------------------+-------+
|[What is my name?]|[My name is Clara and I live in Berkeley.]|[Clara]|
+------------------+------------------------------------------+-------+
""")
# Model Information - Question Answering
st.markdown('Model Information
', unsafe_allow_html=True)
st.markdown("""
- Model Name: distilbert_base_cased_qa_squad2
- Compatibility: Spark NLP 5.2.0+
- License: Open Source
- Edition: Official
- Input Labels: [document_question, document_context]
- Output Labels: [answer]
- Language: English (en)
- Model Size: 243.8 MB
""", unsafe_allow_html=True)
# References and Further Reading - Question Answering
st.markdown('References and Further Reading
', unsafe_allow_html=True)
st.markdown("""
""", unsafe_allow_html=True)