import streamlit as st # Page configuration st.set_page_config( layout="wide", initial_sidebar_state="auto" ) # Custom CSS for better styling st.markdown(""" """, unsafe_allow_html=True) # Title st.markdown('
Introduction to DistilBERT Annotators in Spark NLP
', unsafe_allow_html=True) # Subtitle st.markdown("""

Spark NLP provides a range of DistilBERT-based annotators designed for various natural language processing tasks. DistilBERT offers a more efficient and lightweight alternative to the original BERT model while maintaining competitive performance. Below, we provide an overview of the four key DistilBERT annotators:

""", unsafe_allow_html=True) tab1, tab2, tab3, tab4 = st.tabs(["DistilBERT for Token Classification", "DistilBERT for Zero-Shot Classification", "DistilBERT for Sequence Classification", "DistilBERT for Question Answering"]) with tab1: st.markdown("""

DistilBERT for Token Classification

The DistilBertForTokenClassification annotator is designed for Named Entity Recognition (NER) tasks using DistilBERT, a smaller and faster variant of BERT. This model efficiently handles token classification, which involves labeling tokens in a text with tags that correspond to specific entities. The DistilBERT model retains 97% of BERT's language understanding while being lighter and faster, making it suitable for real-time applications.

Token classification with DistilBERT enables:

Here is an example of how DistilBERT token classification works:

Entity Label
Apple ORG
Steve Jobs PER
California LOC
""", unsafe_allow_html=True) # DistilBERT Token Classification - NER CoNLL st.markdown('
DistilBERT Token Classification - NER CoNLL
', unsafe_allow_html=True) st.markdown("""

The distilbert_base_token_classifier_conll03 is a fine-tuned DistilBERT model for token classification tasks, specifically adapted for Named Entity Recognition (NER) on the CoNLL-03 dataset. It is designed to recognize four types of entities: location (LOC), organizations (ORG), person (PER), and Miscellaneous (MISC).

""", unsafe_allow_html=True) # How to Use the Model - Token Classification st.markdown('
How to Use the Model
', unsafe_allow_html=True) st.code(''' from sparknlp.base import * from sparknlp.annotator import * from pyspark.ml import Pipeline from pyspark.sql.functions import col, expr document_assembler = DocumentAssembler() \\ .setInputCol('text') \\ .setOutputCol('document') tokenizer = Tokenizer() \\ .setInputCols(['document']) \\ .setOutputCol('token') tokenClassifier = DistilBertForTokenClassification \\ .pretrained('distilbert_base_token_classifier_conll03', 'en') \\ .setInputCols(['token', 'document']) \\ .setOutputCol('ner') \\ .setCaseSensitive(True) \\ .setMaxSentenceLength(512) # Convert NER labels to entities ner_converter = NerConverter() \\ .setInputCols(['document', 'token', 'ner']) \\ .setOutputCol('entities') pipeline = Pipeline(stages=[ document_assembler, tokenizer, tokenClassifier, ner_converter ]) example = spark.createDataFrame([["""Apple Inc. is planning to open a new headquarters in Cupertino, California. The CEO, Tim Cook, announced this during the company's annual event on March 25th, 2023. Barack Obama, the 44th President of the United States, was born on August 4th, 1961, in Honolulu, Hawaii. He attended Harvard Law School and later became a community organizer in Chicago. Amazon reported a net revenue of $125.6 billion in Q4 of 2022, an increase of 9% compared to the previous year. Jeff Bezos, the founder of Amazon, mentioned that the company's growth in cloud computing has significantly contributed to this rise. Paris, the capital city of France, is renowned for its art, fashion, and culture. Key attractions include the Eiffel Tower, the Louvre Museum, and the Notre-Dame Cathedral. Visitors often enjoy a stroll along the Seine River and dining at local bistros. The study, conducted at the Mayo Clinic in Rochester, Minnesota, examined the effects of a new drug on patients with Type 2 diabetes. Results showed a significant reduction in blood sugar levels over a 12-month period. Serena Williams won her 24th Grand Slam title at the Wimbledon Championships in London, England. She defeated Naomi Osaka in a thrilling final match on July 13th, 2023. Google's latest smartphone, the Pixel 6, was unveiled at an event in New York City. Sundar Pichai, the CEO of Google, highlighted the phone's advanced AI capabilities and improved camera features. The Declaration of Independence was signed on July 4th, 1776, in Philadelphia, Pennsylvania. Thomas Jefferson, Benjamin Franklin, and John Adams were among the key figures who drafted this historic document."""]]).toDF("text") result = pipeline.fit(example).transform(example) result.select( expr("explode(entities) as ner_chunk") ).select( col("ner_chunk.result").alias("chunk"), col("ner_chunk.metadata.entity").alias("ner_label") ).show(truncate=False) ''', language='python') # Results st.text(""" +--------------------+---------+ |chunk |ner_label| +--------------------+---------+ |Apple Inc. |ORG | |Cupertino |LOC | |California |LOC | |Tim Cook |PER | |Barack Obama |PER | |President |MISC | |United States |LOC | |Honolulu |LOC | |Hawaii |LOC | |Harvard Law School |ORG | |Chicago |LOC | |Amazon |ORG | |Jeff Bezos |PER | |Amazon |ORG | |Paris |LOC | |France |LOC | |Eiffel Tower |LOC | |Louvre Museum |LOC | |Notre-Dame Cathedral|LOC | |Seine River |LOC | +--------------------+---------+ only showing top 20 rows """) # Performance Metrics st.markdown('
Performance Metrics
', unsafe_allow_html=True) st.markdown("""

Here are the detailed performance metrics for the DistilBERT token classification model:

Entity Precision Recall F1-Score Support
B-LOC 0.93 0.85 0.89 1668
B-MISC 0.77 0.78 0.78 702
B-ORG 0.81 0.89 0.85 1661
B-PER 0.95 0.93 0.94 1617
I-LOC 0.80 0.76 0.78 257
I-MISC 0.60 0.69 0.64 216
I-ORG 0.80 0.92 0.86 835
I-PER 0.98 0.98 0.98 1156
O 0.99 0.99 0.99 38323
Overall 0.97 0.97 0.97 46435

Additional metrics:

Detailed breakdown for each category:

""", unsafe_allow_html=True) # Model Information - Token Classification st.markdown('
Model Information
', unsafe_allow_html=True) st.markdown("""
""", unsafe_allow_html=True) # References - Token Classification st.markdown('
References
', unsafe_allow_html=True) st.markdown("""
""", unsafe_allow_html=True) with tab2: st.markdown("""

DistilBERT for Zero-Shot Text Classification

The DistilBertForZeroShotClassification annotator offers cutting-edge capabilities for zero-shot text classification, particularly tailored for English. This model utilizes the principles of natural language inference (NLI) to predict labels for text that it has not been explicitly trained on. This adaptability is invaluable for scenarios where predefined labels are either unavailable or may evolve over time.

Key Applications:

This annotator is fine-tuned using the DistilBERT Base Uncased model, offering a balance between efficiency and scalability. Its zero-shot classification capability makes it an excellent choice for dynamic environments where data and categories are constantly evolving.

Text Predicted Category
"I have a problem with my iPhone that needs to be resolved asap!!" Urgent
"The weather today is perfect for a hike in the mountains." Weather
"I just watched an amazing documentary about space exploration." Movie
""", unsafe_allow_html=True) # DistilBERT Zero-Shot Classification Base - MNLI st.markdown('
DistilBERT Zero-Shot Classification - MNLI Base
', unsafe_allow_html=True) st.markdown("""

The distilbert_base_zero_shot_classifier_uncased_mnli model is fine-tuned on the MNLI (Multi-Genre Natural Language Inference) dataset, which is well-suited for zero-shot classification tasks. Built on the DistilBERT Base Uncased architecture, this model offers the flexibility to define and apply new labels at runtime, making it adaptable to a wide range of applications without the need for retraining.

Model Highlights:

""", unsafe_allow_html=True) # How to Use the Model - Zero-Shot Classification st.markdown('
How to Use the Model
', unsafe_allow_html=True) st.code(''' from sparknlp.base import * from sparknlp.annotator import * from pyspark.ml import Pipeline # Document Assembler document_assembler = DocumentAssembler() \\ .setInputCol('text') \\ .setOutputCol('document') # Tokenizer tokenizer = Tokenizer() \\ .setInputCols(['document']) \\ .setOutputCol('token') # Zero-Shot Classifier zeroShotClassifier = DistilBertForZeroShotClassification \\ .pretrained('distilbert_base_zero_shot_classifier_uncased_mnli', 'en') \\ .setInputCols(['token', 'document']) \\ .setOutputCol('class') \\ .setCaseSensitive(True) \\ .setMaxSentenceLength(512) \\ .setCandidateLabels(["urgent", "mobile", "travel", "movie", "music", "sport", "weather", "technology"]) # Pipeline Setup pipeline = Pipeline(stages=[document_assembler, tokenizer, zeroShotClassifier]) # Sample Data for Testing example = spark.createDataFrame([['I have a problem with my iPhone that needs to be resolved asap!!']]).toDF("text") # Run the Pipeline result = pipeline.fit(example).transform(example) # Show Results result.select('document.result', 'class.result').show(truncate=False) ''', language='python') st.text(""" +------------------------------------------------------------------+-------+ |result |result | +------------------------------------------------------------------+-------+ |[I have a problem with my iPhone that needs to be resolved asap!!]|[music]| +------------------------------------------------------------------+-------+ """) # Model Information - Zero-Shot Classification st.markdown('
Model Information
', unsafe_allow_html=True) st.markdown("""
""", unsafe_allow_html=True) # References and Further Reading - Zero-Shot Classification st.markdown('
References and Further Reading
', unsafe_allow_html=True) st.markdown("""
""", unsafe_allow_html=True) with tab3: st.markdown("""

DistilBERT for Emotion Detection and Sequence Classification

The DistilBertForSequenceClassification annotator leverages a fine-tuned version of the DistilBERT model, specifically trained to classify text sequences into predefined categories. This model, distilbert_base_uncased_finetuned_emotion_yoahqiu, is designed for emotion detection in English text, making it a powerful tool for analyzing the emotional tone in various types of content.

This model was originally developed by yoahqiu and adapted from Hugging Face for production environments using Spark NLP. It offers a lightweight yet efficient alternative to BERT, maintaining strong performance while being optimized for faster inference.

Applications:

By incorporating this model into your text analytics workflow, you can unlock deeper insights into customer emotions and sentiments, enabling more informed decision-making and more effective communication strategies.

""", unsafe_allow_html=True) # How to Use the Model - Sequence Classification st.markdown('
How to Use the Model
', unsafe_allow_html=True) st.code(''' from sparknlp.base import * from sparknlp.annotator import * from pyspark.ml import Pipeline # Document Assembler document_assembler = DocumentAssembler() \\ .setInputCol("text") \\ .setOutputCol("document") # Tokenizer tokenizer = Tokenizer() \\ .setInputCols(["document"]) \\ .setOutputCol("token") # Sequence Classifier sequenceClassifier = DistilBertForSequenceClassification.pretrained("distilbert_base_uncased_finetuned_emotion_yoahqiu", "en") \\ .setInputCols(["document", "token"]) \\ .setOutputCol("class") # Pipeline pipeline = Pipeline().setStages([document_assembler, tokenizer, sequenceClassifier]) # Apply the Pipeline result = pipeline.fit(data).transform(data) # Show the Result result.select("document.result", "class.result").show(truncate=False) ''', language='python') st.text(""" +------------------------------------------------------------------------------------------------------------------+------+ |result |result| +------------------------------------------------------------------------------------------------------------------+------+ |[I had a fantastic day at the park with my friends and family, enjoying the beautiful weather and fun activities.]|[joy] | +------------------------------------------------------------------------------------------------------------------+------+ """) # Model Information - Sequence Classification st.markdown('
Model Information
', unsafe_allow_html=True) st.markdown("""
""", unsafe_allow_html=True) # References and Further Reading st.markdown('
References and Further Reading
', unsafe_allow_html=True) st.markdown("""
""", unsafe_allow_html=True) with tab4: st.markdown("""

DistilBERT for Question Answering

The DistilBertForQuestionAnswering model is a state-of-the-art tool for extracting precise answers from text passages based on a given question. This model, based on the distilbert-base-cased-distilled-squad architecture, was originally developed by Hugging Face and is fine-tuned for high performance and scalability using Spark NLP.

This model is highly effective for:

Its capabilities make it an essential tool for applications requiring precise information retrieval from large corpora of text.

""", unsafe_allow_html=True) # Predicted Entities st.markdown('
Predicted Entities
', unsafe_allow_html=True) st.markdown("""

The model provides answers by identifying the relevant span of text in the context that best responds to the provided question.

Question Context Predicted Answer
What is my name? My name is Clara and I live in Berkeley. Clara
Where do I live? My name is Clara and I live in Berkeley. Berkeley
What is the capital of France? The capital of France is Paris, a beautiful city known for its culture and landmarks. Paris
""", unsafe_allow_html=True) # How to Use the Model - Question Answering st.markdown('
How to Use the Model
', unsafe_allow_html=True) st.code(''' from sparknlp.base import * from sparknlp.annotator import * from pyspark.ml import Pipeline # Document Assembler for Questions and Contexts documentAssembler = MultiDocumentAssembler() \\ .setInputCols(["question", "context"]) \\ .setOutputCols(["document_question", "document_context"]) # DistilBERT Question Answering Model spanClassifier = DistilBertForQuestionAnswering.pretrained("distilbert_base_cased_qa_squad2", "en") \\ .setInputCols(["document_question", "document_context"]) \\ .setOutputCol("answer") \\ .setCaseSensitive(True) # Building the Pipeline pipeline = Pipeline(stages=[documentAssembler, spanClassifier]) # Sample Data data = spark.createDataFrame([["What is my name?", "My name is Clara and I live in Berkeley."]]).toDF("question", "context") # Applying the Pipeline result = pipeline.fit(data).transform(data) # Showing Results result.select('document_question.result', 'document_context.result', 'answer.result').show(truncate=False) ''', language='python') st.text(""" +------------------+------------------------------------------+-------+ |result |result |result | +------------------+------------------------------------------+-------+ |[What is my name?]|[My name is Clara and I live in Berkeley.]|[Clara]| +------------------+------------------------------------------+-------+ """) # Model Information - Question Answering st.markdown('
Model Information
', unsafe_allow_html=True) st.markdown("""
""", unsafe_allow_html=True) # References and Further Reading - Question Answering st.markdown('
References and Further Reading
', unsafe_allow_html=True) st.markdown("""
""", unsafe_allow_html=True)