Spaces:
Runtime error
Runtime error
File size: 5,893 Bytes
9f7f573 b24d167 9f7f573 802f11a 9f7f573 90966f7 b24d167 9f7f573 b24d167 802f11a b24d167 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 |
import pandas as pd
from datasets import load_dataset
import streamlit as st
from clarin_datasets.dataset_to_show import DatasetToShow
class PunctuationRestorationDataset(DatasetToShow):
def __init__(self):
DatasetToShow.__init__(self)
self.data_dict_named = None
self.dataset_name = "clarin-pl/2021-punctuation-restoration"
self.description = [
"""
Speech transcripts generated by Automatic Speech Recognition (ASR) systems typically do
not contain any punctuation or capitalization. In longer stretches of automatically recognized speech,
the lack of punctuation affects the general clarity of the output text [1]. The primary purpose of
punctuation (PR) and capitalization restoration (CR) as a distinct natural language processing (NLP) task is
to improve the legibility of ASR-generated text, and possibly other types of texts without punctuation. Aside
from their intrinsic value, PR and CR may improve the performance of other NLP aspects such as Named Entity
Recognition (NER), part-of-speech (POS) and semantic parsing or spoken dialog segmentation [2, 3]. As useful
as it seems, It is hard to systematically evaluate PR on transcripts of conversational language; mainly
because punctuation rules can be ambiguous even for originally written texts, and the very nature of
naturally-occurring spoken language makes it difficult to identify clear phrase and sentence boundaries [4,
5]. Given these requirements and limitations, a PR task based on a redistributable corpus of read speech was
suggested. 1200 texts included in this collection (totaling over 240,000 words) were selected from two
distinct sources: WikiNews and WikiTalks. Punctuation found in these sources should be approached with some
reservation when used for evaluation: these are original texts and may contain some user-induced errors and
bias. The texts were read out by over a hundred different speakers. Original texts with punctuation were
forced-aligned with recordings and used as the ideal ASR output. The goal of the task is to provide a
solution for restoring punctuation in the test set collated for this task. The test set consists of
time-aligned ASR transcriptions of read texts from the two sources. Participants are encouraged to use both
text-based and speech-derived features to identify punctuation symbols (e.g. multimodal framework [6]). In
addition, the train set is accompanied by reference text corpora of WikiNews and WikiTalks data that can be
used in training and fine-tuning punctuation models.
""",
"Task description",
"The purpose of this task is to restore punctuation in the ASR recognition of texts read out loud.",
"clarin_datasets/punctuation_restoration_task.png"
]
def load_data(self):
raw_dataset = load_dataset(self.dataset_name)
self.data_dict = {
subset: raw_dataset[subset].to_pandas() for subset in self.subsets
}
self.data_dict_named = {}
for subset in self.subsets:
references = raw_dataset[subset]["tags"]
references_named = [
[
raw_dataset[subset].features["tags"].feature.names[label]
for label in labels
]
for labels in references
]
self.data_dict_named[subset] = pd.DataFrame(
{
"tokens": self.data_dict[subset]["tokens"],
"tags": references_named,
}
)
def show_dataset(self):
header = st.container()
description = st.container()
dataframe_head = st.container()
class_distribution = st.container()
with header:
st.title(self.dataset_name)
with description:
st.header("Dataset description")
st.write(self.description[0])
st.subheader(self.description[1])
st.write(self.description[2])
st.image(self.description[3])
full_dataframe = pd.concat(self.data_dict.values(), axis="rows")
with dataframe_head:
st.header("First 10 observations of the chosen subset")
subset_to_show = st.selectbox(label="Select subset to see", options=self.subsets)
df_to_show = self.data_dict[subset_to_show].head(10)
st.dataframe(df_to_show)
st.text_area(label="LaTeX code", value=df_to_show.style.to_latex())
class_distribution_dict = {}
for subset in self.subsets:
all_labels_from_subset = self.data_dict_named[subset]["tags"].tolist()
all_labels_from_subset = [
x
for subarray in all_labels_from_subset
for x in subarray
if x != "O"
]
all_labels_from_subset = pd.Series(all_labels_from_subset)
class_distribution_dict[subset] = (
all_labels_from_subset.value_counts(normalize=True)
.sort_index()
.reset_index()
.rename({"index": "class", 0: subset}, axis="columns")
)
class_distribution_df = pd.merge(
class_distribution_dict["train"],
class_distribution_dict["test"],
on="class",
)
with class_distribution:
st.header("Class distribution in each subset (without 'O')")
st.dataframe(class_distribution_df)
st.text_area(
label="LaTeX code", value=class_distribution_df.style.to_latex()
)
|