text_extensions_for_pandas/spanner/extract.py
def load_dict(file_name: str, tokenizer: "spacy.tokenizer.Tokenizer" = None):
    """
    Load a SystemT-format dictionary file. File format is one entry per line.

    Tokenizes and normalizes the dictionary entries.

    :param file_name: Path to dictionary file

    :param tokenizer: Preconfigured tokenizer object for tokenizing
    dictionary entries.  **Must be the same configuration as the tokenizer
    used on the target text!**  If None, this method will use SpaCy's default
    English tokenizer.

    :return: a `pd.DataFrame` with the normalized, tokenized dictionary entries.
    """
    with open(file_name, "r") as f:
        lines = [
            line.strip()
            for line in f.readlines()
            if len(line.strip()) > 0 and line[0] != "#"
        ]

    return create_dict(lines, tokenizer)

    Load a SystemT-format dictionary file. File format is one entry per line.

    Tokenizes and normalizes the dictionary entries.

    :param file_name: Path to dictionary file

    :param tokenizer: Preconfigured tokenizer object for tokenizing
    dictionary entries.  **Must be the same configuration as the tokenizer
    used on the target text!**  If None, this method will use SpaCy's default
    English tokenizer.

    :return: a `pd.DataFrame` with the normalized, tokenized dictionary entries.
    
Load a SystemT-format dictionary file. File format is one entry per line.
Tokenizes and normalizes the dictionary entries.
Load a SystemT - format dictionary file . File format is one entry per line . Tokenizes and normalizes the dictionary entries .
def load_dict(file_name: str, tokenizer: "spacy.tokenizer.Tokenizer" = None):
    with open(file_name, "r") as f:
        lines = [
            line.strip()
            for line in f.readlines()
            if len(line.strip()) > 0 and line[0] != "#"
        ]
    return create_dict(lines, tokenizer)
def load_dict ( file_name : str , tokenizer : "spacy.tokenizer.Tokenizer" = None ) : with open ( file_name , "r" ) as f : lines = [ line . strip ( ) for line in f . readlines ( ) if len ( line . strip ( ) ) > 0 and line [ 0 ] != "#" ] return create_dict ( lines , tokenizer )
Load a SystemT-format dictionary file.
Load a SystemT - format dictionary file .
"""
    Load a SystemT-format dictionary file. File format is one entry per line.

    Tokenizes and normalizes the dictionary entries.

    :param file_name: Path to dictionary file

    :param tokenizer: Preconfigured tokenizer object for tokenizing
    dictionary entries.  **Must be the same configuration as the tokenizer
    used on the target text!**  If None, this method will use SpaCy's default
    English tokenizer.

    :return: a `pd.DataFrame` with the normalized, tokenized dictionary entries.
    """
a `pd.DataFrame` with the normalized, tokenized dictionary entries.
a ` pd . DataFrame ` with the normalized tokenized dictionary entries .
file_name
str
Path to dictionary file
Path to dictionary file
tokenizer
"spacy.tokenizer.Tokenizer"
Preconfigured tokenizer object for tokenizing
dictionary entries.  **Must be the same configuration as the tokenizer
used on the target text!**  If None, this method will use SpaCy's default
English tokenizer.
Preconfigured tokenizer object for tokenizing dictionary entries . ** Must be the same configuration as the tokenizer used on the target text! ** If None this method will use SpaCy ' s default English tokenizer .