text_extensions_for_pandas/io/spacy.py
def token_features_to_tree(
    token_features: pd.DataFrame,
    text_col: str = "span",
    tag_col: str = "tag",
    label_col: str = "dep",
    head_col: str = "head",
):
    """
    Convert a DataFrame in the format returned by `make_tokens_and_features()`
    to the public input format of displaCy's dependency tree renderer.

    :param token_features: A subset of a token features DataFrame in the format
        returned by `make_tokens_and_features()`. Must at a minimum contain the
        `head` column and an integer index that corresponds to the ints
        in the `head` column.
    :param text_col: Name of the column in `token_features` from which the
        'covered text' label for each node of the parse tree should be extracted,
        or `None` to leave those labels blank.
    :param tag_col: Name of the column in `token_features` from which the
        'tag' label for each node of the parse tree should be extracted; or `None`
        to leave those labels blank.
    :param label_col: Name of the column in `token_features` from which the
        label for each edge of the parse tree should be extracted; or `None`
        to leave those labels blank.
    :param head_col: Name of the column in `token_features` from which the
        head node of each parse tree node should be extracted.

    :returns: Native Python type representation of the parse tree in a format
        suitable to pass to ``displacy.render(manual=True ...)``
        See https://spacy.io/usage/visualizers for the specification of this format.
    """

    # displaCy expects most inputs as strings. Centralize this conversion.
    def _get_text(col_name):
        if col_name is None:
            return np.zeros(shape=len(token_features.index), dtype=str)
        series = token_features[col_name]
        if isinstance(series.dtype, (SpanDtype, TokenSpanDtype)):
            return series.values.covered_text
        else:
            return series.astype(str)

    # Renumber the head column to a dense range starting from zero
    tok_map = {token_features.index[i]: i for i in range(len(token_features.index))}
    # Note that we turn any links to tokens not in our input rows into
    # self-links, which will get removed later on.
    head_tok = token_features[head_col].values
    remapped_head_tok = []
    for i in range(len(token_features.index)):
        remapped_head_tok.append(tok_map[head_tok[i]] if head_tok[i] in tok_map else i)

    words_df = pd.DataFrame({"text": _get_text(text_col), "tag": _get_text(tag_col)})
    edges_df = pd.DataFrame(
        {
            "from": range(len(token_features.index)),
            "to": remapped_head_tok,
            "label": _get_text(label_col),
        }
    )
    # displaCy requires all arcs to have their start and end be in
    # numeric order. An additional attribute "dir" tells which way
    # (left or right) each arc goes.
    arcs_df = pd.DataFrame(
        {
            "start": edges_df[["from", "to"]].min(axis=1),
            "end": edges_df[["from", "to"]].max(axis=1),
            "label": edges_df["label"],
            "dir": "left",
        }
    )
    arcs_df["dir"].mask(edges_df["from"] > edges_df["to"], "right", inplace=True)

    # Don't render self-links
    arcs_df = arcs_df[arcs_df["start"] != arcs_df["end"]]

    return {
        "words": words_df.to_dict(orient="records"),
        "arcs": arcs_df.to_dict(orient="records"),
    }

    Convert a DataFrame in the format returned by `make_tokens_and_features()`
    to the public input format of displaCy's dependency tree renderer.

    :param token_features: A subset of a token features DataFrame in the format
        returned by `make_tokens_and_features()`. Must at a minimum contain the
        `head` column and an integer index that corresponds to the ints
        in the `head` column.
    :param text_col: Name of the column in `token_features` from which the
        'covered text' label for each node of the parse tree should be extracted,
        or `None` to leave those labels blank.
    :param tag_col: Name of the column in `token_features` from which the
        'tag' label for each node of the parse tree should be extracted; or `None`
        to leave those labels blank.
    :param label_col: Name of the column in `token_features` from which the
        label for each edge of the parse tree should be extracted; or `None`
        to leave those labels blank.
    :param head_col: Name of the column in `token_features` from which the
        head node of each parse tree node should be extracted.

    :returns: Native Python type representation of the parse tree in a format
        suitable to pass to ``displacy.render(manual=True ...)``
        See https://spacy.io/usage/visualizers for the specification of this format.
    
Convert a DataFrame in the format returned by `make_tokens_and_features()`
to the public input format of displaCy's dependency tree renderer.
Convert a DataFrame in the format returned by ` make_tokens_and_features () ` to the public input format of displaCy ' s dependency tree renderer .
def token_features_to_tree(
    token_features: pd.DataFrame,
    text_col: str = "span",
    tag_col: str = "tag",
    label_col: str = "dep",
    head_col: str = "head",
):
    def _get_text(col_name):
        if col_name is None:
            return np.zeros(shape=len(token_features.index), dtype=str)
        series = token_features[col_name]
        if isinstance(series.dtype, (SpanDtype, TokenSpanDtype)):
            return series.values.covered_text
        else:
            return series.astype(str)
    tok_map = {token_features.index[i]: i for i in range(len(token_features.index))}
    head_tok = token_features[head_col].values
    remapped_head_tok = []
    for i in range(len(token_features.index)):
        remapped_head_tok.append(tok_map[head_tok[i]] if head_tok[i] in tok_map else i)
    words_df = pd.DataFrame({"text": _get_text(text_col), "tag": _get_text(tag_col)})
    edges_df = pd.DataFrame(
        {
            "from": range(len(token_features.index)),
            "to": remapped_head_tok,
            "label": _get_text(label_col),
        }
    )
    arcs_df = pd.DataFrame(
        {
            "start": edges_df[["from", "to"]].min(axis=1),
            "end": edges_df[["from", "to"]].max(axis=1),
            "label": edges_df["label"],
            "dir": "left",
        }
    )
    arcs_df["dir"].mask(edges_df["from"] > edges_df["to"], "right", inplace=True)
    arcs_df = arcs_df[arcs_df["start"] != arcs_df["end"]]
    return {
        "words": words_df.to_dict(orient="records"),
        "arcs": arcs_df.to_dict(orient="records"),
    }
def token_features_to_tree ( token_features : pd . DataFrame , text_col : str = "span" , tag_col : str = "tag" , label_col : str = "dep" , head_col : str = "head" , ) : def _get_text ( col_name ) : if col_name is None : return np . zeros ( shape = len ( token_features . index ) , dtype = str ) series = token_features [ col_name ] if isinstance ( series . dtype , ( SpanDtype , TokenSpanDtype ) ) : return series . values . covered_text else : return series . astype ( str ) tok_map = { token_features . index [ i ] : i for i in range ( len ( token_features . index ) ) } head_tok = token_features [ head_col ] . values remapped_head_tok = [ ] for i in range ( len ( token_features . index ) ) : remapped_head_tok . append ( tok_map [ head_tok [ i ] ] if head_tok [ i ] in tok_map else i ) words_df = pd . DataFrame ( { "text" : _get_text ( text_col ) , "tag" : _get_text ( tag_col ) } ) edges_df = pd . DataFrame ( { "from" : range ( len ( token_features . index ) ) , "to" : remapped_head_tok , "label" : _get_text ( label_col ) , } ) arcs_df = pd . DataFrame ( { "start" : edges_df [ [ "from" , "to" ] ] . min ( axis = 1 ) , "end" : edges_df [ [ "from" , "to" ] ] . max ( axis = 1 ) , "label" : edges_df [ "label" ] , "dir" : "left" , } ) arcs_df [ "dir" ] . mask ( edges_df [ "from" ] > edges_df [ "to" ] , "right" , inplace = True ) arcs_df = arcs_df [ arcs_df [ "start" ] != arcs_df [ "end" ] ] return { "words" : words_df . to_dict ( orient = "records" ) , "arcs" : arcs_df . to_dict ( orient = "records" ) , }
Convert a DataFrame in the format returned by `make_tokens_and_features()`
to the public input format of displaCy's dependency tree renderer.
Convert a DataFrame in the format returned by ` make_tokens_and_features () ` to the public input format of displaCy ' s dependency tree renderer .
"""
    Convert a DataFrame in the format returned by `make_tokens_and_features()`
    to the public input format of displaCy's dependency tree renderer.

    :param token_features: A subset of a token features DataFrame in the format
        returned by `make_tokens_and_features()`. Must at a minimum contain the
        `head` column and an integer index that corresponds to the ints
        in the `head` column.
    :param text_col: Name of the column in `token_features` from which the
        'covered text' label for each node of the parse tree should be extracted,
        or `None` to leave those labels blank.
    :param tag_col: Name of the column in `token_features` from which the
        'tag' label for each node of the parse tree should be extracted; or `None`
        to leave those labels blank.
    :param label_col: Name of the column in `token_features` from which the
        label for each edge of the parse tree should be extracted; or `None`
        to leave those labels blank.
    :param head_col: Name of the column in `token_features` from which the
        head node of each parse tree node should be extracted.

    :returns: Native Python type representation of the parse tree in a format
        suitable to pass to ``displacy.render(manual=True ...)``
        See https://spacy.io/usage/visualizers for the specification of this format.
    """ # displaCy expects most inputs as strings. Centralize this conversion. # Renumber the head column to a dense range starting from zero # Note that we turn any links to tokens not in our input rows into # self-links, which will get removed later on. # displaCy requires all arcs to have their start and end be in # numeric order. An additional attribute "dir" tells which way # (left or right) each arc goes. # Don't render self-links
None
token_features
pd.DataFrame
A subset of a token features DataFrame in the format
returned by `make_tokens_and_features()`. Must at a minimum contain the
`head` column and an integer index that corresponds to the ints
in the `head` column.
A subset of a token features DataFrame in the format returned by ` make_tokens_and_features () ` . Must at a minimum contain the ` head ` column and an integer index that corresponds to the ints in the ` head ` column .
text_col
str
Name of the column in `token_features` from which the
'covered text' label for each node of the parse tree should be extracted,
or `None` to leave those labels blank.
Name of the column in ` token_features ` from which the ' covered text ' label for each node of the parse tree should be extracted or ` None ` to leave those labels blank .
tag_col
str
Name of the column in `token_features` from which the
'tag' label for each node of the parse tree should be extracted; or `None`
to leave those labels blank.
Name of the column in ` token_features ` from which the ' tag ' label for each node of the parse tree should be extracted ; or ` None ` to leave those labels blank .
label_col
str
Name of the column in `token_features` from which the
label for each edge of the parse tree should be extracted; or `None`
to leave those labels blank.
Name of the column in ` token_features ` from which the label for each edge of the parse tree should be extracted ; or ` None ` to leave those labels blank .
head_col
str
Name of the column in `token_features` from which the
head node of each parse tree node should be extracted.
Name of the column in ` token_features ` from which the head node of each parse tree node should be extracted .