class_docstr: Tokens from the labour data.
docstr: Loads each tokenized year, and combines them into a single tokenized dataframe.
docstr_tok: ["Loads","each","tokenized","year",",","and","combines","them","into","a","single","tokenized","dataframe","."]
code: def tokenized(self) -> dd.DataFrame:
      """
      Loads each tokenized year, and combines them into a single tokenized dataframe.
      """
  
      parts = [
          self.tokenized_part(year)
          for year in range(self.start_year, self.end_year + 1)
      ]
      result = concat_sorted(parts, columns=["START_DATE"])
      result = result.reset_index().set_index(
          "PERSON_ID", npartitions="auto", sorted=True
      )  ## this line makes computations longer, but provides robust partitions
  
      assert isinstance(result, dd.DataFrame)
      return result
  
imports: from dataclasses import dataclass, field
  from pathlib import Path
  from typing import Dict, List
  import dask.dataframe
  import pandas
  from ..ops import concat_sorted, sort_partitions
  from ..decorators import save_parquet
  from ..serialize import DATA_ROOT
  from .base import FIELD_TYPE, Binned, TokenSource
label_desc: Use image node_id 1 for calling the LabourTokens obj's underlying member method code with example usage: obj.tokenized() and returns: result