How do I use slicing as I pass a transformer dataset to Trainer?
In reference to this colab notebook (from Huggingface Transformer course here), if I run
tokenized_datasets[&quot;train&quot;][:8]

the dtype is a dict instead of a Dataset and the slicing would return some data.
If I pass the slicing in here, I get a Key error, which I assume has to do with the fact I'm no longer passing a Dataset.
from transformers import Trainer

trainer = Trainer(
    model,
    training_args,
    train_dataset=tokenized_datasets[&quot;train&quot;][:8],
    eval_dataset=tokenized_datasets[&quot;validation&quot;],
    #data_collator=data_collator,
    tokenizer=tokenizer,
)



trainer.train()
***** Running training *****
  Num examples = 7
  Num Epochs = 3
  Instantaneous batch size per device = 8
  Total train batch size (w. parallel, distributed &amp; accumulation) = 8
  Gradient Accumulation steps = 1
  Total optimization steps = 3
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
&lt;ipython-input-20-3435b262f1ae&gt; in &lt;module&gt;()
----&gt; 1 trainer.train()

Is there a simple way to just pass a subset of the Dataset rows for training or validation?

import transformers
from datasets import load_dataset

datasets = load_dataset('squad')
datasets

output:
DatasetDict({
    train: Dataset({
        features: ['id', 'title', 'context', 'question', 'answers'],
        num_rows: 87599
    })
    validation: Dataset({
        features: ['id', 'title', 'context', 'question', 'answers'],
        num_rows: 10570
    })
})

then we can sample partial dataset:
train_datasets_sampled =  datasets[&quot;train&quot;].shuffle(seed=42).select(range(2000))
eval_dataset_sampled = datasets[&quot;validation&quot;].shuffle(seed=42).select(range(500))

train_datasets_sampled, eval_dataset_sampled

get the following:
(Dataset({
     features: ['id', 'title', 'context', 'question', 'answers'],
     num_rows: 2000
 }), Dataset({
     features: ['id', 'title', 'context', 'question', 'answers'],
     num_rows: 500
 }))

source: https://github.com/huggingface/notebooks/blob/master/transformers_doc/training.ipynb
