@overrides<EOL><INDENT>def gen_batches(self, batch_size: int, shuffle: bool = None)-> Generator[Tuple[List[str], List[int]], Any, None]:<DEDENT>
if shuffle is None:<EOL><INDENT>shuffle = self.shuffle<EOL><DEDENT>if shuffle:<EOL><INDENT>_doc_ids = self.random.sample(self.doc_ids, len(self.doc_ids))<EOL><DEDENT>else:<EOL><INDENT>_doc_ids = self.doc_ids<EOL><DEDENT>if batch_size > <NUM_LIT:0>:<EOL><INDENT>batches = [_doc_ids[i:i + batch_size] for i in<EOL>range(<NUM_LIT:0>, len(_doc_ids), batch_size)]<EOL><DEDENT>else:<EOL><INDENT>batches = [_doc_ids]<EOL><DEDENT>for i, doc_ids in enumerate(batches):<EOL><INDENT>docs = [self.get_doc_content(doc_id) for doc_id in doc_ids]<EOL>doc_nums = [self.doc2index[doc_id] for doc_id in doc_ids]<EOL>yield docs, zip(doc_ids, doc_nums)<EOL><DEDENT>
Gen batches of documents.

        Args:
            batch_size: a number of samples in a single batch
            shuffle: whether to shuffle data during batching

        Yields:
            generated tuple of documents and their ids