In the biomedical domain, researchers have applied pre-training techniques, such as BERT, to various applications, including the analysis of clinical trials. For example, BioBERT is a domain-specific BERT model pre-trained on biomedical articles, achieving state-of-theart performance on NLP tasks related to biomedical questions. Other approaches, such as Doctor2Vec , DeepEnrol , and Compose , also utilize pre-trained models like BERT to embed clinical trial information, yielding promising results in various tasks. MiME utilizes the inherent hierarchical organization of EHRs data and the encoded associations among medical codes to address the challenge of large data volume. These approaches demonstrate the potential of pre-training techniques to advance the analysis and understanding of clinical trial data, potentially leading to improved patient outcomes and better-informed clinical decision-making.