The outcomes of this study align with the notion that transformer models hold significant promise for predicting molecular properties. Zhu et al. showcased an effective strategy employing the RoBERTa transformer architecture integrated with a Graph Neural Network (GNN) . This approach enabled inputting to the model two distinct molecular representationsthe chemical structure and SMILES notation -concurrently during the pre-training phase, a concept termed dual-view pretraining. The training objective for the transformer branch was to predict masked token, i.e., MLM method. Moreover, a third objective was to maximize the consistency between the two high-level representations produced by each branch. After pretraining, the fine-tuned transformer branch achieved state-ofthe are performance on different molecular property prediction tasks, with HIV activity precision task characterized by AUROC of 0.810 . Further research into impact of various pre-training methods on molecular property prediction in downstream task, employing either the transformer alone or in conjunction with GNN has the potential to drive additional progress in these domains. This could result in the development of robust models capable of revolutionizing the drug discovery process.