For deeper understanding of the pre-training approach, the embeddings of the molecules from the test set were computed for the MLM and MTR models. These embeddings were derived from the final hidden layer, both before and after the finetuning process. As each molecule's representation consisted of 384 embeddings, Principal Component Analysis (PCA) was performed subsequently for dimensionality reduction. The obtained representations in 2D latent space together with the explained variance ratio are illustrated in Fig. . In case of the MLM model prior to fine-tuning, the representations are dispersed randomly within the latent space. However, after fine-tuning, the representations corresponding to molecules classified by the model as HIV inactive exhibit lower values of PC1 (cf. light and dark blue points), while representations of molecules classified as HIV active are having higher PC1 values (cf. magenta and red points). In the case of the MTR model prior to fine-tuning, molecules correctly classified as HIV active tend to cluster together in their representations. On the other hand, the remaining points are scattered randomly with several outliers. Following fine-tuning, the distribution of representations becomes similar to that of the MLM model. The changes in the distributions of the embeddings before and after fine-tuning indicate that the models become more adept at distinguishing between HIV active and inactive molecules.