The FCD and FCFP neural networks used SiLU (sigmoid linear unit) activation function instead of default ReLU because it allowed us to slightly improve the accuracy of predictions on the validation dataset and prevent overfitting without using other regularization techniques. GNN and CNN neural networks used LeakyReLU because their architectures did not pair well with SiLU activation, and using LeakyReLU subtly improved the accuracy and smoothness of the loss curve for the validation dataset compared to ReLU.