convolutional neural networks use case in the worst-case scenarioyou might have to train the entire network from the ground up in most caseshoweveryou are quite likely to be able to save compute efforts with few or more layers from the pretrained networks using dropout is always good idea for most use casesrelus can be blindly be used as the de-facto activation function for fairly acceptable performanceensure that each class has , or more training samples the morethe better the batch size should be as large as the gpu or cpu can handle optimizing the batch size helps to accelerate the training process gpu is always recommended gpu performance is almost or above for most common use cases the cost of acquiring gpu-based instance has come down significantly all major cloud players provide ready-touse deep learning images or virtual machines that can be provisioned on demand with suitable compute and gpu the entire heavy-lifting task ( installing the required dependenciespackagesand driversand configuring deep learningthe python frameworkworkspaceetc is abstracted with single click the cost has also come down to provide an affordable means to train few experiments todayyou can provision powerful machines with gpu that fair well with most research projects for ~ usd/hour