How do I use Convolutional Neural Nets to classify if there are rain drops or soil on the camera lens? I have certain videos for which the frames are labeled either as dirty (meaning the camera lens is occluded by soil or rain) or as clean.
The goal is to test a convolutional neural net on this data to evaluate how well it can classify if a frame is dirty or clean.

One idea is to use the first layers of an existing network and train last (fully connected) layers using the available data.

But one issue might be that basically all available networks are trained for object classification and might not be very suitable for tasks like soil and rain detection.

Do you have any recommendations about networks or models which might be suitable for this task?
So neural nets can be quite powerful for end-to-end training. So if you have a dataset that contains both clean and dirty images, and also the classes you are trying to detect in the first place, it should be fairly straightforward to try to train you net. 

I'd proceed as follows. 


adjust the network to produce a logits vector that outputs not only the object classification but also a binary classifier for whether your image is clean or dirty. 
then adjust the labels for each image to indicate the classification label and the cleanliness of your image.
I'd then train and test on the dataset to see if a model emerges. 


The convolutional parts of your network should converge to features that are good at
A. classifying the subject of the image, and 
B. detecting whether the camera is dirty or not.

Its a good idea and I might use it myself!