Given an image how should I detect this is a day picture or night picture?
This will work only if you're dealing with natural scenes (no artificial light sources). 

I think there are two other possible ways: 

1) looking for the percentage of dark pixels (define a threshold) that will tell you how many black areas are present in the image. Again, this will work only if we're not considering photos close to artificial light sources. Doing an average of the pixel value will lower the result if you have many lights in the scene, considering only dark areas could lead to better result (if there are enough in the scene).

2) Natural light have some difference in frequencies than artificial light, you could search which one is more present in the photo. I'm thinking at night pictures of cities for example, where lights are often in the yellow range. But i think this seems easier to say than implement, and looks more conditioned to the type of photo (natural panorama, city images, etc.).

P.S: i'm assuming the picture has been taken with the correct exposure time