Do babies resemble their father?
Resemblance between parent and offspring is certainly true. Some traits are more obvious than others, for example skin colour would be very obvious. As babies mature they will gradually begin to look more and more like their parents, the features of babies often change rapidly, for example pretty much every baby has a little nose, but by the time they are a fully grown adult it probably resembles that of one the parents fairly well.

More obvious example would be height - imagine being presented with a baby and two males who are identical in every way, except one is two metres tall, and the other is 1.5 metres tall. As a baby it would be very difficult to assign paternity on the criteria of height/size, but as the child develops it would likely become easier and easier.

The resemblance between parent and offspring relates to the heritability of a trait but, because there can be temporal variation within an individual, heritability can only be properly determined by measuring the traits in parent and offspring at the same time point (e.g. father's height at 18 years old and son's height at 18 years old), otherwise it will introduce noise in to the estimation, likely reducing the estimate of heritability (and specific to your question, making it harder to assign paternity based on phenotypic characters).

In the context of your question: Ability to assign paternity would depend on a) the resolution with which the traits available can be distinguished among all individuals in the test and b) the variance among all individuals in the test. E.g. If the only trait available was number of fingers per hand, and all potential fathers and the child had five then the lack of variance would make it impossible to use this as a criterion, while if both one of the potential fathers, and the focal baby, had six fingers on one hand then there would be pretty good clue.

Whether babies more resemble their fathers more than mothers (which I think is the true question you are asking here) there is evidence from Drosophila that within-sex heritability is much higher than cross-sex for lifespan and aging, i.e. the lifespan of males (females) far more resembles the lifespan of the father (mother) than the mother (father). This would suggest that there is a sex specific element. Meanwhile there is evidence of high cross-sex genetic correlations in humans preventing males and females from becoming more different (thus maintaining resemblance to opposite sex parents). I would think that resemblance to the father could offer some benefit in assigning paternity by care giving males (stopping males devoting resources to other males offspring) but some of the resemblance could be generated by genetic correlations to sex-specific genetic variance, for example, males (females) will look more like their father (mother) in many ways purely because they are of the same sex, so it will heavily depend on the traits you look at to judge resemblance.

For a related answer about heritability on biology SE see:

Is it possible for a child to grow taller than their tallest parent?