What determines if an allele is dominant or recessive?
Dominance is seldom complete. Owing to effects like co-dominance, incomplete dominance, collaborative (additive) effects of polygenes, our classical concept of dominance doesn't work. Having said that, there are certain ways in which a gene, if showing complete dominance can be analysed from the molecular level.   

In an individual heterozygous for a certain trait, the following possibilities exist:


Both the alleles, though different, code for a protein (which might be an enzyme or a regulatory protein) which is functional and is indistinguishable atleast in terms of its functions. In this case, no matter if the person is heterozygous or homozygous for either allele, the concerned protein will be in sufficient amounts and will always be functional. These alleles can be treated as the same allele while performing Mendelian analysis though their products might differ slightly, leading to formation of two (or more) functionally similar Alloenzymes (as opposed to Isozymes which are similar enzymes produced by genes at different loci).  
Haplosufficiency. In this case, one of the two alleles codes for a functional protein and the other either codes for a non-functional protein (or does not code at all). But, even in heterozygous condition where only one allele producing functional protein is present, the amount of protein produced is sufficient to show the phenotype and hence, even in heterozygous individuals, enough protein is coded for by the single functional allele exhibiting the normal phenotype. If the two non-functional alleles are present as a homozygous pair, no functional protein is synthesised and hence the phenotype is not shown (i.e a different phenotype is shown). Here, the functional allele is Haplosufficient (able to produce enough protein in heterozygous condition) and is called the dominant since it expresses its phenotype both in heterozygous and homozygous conditions. this is pretty common.  
Complete Haploinsufficiency. Here again one allele codes for a functional protein while the other does not. But this time, if the functional allele is present in heterozygous condition, then the amount of the protein produced is not at all sufficient and hence the phenotype is not exhibited. Here the non-functional allele is said to be dominant because heterozygous phenotype resembles the phenotype of homozygous non-functional allele (where no protein is synthesized) since the functional allele is completely haploinsufficient (unable to produce enough protein in heterozygous condition). This method is pretty rare.(e.g rare autosomal dominant dyskeratosis congenita) 


If there is partial haplosufficiency /partial haploinsufficiency, the phenomenon of incomplete dominance can be explained. (heterozygous individual produces protein not sufficient for a full-blown phenotype but just a partial expression of the phenotype)

Here, "functional" refers to being actually "functional" as in case of flower colour (anthocyanin synthesis) or just performing certain kind of a conversion leading to a particular effect on thephenotype.