Failed esterification of 1-butanol and ethanoic acid
I don't think the brown color necessarily means that the reaction failed. It could be due to some contaminant in your reagents but in a small quantity that wouldn't cause the reaction to fail. The fruity smell could very well be due to your product see: n-butyl acetate has a sweet banana smell. I would take the distillate you collected (or skip the distillation altogether) and try an aqueous work-up: Add very carefully aqueous $\ce{Na2CO3}$ until basic $\ce{pH}$ and then extract this mixture with ether. This way any ester product will be in the the organic layer and sulfuric/acetic acid will stay in the aqueous together with most of the alcohol. 

To finish the work-up dry the ether layers over $\ce{Na2SO4}$, filter and remove the ether (unless you have access to a Buchi evaporator, the safest way is to leave it in an open flask/beaker overnight). You can further purify the final product by distillation.

You might improve the yield if you cool the reaction mixture in an ice bath before adding sulfuric acid