In late 1964 , Gaye recorded `` How Sweet It Is ( To Be Loved By You ) '' , which peaked at the top ten of the American pop charts and became one of his first hits abroad reaching the top fifty in the United Kingdom . Gaye 's hits continued throughout 1964 .