Kaye made his screen debut with a small role in Hal Roach's 1937 Our Gang comedy Glove Taps, with his first speaking part in Hearts are Thumps, released the same year.
Kaye remained a semi-regular in Our Gang through 1940, by which time production of the series had moved from the Hal Roach studio to Metro-Goldwyn-Mayer. Minor roles in other films included the musical Best Foot Forward with Lucille Ball and Kansas City Kitty, in both of which he played "Killer".