Kilgariff was born in Adelaide, South Australia, and in 1929 arrived in Alice Springs (then called Stuart) with his family on one of the first Ghan trains from Adelaide.
He attended the Hartley Street School, and in 1938, the local Catholic school.