Chadwick was born into the large family of a railway signalman.
When he left the school in 1939 at the age of sixteen, he was unsure of his vocation to ordination, and he spent the first three years of the Second World War maintaining station clocks on the railway line from Swansea to mid-Wales.