Prior to the arrival of the First Fleet in 1788, Prospect was inhabited by different groups of the Darug people including the Warmuli. The Aboriginals there were of the woods culture. As European settlement expanded, the aboriginal people's ability to pursue their traditional lifestyle, which was already severely limited, disappeared. Prospect Hill had been the frontier, which was the first, and perhaps only, area where large scale organised resistance by aboriginal people took place.