The hamlet draws its name from the Carle House - a 32-room house built by Silas Carle in Westbury in the 1800s.
Legend has it that as New York City residents and friends of Carle visited Silas Carle, Long Island Railroad conductors would announce "everyone getting off for Carle's Place" to get off the train is how the name Carle Place evolved.