Delmar, Alabama. Delmar is a small, rural, unincorporated community in west-central Winston County, United States. Delmar is located six miles north of Natural Bridge, five miles south of Haleyville and 15 miles west of Double Springs, the county seat of what was once the Free State of Winston.. Delmar has an elevation of 881 feet above sea level.