1882 and 1883 he bought the site on which the church now stands from Sir Julian Goldsmid, who owned it by that stage. Congreve founded a Young Women's Christian Institute in 1882, and an iron building was erected on the site to accommodate it. John Wills, a prolific designer of buildings for Nonconfirmist communities, was asked to design a permanent church for the congregation. (One of his later commissions was the Hove Methodist Church.) Congreve paid the full cost of construction and also became the church's first treasurer. Building work took place during 1887, and the first service was held on 29 July 1887. Charles Spurgeon, an influential Baptist preacher, was invited, but he could not attend because of illness. He helped in other ways, however: he sent his brother to preach in his place at the inaugural service and selected the church's first pastor; and one of his sons took this role subsequently. The first pastor, Rev. David Davies, served from 1887 until 1907; on his initiative, Gwydyr Mansions—an "elegant" block of mansion flats in the Flemish Renaissance style— were built opposite the church in 1890. The temporary iron building was removed to allow the church to be built, and the Young Women's Christian Institute moved into new premises next to the church. A similar institute for men was established in 1899, but neither remain in existence. The church helped to establish two other Baptist places of worship in Hove. In 1901, a mission church established four years earlier in the west of Hove was connected administratively with Holland Road Baptist Church, and by 1904 it was able to move to its own 400-capacity premises. The Stoneham Road Baptist Church remained open on the same site until 2008, but the building was sold and demolished in that year. In 1957, a deaconess at Holland Road founded a new church on the rapidly developing Hangleton housing estate (the population grew from 109 to 6,158 between 1931 and 1951). It is now called the Oasis Christian Fellowship Church and is part of both the Baptist Union of Great Britain and the Evangelical Alliance. In the 1970s, the church hall was redesigned, and exterior alterations and cleaning took place between 1980 and 1981. Work on the interior of the church had been proposed for many years, and an architect was appointed in 1991. When the plans were ratified in 1997, the structure