Born on 2 September 1824 at 3 Gray's Walk, Lambeth, he was a younger son (in a family of 13 children) of Joseph Ebsworth and Mary Emma Ebsworth, writer for the stage; Thomas Woodfall, son of Henry Sampson Woodfall, was his godfather.
At 14 Ebsworth entered the board of trustees' school of art, where he studied successively under Charles Heath Wilson, Sir William Allan, and David Scott.