Stern married Edith Stern (née Rosenwald), daughter of Sears Roebuck magnate and philanthropist Julius Rosenwald, in 1921.
Stern's brother S. Walter Stern was also in the cotton business and was a philanthropist in the city of New Orleans [REF].