 Barbera and his wife briefly separated when he went to California . They reunited but were on the verge of another separation when they discovered that Dorothy was pregnant with their first child . They had 4 children : two sons ( Neal and an infant boy who died two days after his birth ) and two daughters ( Lynn and Jayne ) . The marriage officially ended in 1963 . Shortly after his divorce , Barbera met his second wife , Sheila Holden , at Musso & Frank 's restaurant , where she worked as bookkeeper and cashier . Unlike Dorothy , who had preferred to stay at home with the children , Sheila enjoyed the Hollywood social scene that Barbera often frequented . 
