He was the youngest son of Joseph P. Kennedy Sr. and Rose Fitzgerald Kennedy . He was the younger brother of former President John F. Kennedy and Senator Robert F. Kennedy , both of whom were assassinated , as well as Joe Kennedy , who was killed in World War II . For many years the most prominent living member of the Kennedy family , he was the last surviving son of Joseph P. Kennedy , Sr. ; the youngest brother of President John F. Kennedy and Senator Robert F. Kennedy , both victims of assassination , and Joseph P. Kennedy , Jr. , killed in action in World War II ; and the father of Congressman Patrick J. Kennedy .