She was the only legitimate child of the poet Lord Byron and Anne Isabella Milbanke . She had no relationship with her father , who died when she was nine . She was the only legitimate child of the poet Lord Byron ( with Anne Isabella Milbanke ) , but had no relationship with her father , who died when she was nine .