Fitzgerald and Brown were both often busy touring and recording , Ray was mostly looked after by Fitzgerald 's aunt , Virginia . With Fitzgerald and Brown often busy touring and recording , the child was largely raised by her aunt , Virginia .