i have the floor plan of the castle memorized ; we should be at the back part of the castle , near the old servants quarters , olivia explained , as she stepped up to push open the doorway into the castle proper .