To close multiple windows, click the Close button in the upper-right corner of the window or press CTRL+F4. When you have closed all multiple windows, the workbook name no longer displays a window number.