after toby replied in a flushed and happy state , they continued with the remainder of the meal , until lady wentworth gave the signal for ladies to retire , leaving the gentlemen to their port and tobacco .