If I stop consuming protein, will I still excrete urea?
Yes, you will still excrete urea, but the amount you excrete will be diminished. As you might expect, when the body is low on protein, there's a physiologic drive to hold onto as much nitrogen products as possible, in order to conserve the limited supply. 

Normally, your kidneys excrete approximately 40% of the urea that is filtered through the glomerulus. (It's slightly more complicated, because there's both secretion and reabsorption that happens along the nephron, but that's approximately the end result.) However, in protein malnutrition, the kidneys reabsorb more of the urea, and that number can drop to excreting only 10% of the filtered urea. (Source) It appears that this is an active process to reabsorb the urea, rather than a passive effect from other effects that malnutrition has on the body.

However, the body isn't capable of stopping excretion of urea completely... there will still be some that is excreted. Urea transporters in the kidney aren't 100% efficient, but probably the bigger issue is that the body needs to use that urea to regulate other functions. Urea is important to the body's ability to concentrate urine, which in turn is related to the body's regulation of blood volume and sodium concentration. And in general, the body cares more about maintaining proper blood volume and tissue perfusion than anything else.