The menu that shows up when a new mesh is added disappears when I click on it
Adjust Last Operation panel can be used just after some operation is performed. It does not disappear if you click on it, however it will disappear when you click somewhere else. 

It is possible to bring it back by going to Edit-> Adjust Last Operation or with F9, however this is only until you perform some other operation. When you do, it becomes the new last performed operation, the previous operation then becomes completely finished and you cannot change its parameters any more.