How to make good coffee in a Moka pot?
Moka coffee pots work well if you use them every day. This way the oils that the coffee leaves inside them do not have the opportunity to go rancid (all fats go rancid eventually) and impart that horrible taste to your coffee. If you use them every day, it is true that you don't really need to clean them, except to remove calcium scale buildup.

If you use it rarely, like in your case or the frequent Moka pot left at a vacation house, you need to clean it with dish detergent and a scrubby sponge, you don't need anything harsher. Make sure that you get all the coffee brown oils out. Yes, it is an unpleasant job. Make sure you rinse it well before putting it away to dry, you don't want to simply replace the bitterness with soapiness.

...and this is the Italian doctrine! :-)