Fish always giving bitter/unpleasent taste/smell
Start with a fish stock instead of water. Cook that with whatever vegetables you want to put in it, add seasoning and get the broth to the flavor that you want. 

During the last 3 - 4 minutes of cooking, add your cut fish and just allow it to poach in the broth for a minute or two, cut the heat, let it sit for a few to finish poaching gently, then serve.

It sounds like you're trying to extract flavor from the fish as you cook it, that's not going to work, you'll just overcook the fish which is what's giving you that awful taste. 

Fish stock is made from the very gelatinous bones of the fish, skin, scales, heads, etc - then strained. You can buy it in most grocery stores, just read the label to see how much sodium it contains (go for the least amount, add salt yourself).