Should I switch to a smaller chain ring?
If I were in your situation, I would definitely get a compact (50/34) crank.  Gaining lower gears for climbing makes some uphills possible and other uphills easier. All you have to give up is maybe some top speed on downhills. You're still able to ride those downhills, only you'll be going 55kph instead of 60kph.  Or increase your cadence.  In general I come across a lot of people who think they need huge gears because they somehow want to have a really low cadence.

Since you said you're riding for fitness and weight loss, I don't think that top speed is a priority for you -- you're not competing against others in a race.  It's much more important that you can handle hills and don't hurt your knees.

My top gear is 46/11 and I never find that to be a limitation.  A cadence of 120rpm is quite reasonable for a few minutes (few descents are really long) and 120rpm with a gear of 46/11 is approximately 65kph.  With a top gear of 50/11, it's 71kph.