Buddhist practice and physical exercises
It is ok(ethically) to do physical exercises even for the purpose of being attractive. Lay people are not expected to follow the rules of monks. Such conflicts occur mostly when you try to live a monk's life as a lay person. Is it clinging/attachment that motivates you for physical exercises? Absolutely! But it's similar to asking if it is ok to have love affairs or get married.  They all involve unwholesome mind states. But you are mostly expected to keep to the 5 precepts as a Buddhist layman. So you should focus more on the 5 precepts as they can be easily broken if you are not mindful. If you want to live a more noble life, you can get ordained. Things become conflicting and confusing when you try to walk on the fence, not picking either lifestyle. 

In fact, it is clinging that makes you do the job you do. It is clinging that keeps you in lay life. If you don't have clinging, you can't live as a householder. You have to go get ordained within 7 days or die(parinibbana).