How can I stop someone borrowing my bike anymore?
He probably doesn't feel very empowered to fix his own bikes, and they may have issues that are beyond his means to fix or get fixed for him. He may not really understand like you do that things need to be taken care of to work well, which is perhaps a reason why he's unduly rough on your bike.

It sounds like the allure of a bike in working order and despair over his own situation is causing him to act jealous and selfishly in turns.

As he's your friend, you might consider offering help in fixing his own bikes. Getting some support there may be what he needs to move forward from the habit he's in of leaning on you unfairly. And by learning what it takes to get and keep a bike working well, he may learn to be less unduly hard on them.

Remember you can always post here about whatever mechanical issues you encounter.

Also, I feel this question is reasonably within the bounds of 'bike culture' and is not out of place here.