My supervisor is trying to "steal" my collaborators: what can I do?
Leave this place immediately and find better support elsewhere.

As a postdoc you're independent and well-aware of it. You have your own network of collaborators, reputation. You know your job better than others. Nonetheless, some bureaucrat who depends on you is openly sapping your career, and you find that "understandable" and believe there's a friendly way around so that everyone is happy. I think you hold all cards there. 

My advice to focus completely on networking and finding a new place. The situation will not improve. Make use of the time and resources you have now to take the best advantage of your freedom to move and communicate. Why not move closer to your collaborators? It's fairly easy finding another postdoctoral position nowadays, visas, etc. There's no point in holding such bad ground. 

Networking will also naturally broaden your horizons, through revealing other career possibilities which steer away from academic parasites. Whatever you decide, just don't allow some parasite to win a game where you hold the cards.