"Hey Siri" also works for other persons than me
As 9to5Mac says in the article iOS 9 includes ‘Hey Siri’ voice training to help Siri better recognize your voice…

This makes the addition of voice training particularly relevant. In previous OS versions, Hey Siri would just work if you toggled a switch in Settings. With the GM, the OS now prompts you to go through a few training exercises before the feature will be enabled. Some readers have claimed that this feature is like Voice ID, so that Hey Siri function will only respond when the true owner of the phone speaks to it. Whilst this would be a nice feature, we cannot reproduce this and believe it is only meant to improve general detection accuracy.
It is true that the iOS screens describing the feature are ambiguous. It says “this helps Siri recognise your voice” which could be interpreted as meaning your voice only. However, the statement is vague enough such that it could also mean overall improvements to Siri voice detection on the device, not specifically the current user.

So it's not necessary trained to recognize your voice, rather, it improves voice detection. There's no real fix for this as far I can tell, voice technology just hasn't come that far yet.