Trying other PhD opportunities while accepting an offer from my master supervisor?
Should I give up the current opportunity, and try to apply other universities?


Why do you have to give up this opportunity to apply to other universities? Find out when your advisor needs your decision by; apply to other universities in the meantime; and when the time comes to give your advisor your decision, evaluate that offer as well as any other offers you have in hand at the time.


  Is it ethical or appropriate to accept the current offer first, then keep on applying other universities?


Not unless you tell your advisor that's what you're doing.


  Another worry is that will he writes a good reference for me if he wants to keep me in his group? 


If your advisor is a reasonably competent advisor, then he will want what's best for you. If that means looking into other opportunities, then he should be supportive. If he isn't supportive, then either


he will make a terrible advisor. Consider yourself lucky that you found out now, and get away fast. Or,
he thinks you'll do best staying where you are. If this is the case, ask him why he thinks so. Consider his answer carefully, then make your own decision.



  The selection results of other universities span across a large time range, if I wait until all selection results are available, it would be too late to accept my current offer. Is there any way I can balance the risk?


Obviously, you'll have to commit to a choice at some point. But this is highly situation-dependent. The best person to advise you on this matter is your advisor. He can tell you when he needs your decision by, estimate your chances of getting into the programs you're interested in, and weigh the risks of each approach with your specific goals in mind.