Applied to two MS programs, but one wants a decision before the other has made a decision
Would it be wrong to accept the offer to University A, and then
  withdraw later?


Ethically, I think this would be wrong unless you expressly told University A of your consideration when you make the decision.  When you accept an admission offer you are taking up limited resources and potentially depriving someone else of admission to the university.


  Or, would it reflect poorly on me if I asked for an
  extension to make a decision?


Not at all - just tell them that you are very interested in A but also have other possibilities and would like to make a fully informed decision.  The worst thing that will happen is that A says "no".


  I find it strange that University A
  wants a decision so quickly, seeing as how you can apply for admission
  for the fall quarter up through mid-August.


Presumably, University A has a limited number of slots that they can fill, and, if you say "no", they would like to extend the slot to someone else before decisions are made.  I'm a bit surprised that these universities are making fall MS decisions in June - these decisions are typically done by mid-April.