Review conference papers under supervisor name
In my experience practices around this vary considerably, between "this is completely normal" to "this is an unthinkable breach of ethics". I recommend talking to trusted colleagues (not your supervisor, whose opinion on the topic is fairly clear through their actions) about whether this is considered acceptable in your field or not. If it turns out that this is uncommon or considered unethical in your field, you may want to approach the supervisor directly and state that you are not comfortable with the arrangement that part of your duty is to review papers in their name. However, I would honestly only suggest doing this if you feel rather strongly about this issue and you have a working relationship where concerns can be raised without fear of fallout. Yes, every working relationship should be like that, but if observing this forum for many years has taught me one thing it is that not all supervisors actually act like that.

Some further considerations:


I wouldn't over-value the exposure one gets from reviewing papers. It's not 0, but realistically the benefit is small enough that it would not seem a worthy hill to die on in case the supervisor is not reasonable or does not see anything wrong in farming out their reviews (in my personal opinion).
Your exposure is one thing, but the more relevant question may be how happy the journal / conference is about how your supervisor handles this. Typically, one invites an individual to review because of their personal expertise on the topic. If your supervisor farms out review requests to other, more junior, researchers without actually checking the manuscript themselves, the journal / conference is not getting the expert review they signed up for.
In many fields it's fairly common to have junior researchers review papers assigned to their supervisors as a training experience. However, part of this is also that the supervisor also reviews the manuscript and validates/corrects the review of the junior. Since you are a newcomer to your field this may be a valuable exercise for you as well (although you already hold a PhD) - but of course it's only of value as a training instrument if your supervisor actually sits down with you afterwards and discusses the review with you (and ensures that the review outcome is consistent with their own expert opinion).