Would I be considered a Masters degree holder right after my course completion?
The date on which you finish courses is not necessarily the date when you earn the degree, nor is the date of the commencement ceremony.

What matters is the date on which the degree is conferred by the university. If you're not sure what that date is, ask the registrar's office.

In some cases if the degree conferral date for students completing their coursework in December/January is after the H1B deadline, the registrar may have some kind of special arrangement (see this example from Stanford), which they'll tell you about if you ask.