How to remove oven baked acrylic paint from glass used for drinking?
I would start with a razor blade and see if you can scrape it off.
If it doesn't readily scrape off, go to chemicals (below are suggestions that work with dried acrylic paint; try one at a time, do not mix them).  The procedure for all of these:

Cover the painted area with several layers of paper towel saturated in the chemical.
Wrap that with plastic wrap and rubber bands to seal it against the glass and minimize evaporation (don't leave any of the paper towel exposed).
Set it on a surface that won't be affected by the chemical you're using (avoid wooden or painted or finished surfaces; plastic or glass is a safer bet).
Use common sense and follow directions for personal safety (good ventilation, wear gloves and eye protection).
Let it soak under the paper towel for 30 minutes to an hour.  Then see if the paint has softened enough to wipe or scrape it off.
If it doesn't seem to be affected, try again with another chemical.  If it helped but there is still residual paint, repeat the process (try a different chemical if the result is unimpressive).

First chemicals to try (the least nasty; pick one):

Household ammonia (full strength)
Isopropyl alcohol, 91% - 99% (may need to shop around for a drug store that carries it at this strength)
Denatured alcohol (available in the paint section of your hardware store)
Goof Off (Pro Strength Remover or Paint Splatter Remover if varieties are available)

Second (nastier) group (outside is better; away from open flames or sources of sparks); most of these are available in the paint section of your hardware store:

Acetone
Lacquer thinner
Paint stripper (one containing methylene chloride and/or methyl ethyl ketone [MEK] is likely to work well)
Mineral spirits / paint thinner is something you may have on hand.  I don't know how well it will work, but if you don't have any of the other chemicals, give it a try.

Note that there are &quot;green&quot; versions of some of these products; many of them water-based.  I don't know which of those will work on dried acrylic or how well.
Some of these chemicals will evaporate pretty fast.  Even with plastic wrap, the paper towel may start to dry out in less than 30-60 minutes, so check it.
If you use any of these chemicals, clean the glass surface with alcohol and let it dry.  Then wash it with soapy water (running it through the dishwasher after the alcohol has dried is fine).