Can I wire a doorbell transformer in-wall this way?
The line voltage parts of the circuit must be contained inside an electrical box which has an accessible cover. The low voltage wires should not be inside the electrical box. 

If I were doing this I would make a new opening for an old work electrical box up near the ceiling of the closet. Then drop a new electrical cable down the stud cavity from the new opening and enter into the existing electrical box (it may turn out to be easier to push the new cable through the existing electrical box up the wall to the new opening where you can reach in and snag the new cable). Once the new box is all installed in place you can mount the new transformer on the box cover of that new box with the low voltage wire terminals being up near the ceiling.

Before you start do check that wall cavity to see if it has a cross member or fire block that would impede pushing the cable up the cavity. If so it may be necessary to make a temporary hole in the drywall near the blocking so that you can drill a hole through it for the new cable. 

The new cable used must match the wire gauge of the existing wiring in the low down electrical box.