What is the best way to decrease the circumference of a vest arm hole?
Cutting into the existing armhole will most likely end badly, because you need a lot more fabric to secure those darts than is available at the arm hole.
If you want to make the hole smaller, you have to sew more material to it. I hope you didn't throw the sleeves away yet, because that is the perfect material for your endeavor.
You didn't post a picture or write any details, so I can only guess where exactly you need to add material.
If you have enough fabric at the arm hole, but not enough to seam it properly, try cutting bias tape from the sleeves and hem the west with that. Since the arm hole is a circle, the tape must be cut on the bias or it will wrinkle and be a pain to sew on. Keep in mind that the tape you cut must be 4 times as wide as the hem should be in the end.
If the west fits well enough at the soulders but has a gaping hole below the arm, it's going to be more complicated. First I would try the bias tape method, but with wider tape below the arm that protrudes from the actual seam (basically extending the fabric there) and narrower tape at the top of the shoulder that doesn't protrude at all.
You could also try opening the shoulder seams and sewing them a little tighter towards the side, but that will change the fit of the whole garment. You should first pin the shoulders and try the vest on to see if it still fits or if the shoulders become too sloped or the collar stands up weirdly.
If the seams are almost invisible in the fabric, you can try inserting a halfmoon shaped piece below the arm. But in most fabrics the seams are too obvious for that.
If nothing else works, buy or knit cuffs in a color that matches the vest (or adds a nice contrast). Add cuffs around the arm hole, at the bottom of the vest and - if it fits and you like the look of it - also at the collar.