Diagnosing problem with new shower grout--coming off to the (dry) touch and cracks/holes appearing
When I set my hand on the shower wall getting in and out, I noticed that a powder stuck to it


I suspect poor / lazy grouting technique. After grouting it is normal for the tiled surface to be covered with a "grout haze", which is just a very thin film of dried grout left by the grout finishing/sponging process. Good sponging practices, e.g. using multiple buckets/sponges (dirty, cleaner, cleanest) and frequent sponge wringing and water changes minimize (but do not eliminate) the haze. Pros will wait till the grout is cured then use a mild acid rinse followed by clear water rinse, to remove all haze.


  I also noticed a few spots where there were small holes and cracks in the grout


Again, not unusual with inferior technique. Common cause of small holes/cracks are poor grout installation, e.g. lame technique with grout float, incorrect water/grout ratio, improper mixing, failure to allow grout to slake, crappy / lazy sponge technique, etc. 

You can try to mix up a bit of additional grout and use a finger to smooth it into uneven areas and small pits/cracks; make sure to moisten existing grout in the area first. This will almost always be visible to a critical eye but may be better than leaving as-is. Alternatively you could grind down all the grout and completely regrout... You are a victim of money vs. craftsmanship, which is more common than not these days.