Do dynamic routes depend on url rewriting?
I first attempted, with the link Mats posted, to edit my nginx config. That resulted in a rather terrifying "can't even find the index" error, but there was a silver lining: after destroying and recreating my Vagrant the urls mysteriously fixed themselves, and I no longer see querystrings on the index. Custom routes also now work. So that's mysterious but I'll take it for now.