caret dummyVars on unseen data
The predict function will work on the new data.  It has some options for how to handle entries that don't appear in the original data's labels.

https://rdrr.io/cran/caret/man/dummyVars.html