After getting hired, department matter
I seriously doubt that it matters, but you might want to develop the story with your new colleagues at some point. The story is interesting, not terrible. It is a bit unusual, that is all. 

In fact, it is in a lot of ways the most honest answer rather than the most bureaucratic answer. 

Relax. Develop a nice story and become a legend.