How can I get iTunes to automatically refresh the media library?
What you call a “design” flaw, is not such thing. It’s a design decision made by the iTunes team. Although iTunes lets you “manually manage your library”, it’s never intended to let you move things around in the filesystem, add, remove or modify files. 

The premise for iTunes happiness is that you always add the items to the library and modify things from within iTunes. You have the choice to let iTunes “organize” the library (which will copy the media to the iTunes library folder and organize it there as he wants) or you can leave it wherever you originally had it, but iTunes doesn’t “read” or “monitor" the filesystem for changes. It knows things changed when the library.xml is changed, and the only application that does that is iTunes itself. 

Is this the best way?

Yes and no, it has advantages, but a lot of people miss the “Winamp-like” ease of use and it’s understandable. On the other hand, it simplifies a lot of things for a lot of people. It has decently worked for quite a few years now.

But that’s the topic for another discussion. :)