How do I see everything ever installed on my Mac in one place?
I'm not sure you'll get everything in one place across so many different installation methods, especially things like Homebrew, Macports and Ruby gems...

But for the 'standard' installations, you could check out CleanApp (or various alternatives). It tends to work best if you install it early on, since it monitors installations, but it can also detect, to some extent, what is already installed.

Another option would be AppFresh (or various alternatives), though AppFresh has gone commercial now, so no longer free.

Some of those (or the alternatives linked) should show you a good selection of App Store, manually installed Applications (DMG/PKG/zip) and the like, but I don't think any will go as far as Homebrew, Macports or Ruby.