Can I install Compass in Yosemite WITHOUT installing Xcode or Command Line Developer Tools?
Compass

Compass shouldn't require the Command Line Developer Tools, just a compatible ruby environment.

I suggest a ruby version tool instead of using the system ruby, so that you can control your ruby version/environment. I favor chruby, but any of these works well and can be installed without needing the Command Line Dev Tools:


chruby  
rbenv  
RVM  


Once you have a ruby env with the version you want you should be able to install Compass from the command line (as detailed on the compass site):  

$ gem install compass


Note: the "$" isn't part of the command, just an indicator of the terminal prompt

Command Line Developer Tools

While the above works for Compass, it sounds like you are asking about general package management with OS X as well. Homebrew is a very popular tool for this, but depends on the Command Line Dev Tools. MacPorts is an alternative but has the same dependency. I'm not aware of a package manager for OS X that doesn't depend on the Command Line Dev Tools.

In case you haven't tried one of these methods to install the tools I'm throwing them out there:


Download directly from https://developer.apple.com/downloads/ (you need to login with Apple ID). The URLs provided there may be resumable if the download is interrupted, and may be more reliable than installing without Xcode via the terminal.
Install without Xcode via the terminal:

$ xcode-select --install



Having the tools definitely makes things easier; I personally love Homebrew (and it's offspring Cask) for managing both CLI and GUI packages.