Where should I put custom CSS files?
In general, Craft doesn't normally come into the equation when we're talking about general frontend assets.

Things like minification, preprocessing of SASS/LESS/Coffescript/WHATEVER, image compression (asset images, not content images) etc. should preferably be handled by a tool like Grunt or Gulp (my favorite).

Even when it comes to CSS/JS for your plugins, you might want to do the actual build process with Gulp.

I usually have all source files in the project root, under a folder named src. In there, you have subfolders named js, scss, img, fonts etc.

If you're unfamiliar with for example Gulp, I might recommend you Mark Goodyear's article Getting started with Gulp.

On the Craft side, you could take a look at John D. Wells' awesome Minimee and Inlin.