How do I build and install gcc5 in OS X 10.11 without Xcode?
You can find several versions of gcc for different OS X flavors here: HPC Mac OS X.

Download my binaries, and cd to the download folder. Then gunzip gcc-5.3-bin.tar.gz (if your browser didn't do so already) and then sudo tar -xvf gcc-5.3-bin.tar -C /. It installs everything in /usr/local. You can invoke the Fortran 95 compiler by simply typing gfortran.
[To compile other sources...] you will also need to have Apple's XCode Tools installed from the Mac App Store. With XCode 4 or better you will need to download the command-line tools as an additional step. You will find the option to download the command-line tools in XCode's Preferences. On 10.9 Mavericks or higher, you can get the command-line tools by simply typing xcode-select --install.

Addition in square brackets by me...

Please also visit Information Security Stack Exchange and search for questions about the security risks associated with precompiled compilers from &quot;unknown&quot; sources.