   Pre-compiled Headers - Tips? Pitfalls?
  Hello,I'm trying to speed up the compilation time of my C++ projects so I can iterate faster and help my team do the same.I'm exploring the use of precompiled headers with msvc and gcc/g++.  My first notion was to refactor the headers in my projects to only add includes unless absolutely necessary and instead prefer forward declarations.I found this to be tedious, error-prone, and easy to forget to do.  Enforcing this would make us all better programmers but in the absence of vigilance its for naught.Using pre-compiled headers requires only a few projects settings and/or compiler switches with the added bonus that with some IDEs the settings can be imported and exported on a need basis with very little pain.Another argument against forward declarations is that hard drives are so fast now that the effectiveness of using them is greatly diminished.  A subjective determination but I can see where it would have merit.In short I feel my team will adopt using pch's much more readily and suffer much less expended effort for doing so.PEOPLE QUESTION 1:  Have I formed a good basis for my conclusion?TECHNICAL QUESTION 1:  I've populated my precompiled header with "unchanging" standard headers and those from 3rd parties like Boost.  The precompiled header includes headers that are also included in the source files that reference it.  Should I eliminate the includes in the source files since there are already in the precompiled header?PEOPLE QUESTION 2:  I don't want to deploy pch files etc to clients.  I don't want the precompiled header to become the goto "include everything" header.  I could write a script or exe that stripped every include of the precompiled header from all translation units etc.  That risks breaking things a the last moment.  Is there no better approach than a "DON'T INCLUDE THIS FOR THAT" policy here?TECHNICAL QUESTION 2:  Does Eclipse or Netbeans offer any assistance in using pch's in their projects settings with gcc/g++?  I can DL and find out for myself but I just thought I might ask first.TECHNICAL QUESTION 3:  I've created a pch in one MSVS project.  Can I use the existing pch in other projects? How can I do that and avoid build order issues?  Assume all projects are part of the same solution.  Eclipse/Netbeans?TECHNICAL QUESTION 4:  I've had to use the Zm option in MSVS to allow a pch to be built due to memory allocation limitations.  Can I just set the memory allocation to some value I know will never be exceeded going forward or are there undesirable affects for doing this?   
  P1. Hard drive speed isn't the issue. It's the speed that the compiler can process all that extra code that's slowing it down. The .pch file stores the state of the compiler at the end of compiling all those header files.T1. No need to remove includes, header guards should mean it won't have a significant performance hit. I'd also suggest precompiling any headers which are included in lots of places, regardless of how often they get modified.P2. The .pch file is built by the compiler. No need to distribute it. Just distribute the .h / .cpp it's built from and the project / solution files. To avoid overuse just have the precompiled header not include anything if a certain define is set, and set up a build configuration to test with it.T2. No idea. Never used them.T3. Try it and see. I'd expect it will work as long as the compiler command lines match. If they don't you'll need separate .pch files, but they could be compiled from the same .h file.T4. Never used it, but a quick look at the docs suggests you don't want it too huge or you'll risk running out of memory for other things.   ;Quote:Original post by Adam_42T1. No need to remove includes, header guards should mean it won't have a significant performance hit. I'd also suggest precompiling any headers which are included in lots of places, regardless of how often they get modified.My research and tests suggests that including frequently altered headers causes the pch to have to be completely regenerated often nullifying its value.  An amenable compromise is to place my own "application" code in a separate pch somehow.  I've seen language which suggests pch files can reference one another but the examples I've found thus far are a bit cryptic.Quote:Original post by Adam_42P2. The .pch file is built by the compiler. No need to distribute it. Just distribute the .h / .cpp it's built from and the project / solution files. To avoid overuse just have the precompiled header not include anything if a certain define is set, and set up a build configuration to test with it.Seems a good idea.  A vsprops file in Visual Studio should be all I need then on Windows platforms.Quote:Original post by Adam_42T3. Try it and see. I'd expect it will work as long as the compiler command lines match. If they don't you'll need separate .pch files, but they could be compiled from the same .h file.I had tried it before posting.  Your latter assertion seems spot on.  Each project has to create its own pch due to pdbs etc. not being generated at the same time for different projects.  A little more work but hopefully worth the effort.   ;  Why are precompiled header files not generated automatically for all header files?Once changed, it needs to be recompiled anyways. When outputting a precompiled header midway doesn't take much time, it can only save time later on...   ;  Ive never used pch's before myself, but I found my build(s) were taking longer and longer and want to speed them up. I found an article that had a method in it, cant find it right now ill have a look, but it said in a nut shell, #include all your .cpp files in another .cpp file, for each of the .cpp's you included exclude them from the build, in VS right click the file go properties, exclude from build. Then make sure the only file you build is the cpp with all the other cpps included in it. It cut some of my builds down from 20-25 seconds to 3. The short fall is that doing incremental builds is now the same as doing a rebuild. The time is so much less that I just do full rebuilds. Might help your situation, might not.Normally, build FileA.cpp, FileB.cpp, FileC.cpp, FileD.cpp, etc.This way, build _FastBuild.cpp, which #include "FileA.cpp"#include "FileB.cpp"#include "FileC.cpp"#include "FileD.cpp"its a little extra work, because when you add a new .cpp you have to exclude it and add it to the single compilation file (well you dont have to, but it will start slowing down from here)checked my bookmarks, I do have the linkhttp://gamesfromwithin.com/physical-structure-and-c-part-2-build-times;Quote:Original post by Adam_42P1. Hard drive speed isn't the issue. It's the speed that the compiler can process all that extra code that's slowing it down. The .pch file stores the state of the compiler at the end of compiling all those header files.Hard drive speed is one of the biggest issues. The speed of preprocessing is largely IO-bound, not CPU-bound, and those header files aren't all stored together on the disk. There's a whole lot of seeking going on. And remember, every #include <someheader> entails searching the contents of several directories.Quote:Why are precompiled header files not generated automatically for all header files?The effect of including a header file on the compiler's internal structures is dependent on all the code that was encountered before the header file was included. So an individual header can't be precompiled fully... the best you could hope to do would be a bit of tokenization on preprocessor directives. As Adam_42 said, precompiled header doesn't just save a "compiled version of the header"; it saves the state of the compiler just after including all of those headers. That, BTW, is why the precompiled header directive has to come before any other code: because any code before the directive could possibly change how the compiler would have interpreted the various headers that come after.   ;Quote:Original post by Adam_42P1. Hard drive speed isn't the issue. It's the speed that the compiler can process all that extra code that's slowing it down. The .pch file stores the state of the compiler at the end of compiling all those header files.Hard disk *is* the issue.For most compilation units, pre-processing takes 90% of entire compilation time. This is why unity builds are used to speed things up.One of biggest culprits is the synchronous nature of searching for defines, smaller part goes to actual pre-processor parsing.Quote: Does Eclipse or Netbeans offer any assistance in using pch's in their projects settings with gcc/g++?Try ccache first.   ;  CCache looks very promising on the surface.  How much have you used this?   ;  Just to put a fine point on the hard drive issue, SlimDX takes 5.5 mins to build on a mid range 7200RPM drive (ie not a 640 Black but a nice SATA 3 drive) and about 2 on an SSD. It is using PCH.   ;  I've always assumed that since .cpp and .h files are relatively small they should end up in the operating systems disk cache, especially for headers that are included in most compilation units. Performance of doing a find in files on the whole solution seems to agree with this - I notice the slowdown when they occasionally drop out of the cache.Also if it was I/O bound using /MP to get the build done on multiple CPU cores wouldn't speed things up much.Of course drive speed is still important for other parts of the build process - compilation generates a lot of fairly large intermediate files, and linking is usually I/O bound with LTCG off.   