<p>I'm working an a very large scale projects, where the compilation time is <strong>very</strong> long. What tools can I use (preferably open source) on Linux, to find the most heavily included files and that optimize their useages?
Just to be clearer, I need a tool which will, given the dependencies, show me which headers are the most included. By the way, we do use distributed compiling</p>

[{'answer_id': 80933, 'author': 'EricSchaefer', 'author_id': 8976, 'author_profile': 'https://Stackoverflow.com/users/8976', 'pm_score': 0, 'selected': False, 'text': '<p>IIRC gcc could create dependency files.</p>\n'}, {'answer_id': 80950, 'author': 'Toni Ruža', 'author_id': 6267, 'author_profile': 'https://Stackoverflow.com/users/6267', 'pm_score': 0, 'selected': False, 'text': '<p>You might want to look at distributed compiling, see for example <a href="http://distcc.samba.org/" rel="nofollow noreferrer">distcc</a></p>\n'}, {'answer_id': 80965, 'author': 'INS', 'author_id': 13136, 'author_profile': 'https://Stackoverflow.com/users/13136', 'pm_score': 2, 'selected': False, 'text': '<p>Check out <a href="http://www.xfree86.org/current/makedepend.1.html" rel="nofollow noreferrer">makdepend</a></p>\n'}, {'answer_id': 80975, 'author': 'slicedlime', 'author_id': 11230, 'author_profile': 'https://Stackoverflow.com/users/11230', 'pm_score': 2, 'selected': False, 'text': '<p>Tools like <a href="http://www.doxygen.org" rel="nofollow noreferrer">doxygen</a> (used with the graphviz options) can generate dependency graphs for include files... I don\'t know if they\'d provide enough overview for what you\'re trying to do, but it could be worth trying.</p>\n'}, {'answer_id': 81003, 'author': 'shodanex', 'author_id': 11589, 'author_profile': 'https://Stackoverflow.com/users/11589', 'pm_score': 0, 'selected': False, 'text': "<p>This is not exactly what you are searchng for, and it might not be easy to setup, but may be you could have a look at lxr : lxr.linux.no is a browseable kernel tree.</p>\n\n<p>In the search box, if you enter a filename, it will give you where it is included.\nBut this is still guessing, and it does not track chained dependencies.</p>\n\n<p>Maybe </p>\n\n<pre><code>strace -e trace=open -o outfile make\ngrep 'some handy regex to match header' \n</code></pre>\n"}, {'answer_id': 81014, 'author': 'Community', 'author_id': -1, 'author_profile': 'https://Stackoverflow.com/users/-1', 'pm_score': 1, 'selected': False, 'text': '<p>If you wish to know which files are included most of all, use this bash command:</p>\n\n<blockquote>\n  <p>find . -name \'<em>.cpp\' -exec egrep \'^[:space:]</em>#include[[:space:]]+["&lt;][[:alpha:][:digit:]_.]+[">]\' {} \\;</p>\n  \n  <blockquote>\n    <p>| sort | uniq -c | sort -k 1rn,1<br>\n     | head -20</p>\n  </blockquote>\n</blockquote>\n\n<p>It will display top 20 files ranked by amount of times they were included.</p>\n\n<p>Explanation: The 1st line finds all *.cpp files and extract lines with "#include" directive from it. The 2nd line calculates how many times each file was included and the 3rd line takes 20 mostly included files.</p>\n'}, {'answer_id': 81016, 'author': 'Dominik Grabiec', 'author_id': 3719, 'author_profile': 'https://Stackoverflow.com/users/3719', 'pm_score': 2, 'selected': False, 'text': '<p>Using the Unix philosophy of "gluing together many small tools" I\'d suggest writing a short script that calls gcc with the -M (or -MM) and -MF (OUTFILE) options (As detailed <a href="http://gcc.gnu.org/onlinedocs/gcc-4.3.2/gcc/Preprocessor-Options.html#Preprocessor-Options" rel="nofollow noreferrer">here</a>). That will generate the dependency lists for the make tool, which you can then parse easily (relative to parsing the source files directly) and extract out the required information.</p>\n'}, {'answer_id': 81030, 'author': 'Agnel Kurian', 'author_id': 45603, 'author_profile': 'https://Stackoverflow.com/users/45603', 'pm_score': 2, 'selected': False, 'text': '<p>The answers <a href="https://stackoverflow.com/questions/42308/tool-to-track-include-dependencies">here</a> will give you tools which track #include dependencies. But there\'s no mention of optimization and such.</p>\n\n<p>Aside: The book "Large Scale C++ Software Design" should help.</p>\n'}, {'answer_id': 81065, 'author': 'paxdiablo', 'author_id': 14860, 'author_profile': 'https://Stackoverflow.com/users/14860', 'pm_score': 2, 'selected': False, 'text': '<p>From the root level of the source tree and do the following (\\t is the tab character):</p>\n\n<pre><code>find . -exec grep \'[ \\t]*#include[ \\t][ \\t]*["&lt;][^"&gt;]["&gt;]\' {} \';\'\n    | sed \'s/^[ \\t]*#include[ \\t][ \\t]*["&lt;]//\'\n    | sed \'s/["&gt;].*$//\'\n    | sort\n    | uniq -c\n    | sort -r -k1 -n\n</code></pre>\n\n<p>Line 1 get all the include lines.\nLine 2 strips off everything before the actual filename.\nLine 3 strips off the end of the line, leaving only the filename.\nLine 4 and 5 counts each unique line.\nLine 6 sorts by line count in reverse order.</p>\n'}, {'answer_id': 81075, 'author': 'Joe Hildebrand', 'author_id': 8388, 'author_profile': 'https://Stackoverflow.com/users/8388', 'pm_score': 1, 'selected': False, 'text': '<p>Use <a href="http://ccache.samba.org/" rel="nofollow noreferrer">ccache</a>.  It will hash the inputs to a compilation, and cache the results, which will drastically increase the speed of these sorts of compiles.</p>\n\n<p>If you wanted to detect the multiple includes, so that you could remove them, you could use makedepend as Iulian Șerbănoiu suggests:</p>\n\n<pre><code>makedepend -m *.c  -f - &gt; /dev/null\n</code></pre>\n\n<p>will give a warning for each multiple include.</p>\n'}, {'answer_id': 81353, 'author': 'Johan Moreau', 'author_id': 12033, 'author_profile': 'https://Stackoverflow.com/users/12033', 'pm_score': 1, 'selected': False, 'text': '<p>Bash scripts found in the page aren\'t good solution. It works only on simple project. In fact, in large project, like discribe in header page, C-preprocessor (#if, #else, ...) are often used. Only good software more complex, like <a href="http://www.xfree86.org/current/makedepend.1.html" rel="nofollow noreferrer">makedepend</a> or <a href="http://www.scons.org/" rel="nofollow noreferrer">scons</a> can give good informations. gcc -E can help, but, on large project, its result analysis is a wasting time.</p>\n'}]
['https://Stackoverflow.com/questions/80923', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/12371/']