Mac Pro Memory - Should I use 8 RAM slots in dual-channel mode or 6 RAM slots in triple-channel mode?
The direct answer to your question is you will get marginally higher peak memory bandwidth with 6 matched memory modules running in triple channel mode than you will with 8 matched memory modules in dual channel mode.  However, there are no guarantees that your machine will perform the tasks you use it for any quicker in either mode.  You need a more in-depth analysis than that.  Firstly, here are the memory replacement instructions for your Mac.  In summary, the instructions for your model go like so:


  Dual-processor computers have eight
  memory slots, four per processor. You
  can install 1 GB, 2 GB, or 4 GB DIMMs.
  Each processor’s memory controller has
  three memory channels. DIMM slots 1,
  2, 5, and 6 have their own channels;
  slots 3 and 4 share a channel and
  slots 7 and 8 share a channel. For
  optimal memory bandwidth, all six
  memory channels should be used, and
  memory should be balanced across the
  six channels. Note: Populating slot 4
  or 8 slightly drops maximum memory
  bandwidth, but depending on the
  applications used, overall system
  performance may benefit from the
  larger amount of memory.


If you have          Fill in these slots
Two DIMMs            1 and 2
Three DIMMs          1, 2, and 3
Four DIMMs           1, 2, and 5, 6
Six DIMMs            1, 2, 3 and 5, 6, 7
Eight DIMMs          1, 2, 3, 4 and 5, 6, 7, 8


To determine whether you would benefit from more total RAM or from keeping your RAM in triple channel mode would require benchmarks.  It's difficult without measuring usage to say whether there will be any benefit from adding more RAM.  If you run a compile with Activity Monitor running are you using all the RAM you have?  Is disk I/O particularly high?  If disk I/O is high but your RAM is not fully utilised you may be better off with an SSD hard drive to run your compilations from rather than adding RAM, check for paging to disk during the compile.  The important thing here is to find the bottleneck rather than hoping that adding RAM will make things run quicker.  It may even turn out that the CPU or the compiler is the bottleneck.  If you have two 6-core CPUs, can your compiler use all that processing power?

I know I've probably raised more questions than I've answered but it's worth considering them if you're looking for a performance improvement for your investment.

If you are a programmer you probably understand these concepts anyway but here's a link to a codinghorror article just explaining the process of finding bottlenecks which I think is easy to read.  The OS in this case is windows but the principles are the same.  You can use Activity Monitor or a third party resource monitoring tool to determine the bottleneck.  

For example, to determine whether your system is running out of RAM, fire up Activity Monitor.  Near the bottom of the window, select the tab button labelled 'System Memory' and take a look at the pie chart and figures listed beneath.  Here's an example from my system.  

  

The figures in this screenshot you should be interested in are Free and Page Outs.  Free shows how much RAM is currently available, Page Outs is an indicator that your system is running out of RAM and has had to write temporary data to disk.  Anything above 0 in Page Outs means you could potentially benefit from more RAM.  You could perform whatever activity you expect to push your system with Activity Monitor open and monitor these figures throughout.  This should help you determine whether you will get any benefit from more RAM.  For completeness, here's what each item represents.  


Free: Memory not in use  
Wired: Data which must remain in RAM and can't be moved to disk  
Active: Data which is currently in memory and has been accessed recently  
Inactive: Data which is currently in memory but is no longer being used.  For example, if you recently quit an application, its data is kept in memory for faster relaunch but it will be used by other application if you run out of Free memory.
Used: The sum of Wired, Active and Inactive memory  


Here's a really nice link too for Mac performance questions http://www.macperformanceguide.com/