title: Comparing many files in Bash
body: <p>I'm trying to automate a task at work that I normally do by hand, that is taking database output from the permissions of multiple users and comparing them to see what they have in common. I have a script right now that uses comm and paste, but it's not giving me all the output I'd like.</p>
  
  <p>Part of the problem comes in comm only dealing with two files at once, and I need to compare at least three to find a trend. I also need to determine if two out of the three have something in common, but the third one doesn't have it (so comparing the output of two comm commands doesn't work). I need these in comma separated values so it can be imported into Excel. Each user has a column, and at the end is a listing of everything they have in common. comm would work perfectly if it could compare more than two files (and show two-out-of-three comparisons).</p>
  
  <p>In addition to the code I have to clean all the extra cruft off the raw csv file, here's what I have so far in comparing four users. It's highly inefficient, but it's what I know.</p>
  
  <pre><code>cat foo1 | sort &gt; foo5
  cat foo2 | sort &gt; foo6
  cat foo3 | sort &gt; foo7
  cat foo4 | sort &gt; foo8
  
  comm foo5 foo6 &gt; foomp
  comm foo7 foo8 &gt; foomp2
  
  paste foomp foomp2 &gt; output2
  sed 's/[\t]/,/g' output2 &gt; output4.csv
  cat output4.csv
  </code></pre>
  
  <p>Right now this outputs two users, their similarities and differences, then does the same for another two users and pastes it together. This works better than doing it by hand, but I know I could be doing more.</p>
  
  <p>An example input file would be something like:</p>
  
  <p>User1</p>
  
  <pre><code>Active Directory
  Internet
  S: Drive
  Sales Records
  </code></pre>
  
  <p>User2</p>
  
  <pre><code>Active Directory
  Internet
  Pricing Lookup
  S: Drive
  </code></pre>
  
  <p>User3</p>
  
  <pre><code>Active Directory
  Internet
  Novell
  Sales Records
  </code></pre>
  
  <p>where they have AD and Internet in common, two out of three have sales records access and S: drive permission, only one of each has Novell and Pricing access.</p>
  
  <p>Can someone give me a hand in what I'm missing?</p>