<p>I have to deal with text files in a motley selection of formats. Here's an example (Columns <strong>A</strong> and <strong>B</strong> are tab delimited):</p>

<pre><code>A   B
a   Name1=Val1, Name2=Val2, Name3=Val3
b   Name1=Val4, Name3=Val5
c   Name1=Val6, Name2=Val7, Name3=Val8
</code></pre>

<p>The files could have headers or not, have mixed delimiting schemes, have columns with name/value pairs as above etc.<br>
I often have the ad-hoc need to extract data from such files in various ways. For example from the above data I might want the value associated with Name2 where it is present. i.e.</p>

<pre><code>A   B
a   Val2
c   Val7
</code></pre>

<p>What tools/techniques are there for performing such manipulations as one line commands, using the above as an example but extensible to other cases?</p>

[{'answer_id': 82282, 'author': 'auramo', 'author_id': 4110, 'author_profile': 'https://Stackoverflow.com/users/4110', 'pm_score': 1, 'selected': False, 'text': '<p>You have all the basic bash shell commands, for example grep, cut, sed and awk at your disposal. You can also use Perl or Ruby for more complex things.</p>\n'}, {'answer_id': 82293, 'author': 'Onorio Catenacci', 'author_id': 2820, 'author_profile': 'https://Stackoverflow.com/users/2820', 'pm_score': 0, 'selected': False, 'text': "<p>From what I've seen I'd start with Awk for this sort of thing and then if you need something more complex, I'd progress to Python.</p>\n"}, {'answer_id': 82300, 'author': 'Cetra', 'author_id': 15087, 'author_profile': 'https://Stackoverflow.com/users/15087', 'pm_score': 0, 'selected': False, 'text': "<p>I would use sed:</p>\n\n<pre><code>   # print section of file between two regular expressions (inclusive)\n   sed -n '/Iowa/,/Montana/p'             # case sensitive\n</code></pre>\n"}, {'answer_id': 82330, 'author': 'Community', 'author_id': -1, 'author_profile': 'https://Stackoverflow.com/users/-1', 'pm_score': 0, 'selected': False, 'text': '<p>Since you have cygwin, I\'d go with Perl. It\'s the easiest to learn (check out the O\'Reily book: <a href="http://books.google.com/books?id=bS--s5DAIHsC&amp;dq=%22learning+perl%22&amp;pg=PP1&amp;ots=udTVanap5Y&amp;sig=gJW55LqGpEXCHqRMgHfTJ8nmP70&amp;hl=en&amp;sa=X&amp;oi=book_result&amp;resnum=1&amp;ct=result" rel="nofollow noreferrer">Learning Perl</a>) and widely applicable.</p>\n'}, {'answer_id': 82353, 'author': 'Weidenrinde', 'author_id': 11344, 'author_profile': 'https://Stackoverflow.com/users/11344', 'pm_score': 2, 'selected': True, 'text': '<p>I don\'t like sed too much, but it works for such things:</p>\n\n<pre><code>var="Name2";sed -n "1p;s/\\([^ ]*\\) .*$var=\\([^ ,]*\\).*/\\1 \\2/p" &lt; filename\n</code></pre>\n\n<p>Gives you:</p>\n\n<pre><code> A B\n a Val2\n c Val7\n</code></pre>\n'}, {'answer_id': 82557, 'author': 'deterb', 'author_id': 15585, 'author_profile': 'https://Stackoverflow.com/users/15585', 'pm_score': 0, 'selected': False, 'text': '<p>I would use Perl.  Write a small module (or more than one) for dealing with the different formats.  You could then run perl oneliners using that library.  Example for what it would\nlook like as follows:</p>\n\n<pre><code>perl -e \'use Parser;\' -e \'parser("in.input").get("Name2");\'\n</code></pre>\n\n<p>Don\'t quote me on the syntax, but that\'s the general idea.  Abstract the task at hand to allow you to think in terms of what you need to do, not how you need to do it.  Ruby would be another option, it tends to have a cleaner syntax, but either language would work.</p>\n'}]
['https://Stackoverflow.com/questions/82268', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/6387/']