! grep 'Annie Cyprus' data/SampleSuperstoreSales.csv | head -3

"""
Explanation: CSV command-line kung fu
You might be surprised how much data slicing and dicing you can do from the command line using some simple tools and I/O redirection + piping. (See A Quick Introduction to Pipes and Redirection). We've already seen I/O redirection where we took the output of a command and wrote it to a file (/tmp/t.csv):
bash
$ iconv -c -f utf-8 -t ascii SampleSuperstoreSales.csv &gt; /tmp/t.csv
Set up
bash
pip install csvkit
Extracting rows with grep
Now, let me introduce you to the grep command that lets us filter the lines in a file according to a regular expression. Here's how to find all rows that contain Annie Cyprus:
End of explanation
"""


! grep 'Annie Cyprus' data/SampleSuperstoreSales.csv > /tmp/Annie.csv
! head -3 /tmp/Annie.csv # show first 3 lines of that new file

"""
Explanation: We didn't have to write any code. We didn't have to jump into a development environment or editor. We just asked for the sales for Annie. If we want to write the data to a file, we simply redirect it:
End of explanation
"""


! csvgrep -c 1 -r '^80$' -e latin1 data/SampleSuperstoreSales.csv

"""
Explanation: Filtering with csvgrep
csvkit is an amazing package with lots of cool CSV utilities for use on the command line. csvgrep is one of them.
If we want a specific row ID, then we need to use the more powerful csvgrep not just grep. We use a different regular expression that looks for a specific string at the left edge of a line (^ means the beginning of a line, $ means end of line or end of record):
End of explanation
"""


! csvgrep -c 1 -r '^80$' -e latin1 data/SampleSuperstoreSales.csv > /tmp/two.csv # write first row
! csvgrep -c 1 -r '^160$' -e latin1 data/SampleSuperstoreSales.csv >> /tmp/two.csv # append second row
! cat /tmp/two.csv

"""
Explanation: What if you want, say, two different rows added to another file?  We do two greps and a &gt;&gt; concatenation redirection:
End of explanation
"""


! head -1 data/SampleSuperstoreSales.csv

"""
Explanation: Beginning, end of files
If we'd like to see just the header row, we can use head:
End of explanation
"""


! tail +2 data/SampleSuperstoreSales.csv | head -2

"""
Explanation: If, on the other hand, we want to see everything but that row, we can use tail (which I pipe to head so then I see only the first two lines of output):
End of explanation
"""


! grep Technology, data/SampleSuperstoreSales.csv | grep High, | wc -l

"""
Explanation: The output would normally be many thousands of lines here so I have piped the output to the head command to print just the first two rows.  We can pipe many commands together, sending the output of one command as input to the next command.
Exercise
Count how many sales items there are in the Technology product category that are also High order priorities? Hint: wc -l counts the number of lines.
End of explanation
"""


! csvcut -c 12 -e latin1 data/SampleSuperstoreSales.csv | head -10

"""
Explanation: Extracting columns with csvcut
Extracting columns is also pretty easy with csvcut. For example, let's say we wanted to get the customer name column (which is 12th by my count).
End of explanation
"""


! csvcut -c 12 -e latin1 data/SampleSuperstoreSales.csv | tail +2 | head -10

"""
Explanation: Actually, hang on a second. We don't want the Customer Name header to appear in the list so we combine with the tail we just saw to strip the header.
End of explanation
"""


! csvcut -c 12 -e latin1 data/SampleSuperstoreSales.csv | tail +2 | sort | uniq | head -10

"""
Explanation: What if we want a unique list? All we have to do is sort and then call uniq:
End of explanation
"""


! csvcut -c 12,2 -e latin1 data/SampleSuperstoreSales.csv |head -10

"""
Explanation: You can get multiple columns at once in the order specified. For example, here is how to get the sales ID and the customer name together (name first then ID):
End of explanation
"""


! csvcut -c 2 -e latin1 data/SampleSuperstoreSales.csv > /tmp/IDs
! csvcut -c 12 -e latin1 data/SampleSuperstoreSales.csv > /tmp/names
! paste /tmp/names /tmp/IDs | head -10

"""
Explanation: Naturally, we can write any of this output to a file using the &gt; redirection operator. Let's do that and put each of those columns into a separate file and then paste them back with the customer name first.
End of explanation
"""


! csvcut -c 12 -e latin1 data/SampleSuperstoreSales.csv | tail +2 | sort | uniq -c | sort -r -n | head -10

"""
Explanation: Amazing, right?! This is often a very efficient means of manipulating data files because you are directly talking to the operating system instead of through Python libraries. We also don't have to write any code, we just have to know some syntax for terminal commands.
Not impressed yet? Ok, how about creating a histogram indicating the number of sales per customer sorted in reverse numerical order? We've already got the list of customers and we can use an argument on uniq to get the count instead of just making a unique set. Then, we can use a second sort with arguments to reverse sort and use numeric rather than text-based sorting. This gives us a histogram:
End of explanation
"""


! csvcut -c 8 -e latin1 data/SampleSuperstoreSales.csv | tail +2 | sort | uniq -c | sort -r -n | head -10

"""
Explanation: Exercise
Modify the command so that you get a histogram of the shipping mode.
End of explanation
"""


