Suppose I want to find all the matches in compressed text file:$ gzcat file.txt.gz | pv --rate -i 5 | grep some-patternpv --rate used here for measuring pipe throughput. On my machine it's about 420Mb/s (after decompression).Now I'm trying to do parallel grep using GNU parallel.$ gzcat documents.json.gz | pv --rate -i 5 | parallel --pipe -j4 --round-robin grep some-patternNow throughput is dropped to a ~260Mb/s. And what is more intresting parallel process itself is using a lot of CPU. More than grep processes (but less than gzcat).EDIT 1: I've tried different block sizes (--block), as well as different values for -N/-L options. Nothing helps me at this point.What am I doing wrong?
How to use GNU parallel effectively
shell;gnu parallel