# UTF-8 Plumber

Did you ever encounter some strange "question mark" characters in the output while streaming through some third-party module? This here may be your solution: Pipe the input file or request stream through the plumber and the following streams won't receive cut-in-half UTF-8 characters.

## The problem

When using modules like `csv`, `fast-csv` or other modules that transform streams into some other strings, most of these modules don't really handle UTF-8 inputs well.

If a stream cut happens to be between two bytes of an UTF-8 char and the module or stream you use doesn't handle it correctly (for example just appending the chars by using `chunk.toString()`), the result will be two unusable bytes in a string - that means question marks in your output.

## The solution

Just pipe the Utf8Plumber right before such a stream. It will prevent unfinished UTF-8 bytes to leak into the current chunk and provide it in the next chunk.

Here is a small example on how to use it (uppercase the first column in a CSV file):

```
var fs = require('fs');
var csv = require('csv');
var plumber = require('Utf8Plumber');

fs.createReadStream('my-file.csv')
  .pipe(plumber())
  .pipe(csv.parse())
  .pipe(csv.transform(function(record) {
    record[0] = record[0].toUpperCase();
    return record;
  })
  .pipe(csv.stringify())
  .pipe(process.stdout)
```

Add Readme with problem description and example
Add Readme with problem description and example

Signed-off-by: Joern Bernhardt <7b85a41a628204b76aba4326273a3ccc74bd009a@campudus.com>
