(Since 1.7.0) This utility is a Transform text-to-text stream. It reads text as chunks of Buffer and transforms it into utf8 text respecting multibyte boundaries. In the case of chunks of string, it serves as a pass-through.

This class can be used as a base class for Transform streams dealing with text as input. Parser and jsonl/Parser are based on it. Verifier borrows the technique.

Internally this class uses String Decoder to do its magic.

The simple example (streaming from a file):

const Utf8Stream = require('stream-json/utils/Utf8Stream'); const utf8Stream = new Utf8Stream(); const fs = require('fs'); const pipeline = fs.createReadStream('european-languages.json').pipe(utf8Stream); // clients of this pipeline will not see breaks inside multibyte characters

The module returns the constructor of Utf8Stream. Being a Transform stream it doesn't have any special interfaces. The only thing required is to configure it during the construction.

options is an optional object described in details in node.js' Stream documentation. The writing part of the stream always works in text mode.

The constructor defines the following instance variables:

• this._buffer is a string variable, which keeps the current input. The initial value: ''.

This method is called from _transform() when new data has arrived. It can inspect this._buffer and should clear it after it was processed. By default, this._buffer value is pushed downstream.

callback is the same argument that was passed in _transform() with the same semantics. It should be called according to the documentation: with an error value in case of some error, and with null otherwise.

Users can override this method to produce different effects.

This method can populate this._buffer with the rest of input data. It is meant to be called from _flush().

This is a documented method of the Transform implementation: _flush(). In this class it is defined like that:

_flush(callback) { this._flushInput(); this._processBuffer(callback); }

Users can override it for different side-effects.