Added options.extensions and options.compilers

`options` are now passed along to `resolve` module Updated README documentation to explain `extensions` and `compilers` options Added CHANGELOG Push to 2.0.0

Author: bminer

CHANGELOG.md

@@ -0,0 +1,20 @@
+# 2.0.0
+
+Almost a complete re-write of the original project.
+
+- Supports streaming CommonJS modules to a Readable stream
+- Supports bundling for the browser, Node.js, and others
+- Added Promise support
+- Dropped support for Node < 4
+- Changed `options` a bit (i.e. `excludeNodeModules` instead of
+`includeNodeModules`)
+- Added `options.extensions` and `options.compilers` (see README)
+- Changed format of `cb`: now `cb(err, stats)` where `stats` exposes files
+included / excluded in the project build
+- Now depends on NPM `resolve` package
+- Fixed a few bugs
+- Updated docs and added a bit more complexity to the test_project
+
+# 1.x
+
+It's old now, and I'm too lazy to fill in the changelog. :)

README.md

@@ -1,12 +1,18 @@
 # node-module-concat
-CommonJS module concatenation library
+Fairly lightweight CommonJS module concatenation tool
 
 ## What is it?
-This library exposes a single function that concatenates CommonJS modules
-within a project. This can be used to obfuscate an entire project into a
-single file. It can also be used to write client-side JavaScript code where
+This library exposes a single function and stream API that concatenates CommonJS
+modules within a project. This can be used to obfuscate an entire project into
+a single file. It can also be used to write client-side JavaScript code where
 each file is written just like a Node.js module.
 
+## Why?
+Because projects like Webpack and Browserify are cool, but they are a little
+heavy for my taste. I just wanted something to compile CommonJS modules into a
+single JavaScript file. This project has one dependency:
+[resolve](https://github.com/substack/node-resolve)
+
 ## Install
 
 `npm install node-module-concat`
@@ -32,25 +38,66 @@ Constructs a [Readable Stream](https://nodejs.org/api/stream.html#stream_class_s
 of the concatenated project.
 - `entryModulePath` - the path to the entry point of the project to be
 concatenated. This might be an `index.js` file, for example.
-- `options` - object to specify any of the following options
+- `options` - object to specify any of the following options:
 - `outputPath` - the path where the concatenated project file will be
 written. Provide this whenever possible to ensure that instances
 of `__dirname` and `__filename` are replaced properly. If
 `__dirname` and `__filename` are not used in your project or your
 project dependencies, it is not necessary to provide this path. This
-has no effect when `browser` option is set.
+has no effect when the `browser` option is set.
 - `excludeFiles` - An Array of files that should be excluded from the
 project even if they were referenced by a `require(...)`.
 
 Note: These `require` statements should probably be wrapped with a
 conditional or a try/catch block to prevent uncaught exceptions.
 - `excludeNodeModules` - Set to `true` if modules loaded from
 `node_modules` folders should be excluded from the project.
+- `extensions` - An Array of extensions that will be appended to the
+required module path to search for the module in the file system.
+Defaults to `[".js", ".json"]`.
+
+For example, `require("./foo")` will search for:
+- `./foo`
+- `./foo.js`
+- `./foo.json`
+in that order, relative to the file containing the require statement.
+
+Another example, `require("./foo.js")` will search for:
+- `./foo.js`
+- `./foo.js.js`
+- `./foo.js.json`
+
+**Note**: ".node" file extensions are considered to be native C/C++
+addons and are always excluded from the build.
+- `compilers` - An Object describing how files with certain file extensions
+should be compiled to JavaScript before being included in the project.
+The example below will allow node-module-concat to handle `require`
+statements pointing to *.coffee files (i.e. `require("./foo.coffee")`).
+These modules are compiled using the coffee-script compiler before
+they are included in the project.
+```javascript
+{
+".coffee": (src, options) => require("coffee-script").compile(src)
+}
+```
+`options` are passed along to the compiler function, as shown above.
+
+**Note**: By default, ".json" files are prepended with
+`module.exports = `. This behavior can be overwritten by explicitly
+specifying the ".json" key in the `compilers` Object.
+
+**Note**: By default, the file extensions specified in `compilers` are
+not added to the `extensions` option, so `require("./foo")` will not
+find `./foo.coffee` unless ".coffee" is explicitly added to `extensions`
+(see above).
 - `browser` - Set to `true` when concatenating this project for the
 browser. In this case, whenever a required library is loaded from
 `node_modules`, the `browser` field in the `package.json` file (if
 found) is used to determine which file to actually include in the
 project.
+- Any [option supported by resolve.sync]
+(https://github.com/substack/node-resolve#resolvesyncid-opts) except
+`basedir` and `packageFilter`, which can be overwritten.
 - Any [option supported by the Readable class]
 (https://nodejs.org/api/stream.html#stream_new_stream_readable_options)
 

lib/moduleConcatStream.js

@@ -63,23 +63,12 @@ API
 - `entryModulePath` - the path to the entry point of the project to be
 concatenated. This might be an `index.js` file, for example.
 - `options` - object to specify any of the following options
-- `outputPath` - the path where the concatenated project file will be
-written. Provide this whenever possible to ensure that instances
-of `__dirname` and `__filename` are replaced properly. If
-`__dirname` and `__filename` are not used in your project or your
-project dependencies, it is not necessary to provide this path.
-- `excludeFiles` - An Array of files that should be excluded from the
-project even if they were referenced by a `require(...)`.
-
-Note: These `require` statements should probably be wrapped with a
-conditional or a try/catch block to prevent uncaught exceptions.
-- `excludeNodeModules` - Set to `true` if modules loaded from
-`node_modules` folders should be excluded from the project.
-- `browser` - Set to `true` when concatenating this project for the
-browser. In this case, whenever a required library is loaded from
-`node_modules`, the `browser` field in the `package.json` file (if
-found) is used to determine which file to actually include in the
-project.
+- `outputPath`
+- `excludeFiles`
+- `excludeNodeModules`
+- `extensions`
+- `compilers`
+- `browser`
 
 See README.md for more details.
 */
@@ -95,6 +84,29 @@ class ModuleConcatStream extends Readable {
 opts.excludeFiles[i] = path.resolve(opts.excludeFiles[i]);
 }
 }
+// Configure default file extensions
+if(!Array.isArray(opts.extensions) ) {
+opts.extensions = opts.extensions ? [opts.extensions] :
+[".js", ".json"];
+}
+// Add ".node" to the list to allow us to find native modules
+opts.extensions.push(".node");
+// Configure default compilers
+opts.compilers = opts.compilers || {};
+if(typeof opts.compilers[".json"] === "undefined") {
+opts.compilers[".json"] =
+(src, options) => "module.exports = " + src;
+}
+/* Use package.json `browser` field instead of `main` if `browser`
+option is set */
+if(opts.browser) {
+opts.packageFilter = (parsedPkgJson, pkgPath) => {
+if(parsedPkgJson.browser) {
+parsedPkgJson.main = parsedPkgJson.browser;
+}
+return parsedPkgJson;
+};
+}
 // List of files already included in the project or pending inclusion
 this._files = [entryModulePath];
 // Index pointing to the next file to included in the project
@@ -105,12 +117,9 @@ class ModuleConcatStream extends Readable {
 this._headerWritten = false;
 }
 
-// Called when we should start/continue processing
+/* Called when we should start/continue processing.
+We should stop processing whenever `this.push` returns `false`. */
 _read(size) {
-this._continueProcessing();
-}
-
-_continueProcessing() {
 // Write the project header
 if(!this._headerWritten) {
 this._headerWritten = true;
@@ -119,23 +128,30 @@ class ModuleConcatStream extends Readable {
 }
 // Write the next file in the project
 while(this._fileIndex < this._files.length) {
-if(!this._addFile(this._files[this._fileIndex]) )
+if(!this.push(this._addFile(this._files[this._fileIndex])) ) {
 return;
+}
 }
 // Write the project footer
 this.push(FOOTER);
 // Write EOF
 this.push(null);
 }
 
-/* Adds the file from the given `filePath` to the project. Returns `true`
-if more data can be added to the stream; `false` otherwise. */
+/* Adds the file from the given `filePath` to the project. Returns the
+modified module contents (with header/footer added), which should be
+added to the stream. */
 _addFile(filePath) {
 try {
 // Read the file synchronously from disk
 let code = fs.readFileSync(filePath, {"encoding": "utf8"});
 // Mark this file as included in the project
 this._fileIndex++;
+// Compile this file if needed
+let compiler = this._options.compilers[path.extname(filePath)];
+if(compiler) {
+code = compiler(code, this._options);
+}
 // Remove some line comments from code
 code = code.replace(/(?:\r\n?|\n)\s*\/\/.*/g, "");
 /* Scan file for `require(...)`, `__dirname`, and `__filename`
@@ -170,24 +186,11 @@ class ModuleConcatStream extends Readable {
 return match;
 }
 // Get ready to resolve the module
-var resolveOpts = {
-"basedir": path.dirname(filePath),
-"extensions": ["", ".js", ".json", ".node"]
-};
-/* Use package.json `browser` field instead of `main` if
-`browser` option is set */
-if(this._options.browser) {
-resolveOpts.packageFilter = (parsedPkgJson, pkgPath) => {
-if(parsedPkgJson.browser) {
-parsedPkgJson.main = parsedPkgJson.browser;
-}
-return parsedPkgJson;
-};
-}
+this._options.basedir = path.dirname(filePath);
 // Thank you, node-resolve for making this easy!
 try {
 // Resolve the module path
-modulePath = resolve.sync(modulePath, resolveOpts);
+modulePath = resolve.sync(modulePath, this._options);
 // Do not replace core modules
 if(resolve.isCore(modulePath) ) {
 return match;
@@ -201,10 +204,11 @@ class ModuleConcatStream extends Readable {
 // Lookup this module's ID
 var index = this._files.indexOf(modulePath);
 if(index < 0) {
-// Not found; add this module to the project
+// Not added to project yet; try to add it
 if(!this._options.excludeFiles ||
 this._options.excludeFiles.indexOf(modulePath) < 0)
 {
+// Not excluded, so we're good to go!
 index = this._files.push(modulePath) - 1;
 }
 else {
@@ -233,28 +237,21 @@ class ModuleConcatStream extends Readable {
 path.relative(path.dirname(outputPath), filePath)
 ) + ")");
 }
-/* Prepend module header and append module footer and write the
-included module to the stream */
-return this.push(
-// Add file header
-FILE_HEADER
+/* Return the modified module contents, prepending the module header
+and appending the module footer. */
+return FILE_HEADER
 .replace(/\$\{id\}/g, this._files.indexOf(filePath) )
 .replace(/\$\{path\}/g, filePath) +
-// Add extra header bit if this is a JSON file
-(path.extname(filePath) === ".json" ?
-"module.exports = " : "") +
-// Add modified project file
 code +
-// Add file footer
 FILE_FOOTER
 .replace(/\$\{id\}/g, this._files.indexOf(filePath) )
-.replace(/\$\{path\}/g, filePath)
-);
+.replace(/\$\{path\}/g, filePath);
 } catch(err) {
 process.nextTick(() => {
 this.emit("error", err);
 });
-return false;
+// Return EOF
+return null;
 }
 }
 

package.json

@@ -1,7 +1,7 @@
 {
 "name": "node-module-concat",
-"version": "2.0.0-beta",
-"description": "CommonJS module concatenation library",
+"version": "2.0.0",
+"description": "Lightweight CommonJS module concatenation tool",
 "main": "index.js",
 "dependencies": {
 "resolve": ">=1.2 <2"

test_project/index.js

@@ -21,3 +21,4 @@ if(require.main === module) {
 console.log("index is **NOT** main!");
 }
 console.log(cool);
+console.log("node-module-concat version:", require("../package.json").version);