++
mongodb-pipeline-builder is a pipeline builder for the db.collection.aggregate, the db.aggregate and the mongoose Model.aggregate methods.
- Simplify pipelines by making them more readable
- Pipelines are easier to edit.
- Pipeline stages appear in an array.
- Sequential stages for documents
All stages except the Out, Merge, GeoNear, ChangeStream, ChangeStreamSplitLargeEvent and Paging stages can appear multiple times in a pipeline.
npm i -S mongodb-pipeline-builder
- Renaming
getPipeline()withbuild() - Added new stages:
ChangeStream,ChangeStreamSplitLargeEvent,Densify,Documents,Fill,ListLocalSessions,ListSampledQueries,ListSearchIndexes,SearchMeta,SetWindowFieldsandShardedDataDistribution. - Added the possibility to insert stages without validation with new
Insertstage. - Checking for non-duplicable stages.
- Replacing the
Payloadsuffix withHelpersuffix - Prefixed with the name of the pipeline stage where they should be used
- Prefixed with the $ symbol
- Rename
MapOperatorto$Map
- To be used if no Paging stage is set
- Removing GetDocs method arguments
- Added new GetElement method to the response object
- To be used exclusively with Paging stage.
Welcome generics! GetResult<Type> and GetPagingResult<Type> now offer the ability to type responses.
const PipelineBuilder = require("mongodb-pipeline-builder").PipelineBuilder; const { LookupEqualityHelper, ProjectOnlyHelper, Field } = require('mongodb-pipeline-builder/helpers'); const { $LessThanEqual, $ArrayElementAt, $Equal, $Expression } = require('mongodb-pipeline-builder/operators');import { PipelineBuilder } from 'mongodb-pipeline-builder'; import { LookupEqualityHelper, ProjectOnlyHelper, Field } from 'mongodb-pipeline-builder/helpers'; import { $LessThanEqual, $ArrayElementAt, $Equal, $Expression } from 'mongodb-pipeline-builder/operators';const myNewPipeline = new PipelineBuilder( 'myPagination', { debug: true } ) .Match( $Expression( $LessThanEqual( '$id', 20 ) ) ) .Project( ProjectOnlyHelper( 'name', 'weight' ) ) .Paging( 5, 3 ) // 5 per page, page 3 .build();is equivalent to
const myNewPipeline = [ { $facet: { docs: [ { $match: { $expr: { $lte: ["$id", 20] } } }, { $project: { _id: 0, name: 1, weight: 1 } }, { $skip: 10 }, { $limit: 5 } ], count: [ { $match: { $expr: { $lte: ["$id", 20] } } }, { $count: "totalElements" } ] } } ];const myNewPipeline = new PipelineBuilder( 'user-skills' ) .Match( $Expression( $Equal( '$id', 123456 ) ) ) .Lookup( LookupEqualityHelper( 'profiles', 'profile', 'id', 'profileId' ) ) .Project( ProjectOnlyHelper( 'firstname', 'lastname', 'email' ) ) .AddFields( Field( 'skills', $ArrayElementAt( '$profile.skills', 0 ) ), Field( 'availability', $ArrayElementAt( '$profile.availability', 0 ) ) ) .Unset( 'profile' ) .build();is equivalent to
const myNewPipeline = [ { $match: { $expr: { $eq: ["$id", 123456] } } }, { $lookup: { from: "profiles", as: "profile", localField: "id", foreignField: "profileId" } }, { $project: { _id: 0, firstname: 1, lastname: 1, email: 1 } }, { $addFields: { skills: { $arrayElemAt: ["$profile.skills", 0] }, availability: { $arrayElemAt: ["$profile.availability", 0] } } }, { $unset: "profile" } ];GetResult<T>(): Promise<GetResultResponse<T>>
GetResult<T>() is an asynchronous method that provides a very easy way to use aggregation responses.
This method returns a GetResultResponse object that contains 3 methods:
GetDocs(): T[]to get all the documents that match the request.GetElement(index: number | 'last'): Tto get a particular document by its index.GetCount(): numberto get the total number of documents found.
const result = await GetResult<DocType>( target, pipeline ); result.GetDocs(); // () => DocType[] result.GetElement(index | 'last'); // () => DocType | undefined result.GetCount(); // () => number$Or
GetResult<DocType>( target, pipeline ).then( result => { result.GetDocs(); // () => DocType[] result.GetElement(index | 'last'); // () => DocType | undefined result.GetCount(); // () => number } );
- A particular document can be retrieved by specifying its index.
- To get the last document, simply provide the string
'last'. - If the specified index is greater than the index of the last document,
GetElement()will return undefined.
// GetDocs() -> [document1, document2, document3, ..., document51] result.GetElement(2); // will return document to index 2, document3 result.GetElement('last'); // will return the last document, document51 result.GetElement(99); // will return undefinedGetPagingResult<T>(): Promise<GetPagingResultResponse<T>>
GetPagingResult<T>() is an asynchronous method that provides a very easy way to use aggregation responses when Paging stage is used.
This method returns a GetPagingResultResponse object that contains three methods:
GetDocs()to get the documents found.GetCount()to get the total number of documents found.GetTotalPageNumber()to get the total number of pages.
const result = await GetPagingResult<DocType>(target, pipeline); result.GetDocs(); // () => DocType[] result.GetCount(); // () => number result.GetTotalPageNumber(); // () => number$Or
GetPagingResult<DocType>(target, pipeline).then( result => { result.GetDocs(); // () => DocType[] result.GetCount(); // () => number result.GetTotalPageNumber(); // () => number } );=> Try the lib on NPM RunKit with the require method <=
// builder = new PipelineBuilder('example');
The Paging stage automatically adds 3 native stages used to paginate documents ($skip, $limit and $count).
Page is optional and defaults to 1.
builder.Paging(5, 2).build(); // pipeline [ { $facet: { docs: [ { '$skip': 5 }, { '$limit': 5 } ], count: [ { '$count': 'totalElements' } ] } } ]The Insert stage allows you to insert a stage without validation.
Usefully when you need to insert a stage that is not yet implemented or when the value fails validation but for some reason you want to keep it.
builder.Insert({ '$myCustomStage': { myField: 'myValue' } }).build(); // pipeline [ { $myCustomStage: { myField: 'myValue' } } ]AddFields(...values)
builder.AddFields( Field('foo', 'value1'), Field('bar', 'value2'), ).build(); // pipeline [ { $addFields: { foo: 'value1', bar: 'value2' } } ]Bucket(value)
builder.Bucket(BucketHelper('$age', [6, 13, 18])).build(); // pipeline [ { $bucket: { groupBy: '$age', boundaries: [ 6, 13, 18 ] } } ]BucketAuto(value)
builder.BucketAuto(BucketAutoHelper('$age', 5)).build(); // pipeline [ { $bucketAuto: { groupBy: '$age', buckets: 5 } } ]ChangeStream(value)
builder.ChangeStream(ChangeStreamHelper()).build(); // pipeline [ { $changeStream: {} } ]builder.ChangeStreamSplitLargeEvent().build(); // pipeline [ { $changeStreamSplitLargeEvent: {} } ]CollStats(value)
builder.CollStats(CollStatsHelper()).build(); // pipeline [ { $collStats: {} } ]Count(value)
builder.Count('counter').build(); // pipeline [ { $count: 'counter' } ]CurrentOp(value)
builder.CurrentOp(CurrentOpHelper()).build(); // pipeline [ { $currentOp: {} } ]Densify(value)
builder.Densify( DensifyHelper( 'altitude', { bounds: 'full', step: 200 }, { partitionByFields: [ 'variety' ] } ), ).build(); // pipeline [ { $densify: { field: 'altitude', range: { bounds: 'full', step: 200 } partitionByFields: [ 'variety' ], } } ]Documents(...values)
builder.Documents({ docId: 1 }, { docId: 2 }, { docId: 3 }).build(); // pipeline [ { $documents: [ { docId: 1 }, { docId: 2 }, { docId: 3 } ] } ]Facet(...values)
builder.Facet( Field('pipeline1', [{ $match: { tag: 'first' }}]), Field('pipeline2', [{ $match: { tag: 'second' }}]), Field('pipeline3', [{ $match: { tag: 'third' }}]), ).build(); // pipeline [ { $facet: { pipeline1: [ { '$match': { tag: 'first' } } ], pipeline2: [ { '$match': { tag: 'second' } } ], pipeline3: [ { '$match': { tag: 'third' } } ] } } ]Fill(value)
builder.Fill( FillHelper({ bootsSold: { value: 0 }, sandalsSold: { value: 0 }, sneakersSold: { value: 0 }, }), ).build(); // pipeline [ { $fill: { output: { bootsSold: { value: 0 }, sandalsSold: { value: 0 }, sneakersSold: { value: 0 } } } } ]GeoNear(value)
builder.GeoNear( GeoNearHelper({ type: "Point", coordinates: [ -73.99279 , 40.719296 ] }, 'calculated') ).build(); // pipeline [ { $geoNear: { near: { type: 'Point', coordinates: [ -73.99279, 40.719296 ] }, distanceField: 'calculated' } } ]GraphLookup(value)
builder.GraphLookup({ from: 'employees', startWith: '$reportsTo', connectFromField: 'reportsTo', connectToField: 'name', as: 'reportingHierarchy', }).build(); // pipeline [ { $graphLookup: { from: 'employees', startWith: '$reportsTo', connectFromField: 'reportsTo', connectToField: 'name', as: 'reportingHierarchy' } } ]Group(value)
builder.Group({ _id: null, count: { $count: { } } }).build(); // pipeline [ { $group: { _id: null, count: { '$count': {} } } } ]builder.IndexStats().build(); // pipeline [ { $indexStats: {} } ]Limit(value)
builder.Limit(10).build(); // pipeline [ { $limit: 10 } ]ListLocalSessions(value)
builder.ListLocalSessions({ allUsers: true }).build(); // pipeline [ { $listLocalSessions: { allUsers: true } } ]ListSampledQueries(value)
builder.ListSampledQueries({ namespace: "social.post" }).build(); // pipeline [ { $listSampledQueries: { namespace: 'social.post' } } ]ListSearchIndexes(value)
builder.ListSearchIndexes({ name: 'searchIndex01' }).build(); // pipeline [ { $listSearchIndexes: { name: 'searchIndex01' } } ]ListSessions(value)
builder.ListSessions({ allUsers: true }).build(); // pipeline [ { $listSessions: { allUsers: true } } ]Lookup(value)
builder.Lookup(LookupConditionHelper('users', 'users')).build(); // pipeline [ { $lookup: { from: 'users', as: 'users' } } ]builder.Lookup( LookupEqualityHelper('users', 'users', 'userId', 'id') ).build(); // pipeline [ { $lookup: { from: 'users', localField: 'userId', foreignField: 'id', as: 'users' } } ]Match(value)
builder.Match(Field('age', 18)).build(); // pipeline [ { $match: { age: 18 } } ]builder.Match($Expression($GreaterThanEqual('$age', 18))).build(); // pipeline [ { $match: { '$expr': { '$gte': [ '$age', 18 ] } } } ]Merge(value)
builder.Merge(MergeHelper('newCollection')).build(); // pipeline [ { $merge: { into: 'newCollection' } } ]Out(value)
builder.Out(OutHelper('users')).build(); // pipeline [ { $out: 'users' } ]builder.PlanCacheStats().build(); // pipeline [ { $planCacheStats: {} } ]Project(...values)
builder.Project( ProjectHelper('age', '$user.age'), ProjectHelper( 'nickname', { $cond: { if: { $eq: [ '', '$user.nickname' ] }, then: '$$REMOVE', else: '$user.nickname', }, }, ), ).build(); // pipeline [ { $project: { age: '$user.age', nickname: { $cond: { if: { $eq: [ '', '$user.nickname' ] }, then: '$$REMOVE', else: '$user.nickname' } } } } ]builder.Project(ProjectIgnoreHelper('password', 'refreshToken')).build(); // pipeline [ { $project: { password: 0, refreshToken: 0 } } ]builder.Project(ProjectOnlyHelper('password', 'refreshToken')).build(); // pipeline [ { $project: { _id: 0, password: 1, refreshToken: 1 } } ]Redact(value)
builder.Redact( $Condition( $GreaterThan($Size($SetIntersection('$tags', ['STLW', 'G'])), 0), '$$DESCEND', '$$PRUNE' ) ).build(); // pipeline [ { $redact: { '$cond': [ { '$gt': [ { '$size': { '$setIntersection': [ '$tags', [ 'STLW', 'G' ] ] } }, 0 ] }, '$$DESCEND', '$$PRUNE' ] } } ]ReplaceRoot(value)
builder.ReplaceRoot({ newRoot: { full_name: { $concat : [ "$first_name", " ", "$last_name" ] } } }).build(); // pipeline [ { $replaceRoot: { newRoot: { full_name: { '$concat': [ '$first_name', ' ', '$last_name' ] } } } } ]ReplaceWith(value)
builder.ReplaceWith('$name').build(); // pipeline [ { $replaceWith: '$name' } ]Sample(value)
builder.Sample(SampleHelper(6)).build(); // pipeline [ { $sample: { size: 6 } } ]Search(value)
builder.Search( SearchHelper({ near: { path: 'released', origin: date, pivot: 7776000000 }, }), ).build(); // pipeline [ { $search: { near: { path: 'released', origin: date, pivot: 7776000000 }, }, } ]SearchMeta(value)
builder.SearchMeta( SearchMetaHelper({ facet: { operator: { near: { path: 'released', origin: date, pivot: 7776000000 }, }, facets: { test: { type: 'number', path: 'released', boundaries: [0, 100] }, }, }, }) ).build(); // pipeline [ { $searchMeta: { facet: { operator: { near: { path: 'released', origin: date, pivot: 7776000000 }, }, facets: { test: { type: 'number', path: 'released', boundaries: [0, 100] }, }, }, } } ]Set(...values)
builder.Set(Field('first', true), Field('second', 2)).build(); // pipeline [ { $set: { first: true, second: 2 } } ]SetWindowFields(value)
builder.SetWindowFields({ partitionBy: "$state", sortBy: { orderDate: 1 }, output: { cumulativeQuantityForState: { $sum: "$quantity", window: { documents: [ "unbounded", "current" ] } } } }).build(); // pipeline [ { $setWindowFields: { partitionBy: '$state', sortBy: { orderDate: 1 }, output: { cumulativeQuantityForState: { '$sum': '$quantity', window: { documents: [ 'unbounded', 'current' ] } } } } } ]ShardedDataDistribution(value)
builder.ShardedDataDistribution({}).build(); // pipeline [ { $shardedDataDistribution: {} } ]Skip(value)
builder.Skip(100).build(); // pipeline [ { $skip: 100 } ]Sort(...values)
builder.Sort( Field('first', -1), Field('second', 1), Field('third', { $meta: "textScore" }), ).build(); // pipeline [ { $sort: { first: -1, second: 1, third: { '$meta': 'textScore' } } } ]SortByCount(value)
builder.SortByCount('$employee').build(); // pipeline [ { $sortByCount: '$employee' } ]UnionWith(value)
builder.UnionWith( UnionWithHelper('cars'), ).build(); // pipeline [ { $unionWith: { coll: 'cars' } } ]builder.UnionWith( UnionWithHelper( undefined, [{ $document: [{ ref: 1 }, { ref: 2 }, { ref: 3 }], }]), ).build(); // pipeline [ { $unionWith: { pipeline: [ { '$document': [ { ref: 1 }, { ref: 2 }, { ref: 3 } ] } ] } } ]builder.UnionWith( UnionWithHelper('cars', [{ $match: { color: 'red' } }]), ).build(); // pipeline [ { $unionWith: { coll: 'cars', pipeline: [ { '$match': { color: 'red' } } ] } } ]Unset(...values)
builder.Unset('users', 'roles').build(); // pipeline [ { $unset: [ 'users', 'roles' ] } ]Unwind(value)
builder.Unwind({ path: '$sizes', preserveNullAndEmptyArrays: true }).build(); // pipeline [ { $unwind: { path: '$sizes', preserveNullAndEmptyArrays: true } } ]$Absolute(-5) // operator { $abs: -5 }$Accumulator( () => ({ count: 0, sum: 0 }), (state: { count: number; sum: number; }, numCopies: number) => ({ count: state.count + 1, sum: state.sum + numCopies, }), ['$copies'], (state1: { count: number; sum: number; }, state2: { count: number; sum: number; }) => ({ count: state1.count + state2.count, sum: state1.sum + state2.sum, }), { finalize: (state: { sum: number; count: number; }) => (state.sum / state.count) }, ) // operator { '$accumulator': { init: [ () => ({ count: 0, sum: 0 }) ], accumulate: [ (state: { count: number; sum: number; }, numCopies: number) => ({ count: state.count + 1, sum: state.sum + numCopies, }) ], accumulateArgs: [ '$copies' ], merge: [ (state1: { count: number; sum: number; }, state2: { count: number; sum: number; }) => ({ count: state1.count + state2.count, sum: state1.sum + state2.sum, }) ], finalize: [ (state: { sum: number; count: number; }) => (state.sum / state.count) ], lang: 'js' } }$ArcCosine({ $divide : [ '$side_b', '$hypotenuse' ] }) // operator { '$acos': { '$divide': [ '$side_b', '$hypotenuse' ] } }$ArcCosineHyperbolic(3) // operator { '$acosh': 3 }$Add('$price', 10) // operator { '$add': [ '$price', 10 ] }$AddToSet('$item') // operator { '$addToSet': '$item' }$AllElementsTrue([ true, 1, "someString" ]) // operator { '$allElementsTrue': [ [ true, 1, 'someString' ] ] }$And(1, 'green') // operator { '$and': [ 1, 'green' ] }$AnyElementTrue([ true, false ]) // operator { '$anyElementTrue': [ [ true, false ] ] }$ArrayElementAt([ 1, 2, 3 ], 0) // operator { '$arrayElemAt': [ [ 1, 2, 3 ], 0 ] }$ArrayToObject([ { "k": "item", "v": "abc123" }, { "k": "qty", "v": "$qty" } ]) // operator { '$arrayToObject': [ { k: 'item', v: 'abc123' }, { k: 'qty', v: '$qty' } ] }$ArrayToObject([ [ "item", "abc123" ], [ "qty", 25 ] ], true) // operator { '$arrayToObject': { '$literal': [ [ 'item', 'abc123' ], [ 'qty', 25 ] ] } }$ArcSine('$value') // operator { '$asin': '$value' }$ArcSineHyperbolic('$value') // operator { '$asinh': '$value' }$ArcTangent('$value') // operator { '$atan': '$value' }$ArcTangent2('$side_b', '$side_a') // operator { '$atan2': [ '$side_b', '$side_a' ] }$ArcTangentHyperbolic('$value') // operator { '$atanh': '$value' }$Average('$value') // operator { '$avg': '$value' }$Average('$value1', '$value2', '$value3') // operator { '$avg': [ '$value1', '$value2', '$value3' ] }$BinarySize('Hello World!') // operator { '$binarySize': 'Hello World!' }$BitwiseAnd('$array') // operator { '$bitAnd': '$array' }$BitwiseAnd(0, 127, 5) // operator { '$bitAnd': [ 0, 127, 5 ] }$BitwiseNot('$long') // operator { '$bitNot': '$long' }$BitwiseOr('$array') // operator { '$bitOr': '$array' }$BitwiseOr(0, 127, 5) // operator { '$bitOr': [ 0, 127, 5 ] }$BitwiseXor('$array') // operator { '$bitXor': '$array' }$BitwiseXor(0, 127, 5) // operator { '$bitXor': [ 0, 127, 5 ] }$Bottom(['field1', 'field2'], { field2: -1 }) // operator { '$bottom': { output: [ 'field1', 'field2' ], sortBy: { field2: -1 } } }$BottomN('field', { field: 1 }, 3) // operator { '$bottomN': { output: 'field', sortBy: { field: 1 }, n: 3 } }$BsonSize('$$ROOT') // operator { '$bsonSize': '$$ROOT' }$Ceil('$value') // operator { '$ceil': '$value' }$Compare('$age', 25) // operator { '$cmp': [ '$age', 25 ] }$Concat('$first', ' - ', '$second') // operator { '$concat': [ '$first', ' - ', '$second' ] }$ConcatArrays('$array', [1, 2, 3]) // operator { '$concatArrays': [ '$array', [ 1, 2, 3 ] ] }$Condition({ $gte: [ '$quantity', 250 ] }, 'true', 'false') // operator { '$cond': [ { '$gte': [ '$quantity', 250 ] }, 'true', 'false' ] }$Convert(100, 'bool') // operator { '$convert': { input: 100, to: 'bool' } }$Cosine('$angle') // operator { '$cos': '$angle' }$CosineHyperbolic({ $degreesToRadians : "$angle" }) // operator { '$cosh': { '$degreesToRadians': '$angle' } }$Count() // operator { '$count': {} }$CovariancePopulation('$numeric1', '$numeric2') // operator { '$covariancePopulation': [ '$numeric1', '$numeric2' ] }$CovarianceSample('$numeric1', '$numeric2') // operator { '$covarianceSample': [ '$numeric1', '$numeric2' ] }$DateAdd('$startDate', 'hour', 2) // operator { '$dateAdd': { startDate: '$startDate', unit: 'hour', amount: 2 } }$DateDifference('$startDate', '$endDate', 'second') // operator { '$dateDiff': { startDate: '$startDate', endDate: '$endDate', unit: 'second' } }$DateFromCalendarParts(2000, { month: 12, day: 31, hour: 12, minute: 25, second: 59, timezone: '+01:00' }) // operator { '$dateFromParts': { year: 2000, month: 12, day: 31, hour: 12, minute: 25, second: 59, timezone: '+01:00' } }$DateFromIsoWeekParts(2000, { isoWeek: 53, isoDayOfWeek: 7, millisecond: 500 }) // operator { '$dateFromParts': { isoWeekYear: 2000, isoWeek: 53, isoDayOfWeek: 7, millisecond: 500 } }$DateFromString('2017-02-08T12:10:40.787', { timezone: 'America/New_York' }) // operator { '$dateFromString': { dateString: '2017-02-08T12:10:40.787', timezone: 'America/New_York' } }$DateSubtract(1697382106124, 'month', 1) // operator { '$dateSubtract': { startDate: 1697382106124, unit: 'month', amount: 1 } }$DateToParts(1697382106124) // operator { '$dateToParts': { date: 1697382106124 } }$DateToString(1697382106124) // operator { '$dateToString': { date: 1697382106124 } }$DateTrunc(1697382106124, 'month') // operator { '$dateTrunc': { date: 1697382106124, unit: 'month' } }$DayOfMonth('$date', 'Europe/Paris') // operator { '$dayOfMonth': { date: '$date', timezone: 'Europe/Paris' } }$DayOfWeek('$date', '+03:30') // operator { '$dayOfWeek': { date: '$date', timezone: '+03:30' } }$DayOfYear('$date') // operator { '$dayOfYear': { date: '$date' } }$DegreesToRadians('$angle_a') // operator { '$degreesToRadians': '$angle_a' }// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator// operator