MongoDB Aggregation Framework in action !

MongoDB Aggregation Framework in action ! MongoDB User Group - Nantes mardi 20 janvier 2015 @sebprunier

/me Sébastien Prunier Développeur chez SERLI @sebprunier http://blog.sebprunier.com

Sondage (2/2) ? Aggregation Framework

MongoDB Aggregation Framework Opérations simples Map-Reduce Aggregation Pipeline

Opérations simple db.collection.count(filter?) db.collection.distinct(attribute) db.collection.group(query)

Map-Reduce db.collection.mapreduce( mapFunction, reduceFunction, options )

Aggregation Pipeline db.collection.aggregate( [ stage1, stage2, ..., stageN ] )

Aggregation Pipeline json json json json $match $unwind $group

Opérateurs vs. SQL SQL Terms, Functions, and Concepts MongoDB Aggregation Operators WHERE $match GROUP BY $group HAVING $match SELECT $project ORDER BY $sort LIMIT $limit SUM() $sum COUNT() $sum join N/A ~ $unwind / arrays

Campings { "_id" : ObjectId("54bd347880d46d750f7a48c2"), "raking_date" : ISODate("2013-06-06T22:00:00Z"), "publication_date" : ISODate("2013-06-06T22:00:00Z"), "typology" : "CAMPING", "ranking" : "2 étoiles", "category" : "Tourisme", "mention" : "-", "commercial_name" : "CAMPING LA BERGERIE", "address" : "4231 route de Giens", "zip_code" : "83400", "city" : "HYÈRES", "phone" : "494589175", "email" : "info@camping-de-la-bergerie.com", "website" : "www.camping-de-la-bergerie.com", "visit_type" : "-", "capacity" : "-", "number_of_rooms" : "-", "number_of_spaces" : "55", "number_of_housing_units" : "-", "number_of_chambers" : "-" } data.gouv.fr

Campings “ Déterminer le nombre de campings pour chaque niveau de classement (1 étoile, 2 étoiles, etc…) ” $group

Campings db.campings.aggregate([ {$group : {_id : "$ranking", total : {$sum : 1}}} ]) { "_id" : "5 étoiles", "total" : 177 } { "_id" : "4 étoiles", "total" : 940 } { "_id" : "3 étoiles", "total" : 2224 } { "_id" : "2 étoiles", "total" : 1681 } { "_id" : "1 étoile", "total" : 389 }

Campings “ Top 5 des villes avec le plus de campings ” $group $sort $limit $project

Campings db.campings.aggregate([ {$group : {_id : "$city", total : {$sum : 1}}}, {$sort : {total : -1}}, {$limit : 5}, {$project: {_id: 0, city : "$_id", total: 1}} ]) { "city" : "ARGELÈS-SUR-MER" , "total" : 29 } { "city" : "AGDE", "total" : 23 } { "city" : "VIAS", "total" : 20 } { "city" : "SAINT-JEAN-DE-MONTS" , "total" : 20 } { "city" : "LES MATHES", "total" : 17 }

Campings “ Nombre de villes avec seulement un camping ” $group $match $project

Campings db.campings.aggregate([ {$group : {_id : "$city", total : {$sum : 1}}}, {$match : {total : 1}}, {$group: {_id: null, count: {$sum: 1 }}}, {$project: {_id: 0, count: 1}} ]) { "count" : 2802 }

Marvel { "_id" : 4, "title" : "Rogue (2004) #5", "description" : "...", "format" : "Comic", "creators" : { "available" : 6, "items" : [...] }, "characters" : { "available" : 1, "items" : [ { "name" : "Rogue" } ] }, ... } developer.marvel.com

Marvel “ Top 5 des personnages apparaissant dans le plus de bandes dessinées ” $match $project $unwind $group $sort $limit

Marvel “Deconstructs an array field from the input documents to output a document for each element.” $unwind { "name" : "john doe", "tags" : ["A", "B", "C"] } { "name" : "john doe", "tags" : "A" } { "name" : "john doe", "tags" : "B" } { "name" : "john doe", "tags" : "C" } $unwind

Marvel db.comics.aggregate([ {$match : {"characters.returned" : {$gt : 0}}}, {$project : {title : 1, characters : 1}}, {$unwind : "$characters.items"}, {$group : {_id : "$characters.items.name", total : {$sum : 1}}}, {$sort : {total : -1}}, {$limit : 5} ]) {"_id": "Spider-Man","total": 2413} {"_id": "X-Men","total": 2320} {"_id": "Iron Man","total": 1904} {"_id": "Wolverine","total": 1594} {"_id": "Captain America" ,"total": 1367}

Marvel “ Créez la collection des personnages à partir de la collections des bandes dessinées ” $match $project $unwind $group $out

Marvel db.comics.aggregate([ {$match : {"characters.returned" : {$gt : 0}}}, {$project : {title : 1, characters : 1}}, {$unwind : "$characters.items"}, {$group : { _id : "$characters.items.name", total : {$sum : 1}, comics : {$push : {id : "$_id", title : "$title"}}} }, {$out : "characters"} ])

Marvel { "_id": "Frog-Man", "total": 2, "comics": [ { "id": 38126, "title": "Spider-Man: New York Stories (Trade Paperback)" }, { "id": 39753, "title": "Spider-Island: Avengers (2011) #1" } ] }

Geeks { "_id" : ObjectId("54bd3f1a84c2c169160a88d5"), "nom" : "Mark Zuckerberg", "ville" : "Palo Alto", "likes" : [ "Facebook", "Tongues", "PHP" ], "imageUrl" : "static/GIT_HASH/img/geek5.jpg", "location" : { "type" : "Point", "coordinates" : [ -122.1430195, 37.4418834 ] } } code-story.net

Geeks “ Rechercher ce qui rassemble le plus de geeks autour de Paris ” $geoNear $project $unwind $group $match $sort

Geeks db.geeks.aggregate([ { $geoNear: { near: { type: "Point", coordinates: [2.352241, 48.856638] }, distanceField: "distance", spherical: true, maxDistance: 10000 } }, ... ])

Geeks { "_id" : "java", "total" : 8, "friends" : [ { "nom" : "...", "distance" : 3.015741215085397 }, { "nom" : "...", "distance" : 9864.484291991206 }, ... ] }

Index ● Index utilisé pour certains opérateurs, en début de pipeline ○ $match ○ $sort ○ $geoNear ● explain ○ db.collection.aggregate([...], {explain: true})

Index ● Optimisations { $sort: { age : -1 } }, { $match: { status: 'A' } } { $match: { status: 'A' } }, { $sort: { age : -1 } } “ Optimizations are subject to change between releases. “

Sharding ● Découpage du pipeline en deux parties ○ 1ère partie (jusqu’au premier $group ou $sort) exécutée sur chaque shard (certains shards peuvent être exclus grâce à un $match) ○ 2ème partie exécutée sur le shard primaire à partir des résultats consolidés de la première partie

The end Merci pour votre attention ! ● Questions / Réponses ● Partage d’expérience

MongoDB Aggregation Framework in action !

More Related Content

What's hot

Viewers also liked

Similar to MongoDB Aggregation Framework in action !

More from Sébastien Prunier

Recently uploaded

MongoDB Aggregation Framework in action !