MongoDB
MongoDB is a powerful NoSQL database known for its flexible, document-oriented
storage that is ideal for handling large-scale, complex data. MongoDB Atlas (a cloud-based
solution), MongoDB Compass (a GUI for data visualization) and the MongoDB Shell for
command-line operations, users can efficiently perform CRUD operations.
The MongoDB Aggregation Framework enables advanced data analysis
with grouping, filtering and joining capabilities. Perfect for scalable applications, MongoDB
supports diverse operators and is optimized for modern data needs, making it a top choice
for dynamic, high-performance database management.
MongoDB uses BSON (Binary JSON) to store documents and offers robust support for a
diverse range of data types which enables flexible and efficient data management.
Below are the enlisted MongoDB data types:
1. String
This is the most commonly used data type in MongoDB to store data, BSON strings are of UTF-
8. So, the drivers for each programming language convert from data types to the string format
of the language to UTF-8 while serializing and de-serializing BSON. The string must be a valid
UTF-8.
Here, the data type of the value of the name field is a string.
2. Integer
In MongoDB, the integer data type is used to store an integer value. We can store integer data
type in two forms 32-bit signed integer and 64-bit signed integer.
3. Double
The double data type is used to store the floating-point values.
4. Boolean
The boolean data type is used to store either true or false.
Example: In the following example we are storing the final result of the student as pass or fail in
boolean values.
5. Null
The null data type is used to store the null value.
Example: In the following example, the student does not have a mobile number so the number
field contains the value null.
6. Array
The Array is the set of values. It can store the same or different data types values in it. In
MongoDB, the array is created using square brackets([]).
7. Object
Object data type stores embedded documents. Embedded documents are also known as nested
documents. Embedded document or nested documents are those types of documents which
contain a document inside another document.
Example: In the following example, we are storing all the information about a book in an
embedded document.
8. Object Id
Whenever we create a new document in the collection MongoDB automatically creates a
unique object id for that document(if the document does not have it). There is an _id field in
MongoDB for each document. The data which is stored in Id is of hexadecimal format and the
length of the id is 12 bytes which consist:
4-bytes for Timestamp value.
5-bytes for Random values. i.e., 3-bytes for machine Id and 2-bytes for process Id.
3- bytes for Counter
You can also create your own id field, but make sure that the value of that id field must be
unique.
Example: In the following example, when we insert a new document it creates a new unique
object id for it.
9. Undefined
This data type stores the undefined values.
Example: In the following example the type of the duration of the project is undefined.
10. Binary Data
This datatype is used to store binary data.
Example: In the following example the value stored in the binaryValue field is of binary type.
11. Date
Date data type stores date. It is a 64-bit integer which represents the number of milliseconds.
BSON data type generally supports UTC datetime and it is signed. If the value of the date data
type is negative then it represents the dates before 1970. There are various methods to return
date, it can be returned either as a string or as a date object. Some method for the date:
Date(): It returns the current date in string format.
new Date(): Returns a date object. Uses the ISODate() wrapper.
new ISODate(): It also returns a date object. Uses the ISODate() wrapper.
Example: In the following example we are using all the above method of the date:
12. Min & Max key
Min key compares the value of the lowest BSON element and Max key compares the value
against the highest BSON element. Both are internal data types.
Example:
13. Symbol
This data type similar to the string data type. It is generally not supported by a mongo shell, but
if the shell gets a symbol from the database, then it converts this type into a string type.
Example:
14. Regular Expression
This datatype is used to store regular expressions.
Example: In the following example we are storing the regular expression gfg:
17. Timestamp
In MongoDB, this data type is used to store a timestamp. It is useful when we modify our data to
keep a record and the value of this data type is 64-bit. The value of the timestamp data type is
always unique.
Example:
18. Decimal
This MongoDB data type store 128-bit decimal-based floating-point value. This data type was
introduced in MongoDB version 3.4
Example:
]
CRUD Operations in MongoDB
This section covers the basics of working with your MongoDB database using CRUD operations.
You'll learn how to Create, Read, Update, and Delete documents, which will allow us to
efficiently manage your data. Get ready to add, retrieve, modify, and remove information easily
Connect to MongoDB
Mongo/Mongosh
Open a terminal and start the MongoDB shell by typing mongo or mongosh.
Create and Use a Database
use databasename.
e.g: use blogs
Use the already existing database.
Create database from mongo compass if does not exists.
Create Collections
// Create a 'posts' collection
db.createCollection("posts")
// Create a 'users' collection
db.createCollection("users")
Create two collections: posts for storing blog posts and users for storing user information.
Insert Operations
Insert a single document into 'posts' collection
db.posts.insertOne({
title: "Introduction to MongoDB",
content: "MongoDB is a NoSQL database.",
author: "John Doe",
tags: ["mongodb", "nosql", "database"]
})
Insert multiple documents into 'users' collection
db.users.insertMany([
{
username: "johndoe",
email: "johndoe@example.com",
age: 30
},
{
username: "janedoe",
email: "janedoe@example.com",
age: 28
}
])
Update Operations
Update a document in 'users' collection
db.users.updateOne(
{ username: "johndoe" },
{ $set: { age: 31 } }
)
Update multiple documents in 'posts' collection
db.posts.updateMany(
{ tags: "mongodb" },
{ $addToSet: { tags: "database" } }
)
Delete Operations
Delete a document from 'users' collection
db.users.deleteOne({ username: "janedoe" })
Delete multiple documents from 'posts' collection
db.posts.deleteMany({ author: "John Doe" })
Drop the entire 'users' collection
db.users.drop()
Query Operations
Find all documents in 'posts' collection
db.posts.find()
Find one document in 'posts' collection
db.posts.findOne({ title: "Introduction to MongoDB" })
Find and modify a document in 'posts' collection
db.posts.findOneAndUpdate(
{ title: "Introduction to MongoDB" },
{ $set: { content: "MongoDB is a flexible and scalable NoSQL database." } }
)
Find one and delete a document in 'posts' collection
db.posts.findOneAndDelete({ author: "John Doe" })
Find one and replace a document in 'posts' collection
db.posts.findOneAndReplace(
{ title: "Introduction to MongoDB" },
{ title: "MongoDB Overview", content: "A detailed guide to MongoDB." }
)
Query with Projections
Find documents with projection (only return 'title' and 'author' fields)
db.posts.find({}, { title: 1, author: 1 })
Query nested documents (e.g., find users with email ending in '.com')
db.users.find({ "email": /.*\.com$/ })
Query documents with null or missing fields
db.users.find({ email: null })
Show Database Information
// Show available databases
show dbs
// Show collections in the current database
show collections
To see a list of available databases and their collections
MongoDB Operators
MongoDB operators are like tools that help you work with your data effectively. They allow
you to find specific information, make changes and analyze your data in MongoDB.
Mastering these operators gives you the ability to manage and explore your data more efficiently,
uncovering valuable insights along the way.
1. Comparison Operators
Find documents where age is greater than 30 in 'users' collection
db.users.find({ age: { $gt: 30 } })
Find documents where age is less than or equal to 28 in 'users' collection
db.users.find({ age: { $lte: 28 } })
Find documents where title is equal to "MongoDB Overview" in 'posts' collection
db.posts.find({ title: { $eq: "MongoDB Overview" } })
Find documents where age is not equal to 30 in 'users' collection
db.users.find({ age: { $ne: 30 } })
In these queries, we utilize the $gt (greater than), $lt (less than), and $eq (equality)
comparison operators to filter documents based on specific criteria. Additionally, we demonstrate
the $ne (not equal) operator to find documents where a field does not match a specified value.
2. Logical Operators
Find documents where age is greater than 25 AND less than 35 in 'users' collection
db.users.find({ $and: [ { age: { $gt: 25 } }, { age: { $lt: 35 } } ] })
Find documents where username is "johndoe" OR email is "janedoe@example.com" in 'users'
collection
db.users.find({ $or: [ { username: "johndoe" }, { email: "janedoe@example.com" } ] })
Find documents where age is NOT equal to 30 in 'users' collection
db.users.find({ age: { $not: { $eq: 30 } } })
Find documents where age is neither 30 nor 31 in 'users' collection
db.users.find({ age: { $nor: [ { $eq: 30 }, { $eq: 31 } ] } })
We use the $and operator to find documents where multiple conditions must be satisfied
simultaneously. The $or operator is utilized to find documents where at least one of the specified
conditions is met. Using the $not operator, we exclude documents where a specific condition is
true. The $nor operator is used to find documents where none of the specified conditions are met.
3. Arithmetic Operators
Let's Add 5 to the age of all users in 'users' collection
db.users.updateMany({}, { $add: { age: 5 } })
Let's Subtract 2 from the age of users aged 30 in 'users' collection
db.users.updateMany({ age: 30 }, { $subtract: { age: 2 } })
Let's Multiply the age of users by 2 in 'users' collection
db.users.updateMany({}, { $multiply: { age: 2 } })
Let's Divide the age of all users by 2 in 'users' collection
db.users.updateMany({}, { $divide: { age: 2 } })
Let's Calculate the absolute value of the age of all users in 'users' collection
db.users.updateMany({}, { $abs: { age: true } })
We use the $add, $subtract, $multiply, and $divide operators to perform addition, subtraction,
multiplication, and division respectively on numeric fields. The $abs operator calculates the
absolute value of numeric fields.
4. Field Update Operators
Let's Update the age of users to the maximum value of 40 in 'users' collection
db.users.updateMany({}, { $max: { age: 40 } })
Let's Update the age of users to the minimum value of 20 in 'users' collection
db.users.updateMany({}, { $min: { age: 20 } })
Let's Increment the age of users by 1 in 'users' collection
db.users.updateMany({}, { $inc: { age: 1 } })
Let's Multiply the age of users by 1.1 in 'users' collection
db.users.updateMany({}, { $mul: { age: 1.1 } })
We use the $max and $min operators to update fields to the maximum or minimum value
respectively. The $inc operator increments numeric fields by a specified value.
The $mul operator multiplies numeric fields by a specified value.
5. Array Expression Operators
MongoDB Aggregation is a database process that allows us to perform complex data
transformations and computations on collections of documents or rows.
It enables us to group, filter, and manipulate data to produce summarized results.
MongoDB Aggregation is typically carried out using the aggregation pipeline which is a
framework for data aggregation modeled on the concept of data processing pipelines.
Each stage of the pipeline transforms the documents as they pass through it and allows
for operations like filtering, grouping, sorting, reshaping and performing calculations on
the data.
Let's Find documents where 'tags' field is an array in 'posts' collection
db.posts.find({ tags: { $isArray: true } })
Let's Find documents in 'posts' collection where the size of the 'tags' array is 3
db.posts.find({ $expr: { $eq: [{ $size: "$tags" }, 3] } })
Let's Find the first element of the 'tags' array in each document of 'posts' collection
db.posts.aggregate([
{ $project: { firstTag: { $arrayElemAt: ["$tags", 0] } } }
])
Let's Concatenate the 'tags' arrays of all documents in 'posts' collection
db.posts.aggregate([
{ $group: { _id: null, allTags: { $concatArrays: "$tags" } } }
])
Let's Reverse the 'tags' array in all documents of 'posts' collection
db.posts.updateMany({}, { $reverseArray: "$tags" })
We use the $isArray operator to find documents where a field is an array. The $size operator is
used to find documents based on the size of an array field. With $arrayElemAt, we retrieve a
specific element from an array field. The $concatArrays operator concatenates arrays.
Finally, $reverseArray reverses the elements of an array.
6. Array Update Operators
Let's Remove all occurrences of "mongodb" from the 'tags' array in 'posts' collection
db.posts.updateMany({}, { $pull: { tags: "mongodb" } })
Let's Remove the last element from the 'tags' array in all documents of 'posts' collection
db.posts.updateMany({}, { $pop: { tags: 1 } })
Let's Remove all occurrences of "nosql" and "database" from the 'tags' array in 'posts' collection
db.posts.updateMany({}, { $pullAll: { tags: ["nosql", "database"] } })
Let's Add "newtag" to the end of the 'tags' array in a specific document in 'posts' collection
db.posts.updateOne({ title: "Introduction to MongoDB" }, { $push: { tags: "newtag" } })
Let's Update the 'tags' array in all documents where "mongodb" is present with "updatedtag"
db.posts.updateMany({ tags: "mongodb" }, { $set: { "tags.$": "updatedtag" } })
7. String Expression Operators
Concatenate the 'title' and 'content' fields into a new field 'fullText' in 'posts' collection
db.posts.aggregate([
{
$project: {
fullText: { $concat: ["$title", " ", "$content"] }
}
}
])
Let's Compare the 'title' field case insensitively to "MongoDB" in 'posts' collection
db.posts.find({ $expr: { $eq: [{ $strcasecmp: ["$title", "MongoDB"] }, 0] } })
Let's Convert the 'title' field to uppercase in 'posts' collection
db.posts.updateMany({}, { $set: { title: { $toUpper: "$title" } } })
Let's Convert the 'title' field to lowercase in 'posts' collection
db.posts.updateMany({}, { $set: { title: { $toLower: "$title" } } })
Let's Extract the first 5 characters from the 'title' field in 'posts' collection
db.posts.aggregate([
{ $project: { firstFiveChars: { $substrCP: ["$title", 0, 5] } } }
])
We use the $concat operator to concatenate fields or strings. $strcasecmp compares strings case
insensitive. The $toUpper operator converts a string to uppercase. $toLower converts a string to
lowercase. $substrCP extracts a substring from a string based on code points.
MongoDB Aggregation Framework
To use MongoDB for aggregating data, follow below steps:
1. Connect to MongoDB: Ensure you are connected to your MongoDB instance.
2. Choose the Collection: Select the collection you want to perform aggregation on, such
as students.
3. Define the Aggregation Pipeline: Create an array of stages, like $group to group
documents and perform operations (e.g., calculate the average grade).
4. Run the Aggregation Pipeline: Use the aggregate method on the collection with your
defined pipeline.
Let's Update documents with aggregation pipeline: multiply 'age' field by 2 and store in
'doubleAge' field
db.users.aggregate([
{ $addFields: { doubleAge: { $multiply: ["$age", 2] } } },
{ $out: "users" }
])
Let's Count the number of documents in 'users' collection
db.users.aggregate([
{ $count: "total_users" }
])
Let's Group documents in 'users' collection by 'age' and calculate the count in each group
db.users.aggregate([
{ $group: { _id: "$age", count: { $sum: 1 } } }
])
Let's Perform a left outer join between 'posts' and 'users' collections based on 'author' field
db.posts.aggregate([
{
$lookup: {
from: "users",
localField: "author",
foreignField: "username",
as: "author_info"
}
}
])
Let's Get the first document in each group sorted by 'age' in descending order in 'users' collection
db.users.aggregate([
{ $sort: { age: -1 } },
{ $group: { _id: null, oldestUser: { $first: "$$ROOT" } } }
])
Let's Perform map-reduce operation to calculate the total age of all users
var mapFunction = function () {
emit("totalAge", this.age);
};
var reduceFunction = function (key, values) {
return Array.sum(values);
};
db.users.mapReduce(
mapFunction,
reduceFunction,
{ out: { inline: 1 } }
);
We use various stages such as $addFields, $out, $count, $group, $lookup, $first, and map-
reduce for different aggregation operations.
Aggregation framework allows us to perform complex computations, transformations, and data
analysis on MongoDB collections efficiently.
MongoDB Indexing
Indexing enhances query performance and allows for efficient data retrieval in MongoDB
Let's Create a single field index on the 'username' field in the 'users' collection
db.users.createIndex({ username: 1 })
Let's Get the list of indexes on the 'users' collection
db.users.getIndexes()
Let's Drop the index on the 'username' field in the 'users' collection
db.users.dropIndex("username_1")
Let's Create a compound index on the 'title' and 'content' fields in the 'posts' collection
db.posts.createIndex({ title: 1, content: 1 })
Let's Create a multikey index on the 'tags' array field in the 'posts' collection
db.posts.createIndex({ tags: 1 })
Let's Create a text index on the 'content' field in the 'posts' collection
db.posts.createIndex({ content: "text" })
Let's Create a unique index on the 'email' field in the 'users' collection
db.users.createIndex({ email: 1 }, { unique: true })
We use createIndex() to create various types of indexes, such as single field, compound,
multikey, text, and unique indexes. getIndexes() retrieves the list of indexes on a collection.
dropIndex() drops an index by its name.
Transactions in MongoDB
MongoDB supports multi-document ACID transactions, allowing for atomicity, consistency,
isolation, and durability.
// Start a session
session = db.getMongo().startSession()
// Start a transaction
session.startTransaction()
try {
// Perform operations within the transaction
db.collection1.insertOne({ field1: "value1" }, { session: session })
db.collection2.updateOne({ field2: "value2" }, { $set: { field3: "value3" } }, { session: session
})
// Commit the transaction
session.commitTransaction()
} catch (error) {
// Abort the transaction on error
session.abortTransaction()
}
Data Modeling in MongoDB
Data modeling in MongoDB involves designing schemas and relationships between documents.
// Relationship: Embedding data in documents
db.users.insertOne({
username: "john_doe",
email: "john@example.com",
posts: [
{ title: "Post 1", content: "Content 1" },
{ title: "Post 2", content: "Content 2" }
]
})
// Relationship: Referencing documents
db.comments.insertOne({
user_id: ObjectId("user_id_here"),
post_id: ObjectId("post_id_here"),
content: "Comment content"
})
// Scaling in MongoDB involves sharding, replication, and proper index usage to distribute data
across multiple servers.
Join Operation
This MongoDB aggregation syntax utilizes the $lookup stage to perform a left join between two
collections.
from: This represents the foreign collection, which we want to join to our current
collection.
localField: LocalField specifies the field in the input documents to perform the join
operation.
foreignField: The foreignField specifies the field from the foreign collection to match
with the localField of the input documents.
as: It denotes the name of the field that will contain the join array data in the output
documents.
db.modelName.aggregate([
{
$lookup: {
from: "Collection to Join",
localField: "Field from the input documents",
foreignField: "Field from the documents of the 'from' collection",
as: "Pick a field-name as output"
}
}
]);
db.books.aggregate([
{
$lookup: {
from: "authors",
localField: "authorId",
foreignField: "_id",
as: "author"
}
}
]);
Exercise 1
Title: To understand the concepts of NoSQL Database
Objective: Students will be able to implement the concept of NoSQL Database MongoDB.
1. Write a MongoDB query to display all the documents in the collection hotel.
2. Write a MongoDB query to display the fields hotel_id, name, Borough and cuisine for all
the documents in the collection hotel.
3. Write a MongoDB query to display the fields hotel_id, name, Borough and cuisine, but
exclude the field _id for all the documents in the collection hotel.
4. Write a MongoDB query to display the fields hotel_id, name, Borough and zip code, but
exclude the field _id for all the documents in the collection hotel. hotel
5. Write a MongoDB query to display all the hotel which is in the Borough Bronx.
Exercise 2
Title: To understand the concepts of NoSQL Database
Objective: Students will be able to implement the concept of NoSQL Database MongoDB.
1. Write a MongoDB query to display the next 5 hotels after skipping first 5 which are in
the Borough Bronx.
2. Write a MongoDB query to find the hotels that achieved a score, more than 80 but less
than 100.
3. Write a MongoDB query to find the hotels which locate in latitude value less than -95.75
4. Write a MongoDB query to find the hotels that do not prepare any cuisine of 'American'
and their grade score more than 70 and latitude less than -65.754168.
Exercise 3
Title: To understand the concepts of NoSQL Database
Objective: Students will be able to implement the concept of NoSQL Database MongoDB.
1. Write a MongoDB query to arrange the name of the cuisine in ascending order and for
that same cuisine Borough should be in descending order.
2. Write a MongoDB query to know whether all the addresses contains the street or not.
3. Write a MongoDB query which will select all documents in the hotels collection where
the coord field value is Double.
4. Write a MongoDB query which will select the hotel Id, name and grades for those hotels
which returns 0 as a remainder after dividing the score by 7.
5. Write a MongoDB query to find the hotel name, Borough, longitude and attitude and
cuisine for those hotels which contains 'mon' as three letters somewhere in its name.
6. Write a MongoDB query to find the hotel name, Borough, longitude and latitude and
cuisine for those hotels which contain 'Mad' as first three letters of its name.
Exercise 4
Title: To understand the concepts of NoSQL Database
Objective: Students will be able to implement the concept of NoSQL Database MongoDB.
1. Write a MongoDB query to find the hotels which do not prepare any cuisine of 'American' and
achieved a score more than 70 and located in the longitude less than -65.754168.
2. Write a MongoDB query to find the hotels which do not prepare any cuisine of 'American '
and achieved a grade point 'A' not belongs to the Borough Brooklyn. The document must be
displayed according to the cuisine in descending order.
3. Write a MongoDB query to find the hotel Id, name, Borough and cuisine for those hotels
which contain 'ces' as last three letters for its name.
4. Write a MongoDB query to find the hotel Id, name, Borough and cuisine for those hotels
which contain 'Reg' as three letters somewhere in its name.
5. Write a MongoDB query to find the hotels which belong to the Borough Bronx and prepared
either American or Chinese dish.
6. Write a MongoDB query to find the hotel Id, name, Borough and cuisine for those hotels
which belong to the Borough Staten Island or Queens or Hyatt.
7. Write a MongoDB query to find the hotel Id, name, Borough and cuisine for those hotels
which are not belonging to the Borough New Delhi or Queens or Hyatt.
8. Write a MongoDB query to find the hotel Id, name, Borough and cuisine for those hotels
which achieved a score which is not more than 10.
9. Write a MongoDB query to find the hotel Id, name, Borough and cuisine for those hotels
which prepared dish except 'American' and 'Chinees' or hotel's name begins with letter 'Wil'.
10. Write a MongoDB query to find the hotel Id, name, and grades for those hotels which
achieved a grade of "A" and scored 11 on an ISODate "2014-08-11T00:00:00Z" among many of
survey dates.
Sharding
Sharding is a method for distributing data across multiple machines. MongoDB uses sharding to
support deployments with very large data sets and high throughput operations.
Database systems with large data sets or high throughput applications can challenge the capacity
of a single server. For example, high query rates can exhaust the CPU capacity of the server.
Working set sizes larger than the system's RAM stress the I/O capacity of disk drives.
There are two methods for addressing system growth: vertical and horizontal scaling.
Vertical Scaling involves increasing the capacity of a single server, such as using a more
powerful CPU, adding more RAM, or increasing the amount of storage space. Limitations in
available technology may restrict a single machine from being sufficiently powerful for a given
workload.
Horizontal Scaling involves dividing the system dataset and load over multiple servers, adding
additional servers to increase capacity as required. While the overall speed or capacity of a single
machine may not be high, each machine handles a subset of the overall workload, potentially
providing better efficiency than a single high-speed high-capacity server. Expanding the capacity
of the deployment only requires adding additional servers as needed, which can be a lower
overall cost than high-end hardware for a single machine. The trade off is increased complexity
in infrastructure and maintenance for the deployment.
Sharded Cluster
A MongoDB sharded cluster consists of the following components:
Shard: A shard is a replica set that contains a subset of the cluster's data. Once sharding is
enabled, which happens when a single MongoDB instance is unable to handle the large dataset,
MongoDB splits the desired collections into multiple shards to achieve horizontal scaling. Each
shard has a primary and one or more secondary shards. The primary shard is responsible for the
writes and replicates the same to the secondary shards.
Mongos instance: The mongos instance acts as a query router for client applications, handling
both read and write operations. It dispatches client requests to the relevant shards and aggregates
the result from shards into a consistent client response. Clients connect to a mongos, not to
individual shards.
Config server replica set: A config server replica set consists of multiple MongoDB replica set
members. They are the authoritative source of sharding metadata. The sharding metadata reflects
the state and organization of the sharded data. The metadata contains the list of sharded
collections, routing information, etc.
The following graphic describes the interaction of components within a sharded cluster:
Advantages of sharding
Sharding allows you to scale your database to handle increased load to a nearly unlimited degree
by providing increased read/write throughput, storage capacity, and high availability. Let’s look
at each of those in a little more detail.
Increased read/write throughput — By distributing the dataset across multiple shards,
both read and write operation capacity is increased as long as read and write operations
are confined to a single shard.
Increased storage capacity — Similarly, by increasing the number of shards, you can also
increase overall total storage capacity, allowing near-infinite scalability.
High availability — Finally, shards provide high availability in two ways. First, since
each shard is a replica set, every piece of data is replicated. Second, even if an entire
shard becomes unavailable since the data is distributed, the database as a whole still
remains partially functional, with part of the schema on different shards.
Disadvantages of sharding
Sharding does come with several drawbacks, namely overhead in query result compilation,
complexity of administration, and increased infrastructure costs.
Query overhead — Each sharded database must have a separate machine or service
which understands how to route a querying operation to the appropriate shard. This
introduces additional latency on every operation. Furthermore, if the data required for the
query is horizontally partitioned across multiple shards, the router must then query each
shard and merge the result together. This can make an otherwise simple operation quite
expensive and slow down response times.
Complexity of administration — With a single unsharded database, only the database
server itself requires upkeep and maintenance. With every sharded database, on top of
managing the shards themselves, there are additional service nodes to maintain. Plus, in
cases where replication is being used, any data updates must be mirrored across each
replicated node. Overall, a sharded database is a more complex system which requires
more administration.
Increased infrastructure costs — Sharding by its nature requires additional machines and
compute power over a single database server. While this allows your database to grow
beyond the limits of a single machine, each additional shard comes with higher costs. The
cost of a distributed database system, especially if it is missing the proper optimization,
can be significant.
Pagination
The limit() method in MongoDB is used to specify the maximum number of documents to return
from a query. The skip() method, on the other hand, is used to skip over a specified number of
documents from the start of the result set. Combined, these methods provide a way to paginate
through documents. Let’s start with some basic examples then move to more advanced scenarios.
Basic Pagination with limit() and skip()
db.collection.find({}).limit(10);
The above code snippet returns the first 10 documents of a collection. Now, to fetch the next set
of documents.
db.collection.find({}).skip(10).limit(10);
This skips the first 10 documents, effectively giving you page 2 of the data.
E.g
const no_of_docs_each_page = 2; // 2 docs in single page
const current_page_number = 2; // 3rd page
db.collection.find({}).skip(no_of_docs_each_page * current_page_number).limit(no_of_docs_each_page)
Just like the find query, in aggregation queries also there’s a similar skip & limit pattern. The
difference is here both skip and limit are two different stages in the aggregation pipelines unlike
the cursor methods in find queries. The $skip stage skips over the specified number
of documents that pass into the stage and passes the remaining documents to the next stage in
the pipeline. — MongoDB Documentation. We have to use another stage, $limit which specifies
how many documents should be passed to the next stage.
Code Example
const no_of_docs_each_page = 2; // 2 docs in single page
const current_page_number = 2; // 3rd pagedb.collection.aggregate([
{ $skip : no_of_docs_each_page * current_page_number },
{ $limit : no_of_docs_each_page }
]);
