I will visit a client's industry location for a few hours and have to copy the data from his mongoDB server. The data is currently above 800gb in size. I tried to replicate the data remotely but it is very slow, and hence I will visit the physical location to clone the data. Few options I found are LVM, mongodump, mongoexport, and until now LVM seems the best option. Is there a js query that can solve this situation or what might be the best course of action for this? Thanks.
- What is the operating system? It seems to be Linux.Mircea Vutcovici– Mircea Vutcovici2019-10-09 17:58:14 +00:00Commented Oct 9, 2019 at 17:58
- Yes it is Linux.Rahul jyoti nath– Rahul jyoti nath2019-10-09 17:59:32 +00:00Commented Oct 9, 2019 at 17:59
- What distribution? and which version?Mircea Vutcovici– Mircea Vutcovici2019-10-09 18:01:37 +00:00Commented Oct 9, 2019 at 18:01
- I am not sure. I will only get to know when I am there. Is there a generic solution where the distribution and version does not matter?Rahul jyoti nath– Rahul jyoti nath2019-10-09 18:03:04 +00:00Commented Oct 9, 2019 at 18:03
- You need to know if the data is on LVM. Also you need to know if you need to copy the data online, or you can take the volume offline.Mircea Vutcovici– Mircea Vutcovici2019-10-09 18:11:50 +00:00Commented Oct 9, 2019 at 18:11
| Show 1 more comment
1 Answer
Make a mirror partition or disk as a LVM PV (physical volume) and make sure both are in sync. Then just take out one of the disks with you.
Before doing any changes in production, experiment with 2 VMs. One that is the "local VM" and another that is "client VM". Have a disk volume that you detach from the first VM with the mirror, then attach it to the 2nd VM. Document all steps and all troubleshooting steps.
By LVM I mean Linux device mapper based Logical Volume Manager
If the mongodb data is not on LVM, or speed is most important limiting factor and you can take the disks offline, you can:
- stop all services using any partition/volume from the volume you want to mirror
- unmount all partitions from the disk you need to copy or at least the mongodb one
- copy using
dd if=/dev/sdX of=/dev/sdY, where sdX is the source disk and sdY is the destination disk. Make sure you understand theddcommand and what a block device is before using it blindly.
- I have an external hard drive that I will take with me to the location. And I will have a few hours in my hand. Will this time be enough to make the mirror partition LVM ?Rahul jyoti nath– Rahul jyoti nath2019-10-09 18:05:14 +00:00Commented Oct 9, 2019 at 18:05
- You need USB3.0 but, if possible, take the disk out of enclosure and attach it to a SATA interface. It should take be enough 2h, but depends on a lot of factors. Also check
--minrecoveryrateoption.Mircea Vutcovici– Mircea Vutcovici2019-10-09 18:10:30 +00:00Commented Oct 9, 2019 at 18:10 - access.redhat.com/documentation/en-us/red_hat_enterprise_linux/…Mircea Vutcovici– Mircea Vutcovici2019-10-09 18:10:51 +00:00Commented Oct 9, 2019 at 18:10
- Thank you very much.Rahul jyoti nath– Rahul jyoti nath2019-10-09 18:15:41 +00:00Commented Oct 9, 2019 at 18:15
- Yes we can take the disks offline during that time as it will be closed hours. I think unmounting and copying the data would be the best option.Rahul jyoti nath– Rahul jyoti nath2019-10-09 18:19:11 +00:00Commented Oct 9, 2019 at 18:19