13

I have a directory filled with data at /var/backups/disk1 that I want to convert into a virtual disk image which I'll then be able to boot using QEMU or KVM (the directory contains the file system for a virtual machine, copied out via rsync).

While there are plenty of instructions out there for converting a complete physical disk to a virtual disk, packaging up only the contents of a single directory into a virtual disk image is turning out to be much more difficult than I expected. Any ideas?

By the way, I know I can use qemu-img to convert a block device into a virtual disk (e.g., qemu-img convert -f /dev/sdc -O qcow2 disk.qcow2), so if only I could get the directory /var/backups/disk1 to appear to be a block device, then theoretically I should be able to accomplish my goal using qemu-img. I've thought about creative ways to expose the directory as a block device using NBD or a loopback device, however, and have not had success.

6 Answers 6

21

First, create a raw image of the required size. I'll assume 10G is enough. Using seek creates a sparse file, which saves space.

dd if=/dev/null of=example.img bs=1M seek=10240 

Next, create a filesystem on it.

mkfs.ext4 -F example.img 

(Note that you need the -F option for mkfs.ext4 to operate on a file as opposed to a disk partition)

Then, mount it.

mkdir /mnt/example mount -t ext4 -o loop example.img /mnt/example 

Now you can copy your files to /mnt/example. Once this is done, unmount it and you can use example.img as a drive in a virtual machine. If you want you can convert it from a raw image to another format like qcow2e using qemu-img, but this isn't required.

2
6

virt-make-fs from libguestfs qcow2 example

https://serverfault.com/a/332114/163884 mentioned it, but here is a full example:

sudo apt-get install libguestfs-tools # Workarounds for Ubuntu 18.04 bugs. See section below. sudo rm -rf /var/cache/.guestfs-* echo dash | sudo tee /usr/lib/x86_64-linux-gnu/guestfs/supermin.d/zz-dash-packages sudo chmod +r /boot/vmlinuz-* mkdir sysroot # Just a test file. dd if=/dev/urandom of=sysroot/myfile bs=1024 count=1024 virt-make-fs --format=qcow2 --type=ext2 sysroot sysroot.ext2.qcow2 

Note how sudo is not required, except for installation and Ubuntu bug workarounds.

I then verified that QEMU can actually read it with:

qemu-system-x86_64 -drive file=sysroot.ext2.qcow2,format=qcow2,if=virtio,snapshot ... 

I am then able to mount the image inside QEMU Linux and read the file.

virt-make-fs ext example

This awesome tool can also make raw ext filesystems, e.g.:

virt-make-fs --format=raw --type=ext2 sysroot sysroot.ext2 virt-make-fs --format=raw --type=ext4 sysroot sysroot.ext4 

which we can verify directly on the host with:

mkdir -p mnt dev="$(sudo losetup --show -f -P sysroot.ext4)" sudo mount -o loop "$dev" mnt cmp sysroot/myfile mnt/myfile 

Image size minimization

A really good feature of virt-make-fs is that it automatically tries to minimize the image size if that is what we want:

Virt-make-fs defaults to minimizing the extra space, but you can use the --size flag to leave space in the filesystem if you want it.

so:

df -h 

tells me that the image is 82% filled:

/dev/loop17 1.5M 1.1M 244K 82% /home/ciro/test/guestfs/mnt 

We can easily add some extra space on top of the minimum with --size-=+:

virt-make-fs --format=raw --size=+8M --type=ext2 sysroot sysroot.ext2 

ext4 journal overhead

The manual also mentions that:

Note that ext3 filesystems contain a journal, typically 1-32 MB in size. If you are not going to use the filesystem in a way that requires the journal, then this is just wasted overhead.

and it is interesting to verify that with:

du -bs * 

which produces:

1052672 sysroot 1446297 sysroot.ext2 2599731 sysroot.ext4 

so we see that the ext4 came out significantly bigger.

libguestfs Ubuntu bugs

Now for the downside: there appears to be no Ubuntu maintainer currently and the library is generally buggy on Ubuntu.

sudo is not in theory required, but it is required due to an Ubuntu packaging bug unless we do a workaround: https://askubuntu.com/questions/1046828/how-to-run-libguestfs-tools-tools-such-as-virt-make-fs-without-sudo/1046829#1046829

libguestfs: error: /usr/bin/supermin exited with error status 1. To see full error messages you may need to enable debugging. Do: export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 and run the command again. For further information, read: http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs You can also run 'libguestfs-test-tool' and post the *complete* output into a bug report or message to the libguestfs mailing list. libguestfs: error: /usr/bin/supermin exited with error status 1. 

After that, without our workarounds, 18.04 (but not 16.04) fails with: https://bugzilla.redhat.com/show_bug.cgi?id=1591617

libguestfs: error: tar_in: write error on directory: /: 

due to a bug that was already fixed upstream.

Tested in Ubuntu 18.04, libguestfs-tools 1:1.36.13-1ubuntu3, QEMU 1:2.11+dfsg-1ubuntu7.3.

5

Seems like libguestfs can do it, see virt-make-fs(1) and maybe virt-copy-in(1).

1

Using Michaels strategy in Fedora 23:

# dnf install -y libguestfs-tools-c # virt-make-fs --format=raw --label mylabel the_directory myfs.raw 

This creates ext2 file system, that you can mount to some directory:

# mount file.fs /mnt 

Now /mnt contains all your files

1
  • If file.fs is a file, shouldn't the mount command be: mount -o loop file.fs /mnt? Commented Nov 9, 2015 at 22:43
1

I prefer (and use) the answer by Ciro Santilli, but here's something that works with a guestfish shell:

# assumes extant directory "rootfs" tar -cf rootfs.tar -C rootfs . guestfish <<EOF disk-create rootfs.qcow2 qcow2 "$((2 * $(stat -c%s rootfs.tar)))" add-drive rootfs.qcow2 run part-disk /dev/sda gpt mkfs ext4 /dev/sda mount /dev/sda / tar-in rootfs.tar / umount-all exit EOF 
0

What about accessing the directory as a network share from a VM that is also mounted to your target virtual disk image? You than than just perform a simple copy from the share to the mounted disk image.

2
  • That should work, and is similar to my current strategy, which is to build a VM and rsync the data in. It would be great if there were a quicker and cleaner solution, though. Copying the data into a new machine gets pretty messy because you have to worry about making sure you exclude certain files that shouldn't be copied, like fstab and most things in /boot Commented Mar 13, 2011 at 21:22
  • "exclude certain files that shouldn't be copied, like fstab and most things in /boot" - may not be that much of an issue when the VM target is an additional volume. Commented Mar 14, 2011 at 12:32

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.