Quickly create a large file on a Linux system

Quickly create a large file on a Linux system

dd from the other answers is a good solution, but it is slow for this purpose. In Linux (and other POSIX systems), we have fallocate, which uses the desired space without having to actually writing to it, works with most modern disk based file systems, very fast:

For example:

fallocate -l 10G gentoo_root.img

This is a common question — especially in todays environment of virtual environments. Unfortunately, the answer is not as straight-forward as one might assume.

dd is the obvious first choice, but dd is essentially a copy and that forces you to write every block of data (thus, initializing the file contents)… And that initialization is what takes up so much I/O time. (Want to make it take even longer? Use /dev/random instead of /dev/zero! Then youll use CPU as well as I/O time!) In the end though, dd is a poor choice (though essentially the default used by the VM create GUIs). E.g:

dd if=/dev/zero of=./gentoo_root.img bs=4k iflag=fullblock,count_bytes count=10G

truncate is another choice — and is likely the fastest… But that is because it creates a sparse file. Essentially, a sparse file is a section of disk that has a lot of the same data, and the underlying filesystem cheats by not really storing all of the data, but just pretending that its all there. Thus, when you use truncate to create a 20 GB drive for your VM, the filesystem doesnt actually allocate 20 GB, but it cheats and says that there are 20 GB of zeros there, even though as little as one track on the disk may actually (really) be in use. E.g.:

 truncate -s 10G gentoo_root.img

fallocate is the final — and bestchoice for use with VM disk allocation, because it essentially reserves (or allocates all of the space youre seeking, but it doesnt bother to write anything. So, when you use fallocate to create a 20 GB virtual drive space, you really do get a 20 GB file (not a sparse file, and you wont have bothered to write anything to it — which means virtually anything could be in there — kind of like a brand new disk!) E.g.:

fallocate -l 10G gentoo_root.img

Quickly create a large file on a Linux system

Linux & all filesystems

xfs_mkfile 10240m 10Gigfile

Linux & and some filesystems (ext4, xfs, btrfs and ocfs2)

fallocate -l 10G 10Gigfile

OS X, Solaris, SunOS and probably other UNIXes

mkfile 10240m 10Gigfile

HP-UX

prealloc 10Gigfile 10737418240

Explanation

Try mkfile <size> myfile as an alternative of dd. With the -n option the size is noted, but disk blocks arent allocated until data is written to them. Without the -n option, the space is zero-filled, which means writing to the disk, which means taking time.

mkfile is derived from SunOS and is not available everywhere. Most Linux systems have xfs_mkfile which works exactly the same way, and not just on XFS file systems despite the name. Its included in xfsprogs (for Debian/Ubuntu) or similar named packages.

Most Linux systems also have fallocate, which only works on certain file systems (such as btrfs, ext4, ocfs2, and xfs), but is the fastest, as it allocates all the file space (creates non-holey files) but does not initialize any of it.

Leave a Reply

Your email address will not be published.