During my ongoing work developing xplorer² v2.0 one of the alpha testers mentioned a possible problem with huge ZIP folders (over 5GB in size). To test it I needed a huge file to play with, but how do you create one easily? One way is to zip your whole C:\ drive, or spread peanuts on the keyboard and release a small pack of monkeys to type away. Both methods would take ages to create the large file.

After some googling around, here's how you create a huge file in no time, which moreover takes almost no storage space on your hard disk (!) using the NTFS sparse file functionality:

Open a command console (CMD.EXE) with admin privileges and use FSUTIL as follows: fsutil file createnew bigfile.txt 68700000000 fsutil sparse setflag bigfile.txt fsutil file setzerodata offset=0 length=68700000000 bigfile.txt This will create a 67GB (68,700,000,000 byte) monster file which as you can see from its properties to the right, consumes only 4KB or otherwise a single sector size. However most programs think it is 67GB in size, e.g. try to open it with notepad...

How does it work? FSUTIL creates a file without actually committing disk space. By marking the file as sparse and then marking it as full of zeros (see steps 2-4 above) it doesn't use any data space at all. Sparse files offer a space saving feature where bytes that are 0 are not stored, only somewhere the filesystem makes a note that the range from [x,y] bytes is zero, and doesn't store the actual 0s. This is a naive type of compression which may remind you of the NTFS compression we saw a while back.

Working with sparse files is awkward and to be honest with you I don't know what real purpose they would serve — apparently they are designed for video and database use. For me and most of you, they are just an oddity suitable for a geeky kid party if the clown doesn't show up <g>

Now I'm going back to my work on xplorer² v2.0, more on that in the near future! (there is an alpha version to download if you are curious)

Post a comment on this topic