[xplorer˛] — Sparse is Spartan
home » blog » 5 June 2011

During my ongoing work developing xplorer˛ v2.0 one of the alpha testers mentioned a possible problem with huge ZIP folders (over 5GB in size). To test it I needed a huge file to play with, but how do you create one easily? One way is to zip your whole C:\ drive, or spread peanuts on the keyboard and release a small pack of monkeys to type away. Both methods would take ages to create the large file.

After some googling around, here's how you create a huge file in no time, which moreover takes almost no storage space on your hard disk (!) using the NTFS sparse file functionality:

  1. Open a command console (CMD.EXE) with admin privileges and use FSUTIL as follows:
  2. fsutil file createnew bigfile.txt 68700000000
  3. fsutil sparse setflag bigfile.txt
  4. fsutil file setzerodata offset=0 length=68700000000 bigfile.txt
This will create a 67GB (68,700,000,000 byte) monster file which as you can see from its properties to the right, consumes only 4KB or otherwise a single sector size. However most programs think it is 67GB in size, e.g. try to open it with notepad...
console for fsutil commands huge sparse file properties

How does it work? FSUTIL creates a file without actually committing disk space. By marking the file as sparse and then marking it as full of zeros (see steps 2-4 above) it doesn't use any data space at all. Sparse files offer a space saving feature where bytes that are 0 are not stored, only somewhere the filesystem makes a note that the range from [x,y] bytes is zero, and doesn't store the actual 0s. This is a naive type of compression which may remind you of the NTFS compression we saw a while back.

Working with sparse files is awkward and to be honest with you I don't know what real purpose they would serve — apparently they are designed for video and database use. For me and most of you, they are just an oddity suitable for a geeky kid party if the clown doesn't show up <g>

Now I'm going back to my work on xplorer˛ v2.0, more on that in the near future! (there is an alpha version to download if you are curious)

Post a comment on this topic



What would you like to do next?

Reclaim control of your files!
  • browse
  • preview
  • manage
  • locate
  • organize
Download xplorer2 free trial
"This powerhouse file manager beats the pants off Microsoft's built-in utility..."

© 2002—2011 Nikos Bozinis, all rights reserved