66

I was going through an article that explained the hibernation procedure in Microsoft Windows. The main points that I get out of it are

  1. Windows dumps the whole RAM (after processing it maybe) in the hiberfil.sys file.
  2. During boot up, the hibernation file is read, and the contents are loaded in the RAM.

My question is when I am usually copying a file of size, say, 1 GB, it takes about 2 minutes to complete.

However, when Windows is writing the hibernation file (during the hibernation procedure), the whole process takes maybe 10-15 seconds. Why is there such a difference in writing speed?

My RAM size is 4 GB. (I am not talking about fast boot technology.)

Benchmarks:

  1. Copying 1 GB file from Disk 1 to Disk 2 (external): 2.3 minutes.
  2. Hibernating the system: 15 seconds.
Excellll
  • 12,627
  • 11
  • 51
  • 78
coder
  • 731
  • 2
  • 6
  • 10
  • 3
    I don't know the answer, but I bet if you checked the book [Windows Internals](https://technet.microsoft.com/en-us/sysinternals/bb963901.aspx) "Chapter 13: Startup and Shutdown" it would tell you (If I had the book myself I would check). – Scott Chamberlain Mar 09 '15 at 17:26
  • 1
    Where in that article does it say that all of RAM is stored to disk? – Eric Lippert Mar 09 '15 at 19:16
  • 2
    This is a good question. When hibernation was first implemented in 1998, it wasn't nearly so fast. – Gabe Mar 09 '15 at 21:45
  • 23
    @coder : the NT system make sure hyberfil.sys have it's full space allocated and that the whole file isn't fragmented. In that condition there is no head jumps on the hard drive during the operation. So you will got effective speeds like 150Mo/s. You can recheck what I said with`fsutil`. – user2284570 Mar 09 '15 at 22:10
  • 1
    While I realize it doesn't explain everything, you are neglecting the amount of time needed to read the file from the first disk. Depending on how your drives are set up, this may not be concurrent. You're also neglecting the overhead that Windows adds when copying files in order to give you a valid estimate of how long the copy will take. My experience is that Windows 7 is slower at file copying in general than Windows XP was, unless I use a third party file copier. – trlkly Mar 10 '15 at 01:49
  • 3
    External disk is typically slower than internal disk, too. – Harry Johnston Mar 10 '15 at 03:46
  • @trlkly You can see the difference when copying a file CLI style. –  Mar 10 '15 at 08:37
  • The space for hibernation on the disk is contiguous, so the data can be written in large blocks, with very few "seeks". Most of the time spent in normal disk operations is doing the "seeks". – Daniel R Hicks Mar 10 '15 at 11:22
  • 2
    @EricLippert - it most certainly doesn't store all of RAM, but that still doesn't explain it. I regularly have few gigabytes of active RAM that needs to be stored (VS2013 or Eclipse + few more things take a lot of ram), and they get stored at the speed that seems to me greater than even theoretical write speed of my non-SSD drive. – Davor Mar 10 '15 at 12:27
  • @HarryJohnston : Only MS-DOS based windows can be installed on external hard disks with some tricky manual changes. Windows to go is only for removable flash devices. – user2284570 Mar 11 '15 at 21:22
  • @user2284570: the OP was copying data from the internal disk to an external disk, and using that as a baseline for disk speed. There's nothing to suggest that he is *booting* from the external disk. – Harry Johnston Mar 11 '15 at 21:43
  • @HarryJohnston : yes, and you know modern USB hard drives achieves speeds around 40Mo/s which is lot of slower. – user2284570 Mar 11 '15 at 22:19
  • @user2284570: yes, that was my point. – Harry Johnston Mar 11 '15 at 22:33
  • @user2284570: Nothing about the NT file system "makes sure hiberfil.sys isn't fragmented". I've had fragmented hiberfil.sys's before, it's very easy to come across them. – user541686 Mar 12 '15 at 05:31
  • just like @trlkly and Harry Johnston said, disks are slow and external disks are even slower. You need to benchmark by copying from RAM to disk like how hibernation do, for example from /dev/urandom or /dev/null – phuclv Jan 06 '16 at 03:37

5 Answers5

46

This is probably an three-fold answer.

One thing that may be at play here is the new Hybrid Shutdown in Windows which effectively closes your applications, logs you off and then proceeds to hibernate the core of the operating system. Already having this data saved out would mean it does not need to "re-hibernate" it potentially.

The second thing would be that the hibernation would not need to save out memory pages that are either paged out to the swap file or are not in use (this would be one reason to aggressively fill the swap file and keep data in memory as well).

The third would be that the hibernation file data is also compressed. Combine that with my second point and if you have only a small set of data to export that contains highly compressible data (executables generally get compressed well) then the amount of data to go out to the hibernation file can be substantially smaller than the working set of data. Note that, as stated in the comments, file caches and other unnecessary buffer data could easily be discarded with no ill effect to reduce the amount of data to be dumped in the hibernation file.

Additionally, current hard drives are quite fast. With a disk that has a sustained write in the order of 100 MB/s you would be able to write out (uncompressed) 4 GB of RAM in under one minute. As hibernation can be done as the last thing after suspending all user processes and before suspending the CPU the OS will generally have the full write speed of the disk. This is one thing your simple benchmark will not have, and copying from disk to disk will potentially be slower than simply writing RAM out to disk.

Combine these things and the amount of data to be written to the hibernation file could be quite small, potentially of the order of 1 GB and would probably be written to one large continuous block in under 10 seconds.

Community
  • 1
Mokubai
  • 89,133
  • 25
  • 207
  • 233
  • 37
    Or, to make it more clear: Your RAM probably isn’t full. Buffers are flushed and cache is discarded on hibernation. Only the memory *actually in use by applications* has to be written to disk. Hybrid shutdown reduces the amount of memory in use by logging out the user. – Daniel B Mar 09 '15 at 17:51
  • 2
    Pages that are not dirty is a more general statement of "paged out to the swap file", this would include executables. (Since executables are somewhat fragmented on disk, this could slow wake-up.) In addition, clean file buffers can presumably just be dropped even if they are not part of a memory mapped file. –  Mar 09 '15 at 20:38
  • @DanielB :`Or, to make it more clear: Your RAM probably isn’t full.`Your are probably confusing with linux swapspace which use RAM compression. Windows isn't not able to this and dump the complete RAM each times. – user2284570 Mar 09 '15 at 22:08
  • @Mokubai : compressing the hyberfil.sys is the way to get hibernation no longer working. – user2284570 Mar 09 '15 at 22:10
  • 1
    @user2284570 I'm not saying that you compress the file, I'm saying that the data that Windows puts in the file gets compressed as part of the process of writing the file. Try reading the answer (and Microsoft document contained therein) that I linked to. – Mokubai Mar 09 '15 at 22:13
  • So what do you mean with`The third would be that the hibernation file data is also compressed.`inside your answer? The general rule I saw on all systems is that hyberfil.sys is equal to 100% of the RAM size. It also apply to 64 and 32 bits. *(this is also my case currently on 8.1)*. Or larger than 100% if RAM module was added and removed later. – user2284570 Mar 09 '15 at 22:15
  • 1
    Read the linked answer, the data in the file is compressed, as stated in a document from Microsoft. – Mokubai Mar 09 '15 at 22:16
  • 3
    @user2284570 from the document I linked in that answer *"Windows supports hibernation by copying the contents of memory to disk. The system compresses memory contents before preserving them on the disk, which reduces the required disk space to less than the total amount of physical memory on the system. "* – Mokubai Mar 09 '15 at 22:20
  • 1
    _executables generally get compressed well_ - any reference for this statement or just personal opinion? – rkosegi Mar 10 '15 at 05:18
  • 2
    According to something I believe I read here, Windows also does not actually write files in memory that are also stored on disk, either to the hibernation file or the page file. Under memory pressure, it just removes the files from memory, and then brings them back from disk when needed. Essentially, each file serves as its own paging file. – trlkly Mar 10 '15 at 05:52
  • 1
    A fourth point is the direction of the data, which is one way. When copying a file, the HDD must be read, then copied to RAM, move to new location, write from RAM, go back to old location and read more... -- That is a substantially longer process then just saying, write RAM to HDD. – David Mar 11 '15 at 07:35
  • 5
    @user2284570: That's because the worst case scenario is 1:1 compression. Windows has to make sure there's enough (reserved) space in hyberfil.sys for any possible memory configuration - even if it only needs a tenth of the RAM-size for a particular hibernation. Add to that that a decent portion of RAM use is files loaded into memory (executables, resources...), but still mapped from the HDD, and you can indeed save a lot of writing. Have a program generate 4 GiB of crypto-random data in RAM, and the hibernation takes significantly longer - and even then, some of it might have been in swap. – Luaan Mar 11 '15 at 08:24
  • @Luaan : No, as I said, the worst case is when you add RAM module, boot windows, and withdraw it. Windows enlarge the file to 100% of the ram size, but the size of the file stay the same after you removed the RAM module. – user2284570 Mar 11 '15 at 16:10
  • 2
    @user2284570 That has no relation whatsoever to what I've said :) – Luaan Mar 11 '15 at 16:56
  • 3
    @user2284570: The file is that large to ensure there's space on the disk to store all the memory. Not all of that space is actually used in hibernation. Sometimes the file will be (say) 7% compressed memory contents, 93% junk. – psmears Mar 11 '15 at 20:55
  • 1
    @iser2285470 Not only is it compressed, but decompression is further accelerated by using additional CPU cores (can't tell if compression takes advantage of multiple cores). There's more information about it, the Session 0 state, and even a quote of as little as 10-15% of RAM being preserved in a typical case, on this [Building Windows 8: Delivering fast boot times in Windows 8](http://blogs.msdn.com/b/b8/archive/2011/09/08/delivering-fast-boot-times-in-windows-8.aspx) post from 2011. – Louis Waweru Mar 12 '15 at 02:22
  • @user2284570 read the [MS document about hibernation in Windows 7](http://download.microsoft.com/download/7/E/7/7E7662CF-CBEA-470B-A97E-CE7CE0D98DC2/HiberFootprint.docx) first. The file is actually compressed and you can reduce the file size to at least 50% the amount of RAM you have. The default since windows 7 is 75% RAM, not 100% – phuclv Jan 06 '16 at 03:44
31

First, the amount of RAM that needs to be saved is surprisingly small. In fact, only the set of mapped dirty pages ("lazy writeback") needs to be flushed, as well as all private pages that have been written to and relocated executable code need to be written.

  • The .text segments of executables is always backed by file mapping. That is also true for at least some DLLs (but not all, depends on whether they need to be relocated).
  • Memory that is similarly backed by file mappings can be discarded (presumed it's not CoW or RW and dirty).
  • Lazy writeback will still have to occur, but other than that, caches can be discarded.
  • Memory that has been allocated but not been written to (usually the greater part of application data!) is backed by the zero page and can be discarded.
  • The larger part of memory pages that are on "standby" status (the actual per-process resident working set on Windows is suprisingly small, a mere 16MB) will have been copied to the page file in the background at some point and can be discarded.
  • Regions of memory that are mapped by certain devices such as the graphics card may (possibly) not need to be saved. Users are sometimes surprised that they plug 8GiB or 16GiB into a computer, and 1GiB or 2GiB are just "gone" for no apparent reason. The major graphics APIs require applications being able with buffer contents becoming invalid "under some conditions" (without telling exactly what this means). It is thus not unreasonable to expect that the memory that is pinned by the graphics driver is just discarded, too. The screen is going to go dark anyway, after all.

Second, contrary to you copying a file, dumping the set of RAM pages that need to be saved disk is a single sequential, contiguous write from the point of view of the drive. The Win32 API even exposes a user-level function for this very operation. Gather write is directly supported by the hardware and works as fast as the disk is physically able to accept data (the controller will directly pull data via DMA).
There are a number of preconditions for this to work (such as alignment, block size, pinning), and it does not play well with caching and there is no such thing as "lazy writeback" (which is a very desirable optimization under normal operation).
That is the reason why not every write works like that all the time. However, when the system is saving the hibernation file, all preconditions are automatically met (all data is page-aligned, page-sized, and pinned) and caching has just become irrelevant because the computer is going to be turned off in a moment.

Third, doing a single contiguous write is very favorable both for spinning disks and for solid state disks.

The swap file and the hibernation file are usually some of the earliest files created and reserved on the disk. They usually have one, at most two fragments. Thus, unless sectors are damaged and the disk has to reallocate physical sectors, a logical sequential write translates to a physical sequential write on a spinning disk.

No read-modify-write operations are necessary on the disk when a huge amount of sequential, contiguous data is being written. This problem is less pronounced on a spinning harddisks which can write single sectors that are quite small (Provided that you don't write single bytes, which caching usually prevents, the device needs not fetch the original contents and write back the modified version.).
This is, however, something that is very noticeable on SSD where every write means that e.g. a 512kB block (that is an usual number, but it could be larger) has to be read and modified by the controller, and written back to a different block. While you can in principle write to (but not overwrite) smaller units on flash disks, you can only ever erase huge blocks, it's how the hardware works. This is the reason why SSDs fare so much better on huge sequential writes.

Damon
  • 4,512
  • 3
  • 22
  • 28
  • Even if a DLL is relocated, the only thing needed to bring it back is the relocated address. Relocation is a deterministic process and can be repeated. – MSalters Mar 10 '15 at 11:37
  • "Gather write"? Do you mean "Rather, write"? – Peter Mortensen Mar 10 '15 at 20:59
  • 3
    @PeterMortensen: No, I really mean _gather_ write (as opposed to scatter read). This means writing to a single file while _gathering_ the data from multiple locations. You supply an array of structures each of which contains a start address and a length (with strict alignment requirements). The operating system passes these to the controller, and the hardware does the rest. – Damon Mar 10 '15 at 21:50
  • 1
    @MSalters: But relocation creates a private copy of the page, and it's then extremely difficult to determine whether any other modifications have been made to the private copy. Contrast with mappings that didn't require fixup, and use copy-on-write. If other modifications are made, there will be a private copy. If not, the page will still be configured for CoW. – Ben Voigt Mar 12 '15 at 15:41
  • 1
    @MSalters It may be a deterministic process, but that doesn't imply that the hibernation code operates at the same layer of the software stack as the linker. If hibernation is at the kernel layer and linking is user layer, then hibernation cannot make any assumptions about what the linker does. – kasperd Mar 14 '15 at 20:55
10

It doesn't dump the whole RAM at hibernate time.

It will already have a large portion of the RAM already duplicated on the disk. This not only allows hibernation to happen quickly but also allows memory to be made available quickly for new programs (so that they can launch quickly).

Therefore it only has to write a small fraction of the 4GB and that can be done in 10-15s.

From microsoft:

When RAM is in short supply (for example, Committed Bytes is greater than installed RAM), the operating system will try to keep a certain fraction of installed RAM available for immediate use by copying virtual memory pages that are not in active use to the pagefile. Therefore, this counter will not reach zero and is not necessarily a good indication of whether your system is short of RAM.

Peter Crotty
  • 195
  • 1
  • 3
2

In addition to all of the above, I think there are a few other factors at play.

One is that, when copying a file, the file has to be read and written; hybernation only requires the file to be written. It is, by definition, already in memory!

Closely related to this, when reading a file and writing it at the same time, to save memory, the process is: read a chunk, write a chunk, update the directory (to show the new size); read a chunk, write a chunk, update the directory.

Each time you move from one part of the disk to another (e.g. read file a to write file b, write file b to write the directory, and write the directory to read the next chunk) the disk has to seek - move the heads, allow the heads to settle, wait for the right part of the disk to come by. This is one of the advantages of a solid state disk - seeking takes no time at all. When hibernating, the data is written end-to-end. The hibernation (swap) file is pre-allocated, so the directory does not need to be updated (you are not changing the size of the hibernation file, just the contents).

And finally, your computer has suspended all other tasks - this is the ONLY thing it is doing (I doubt this will make much difference, but it's bound to make some!). Even things like memory management and task switching are suspended.

AMADANON Inc.
  • 266
  • 1
  • 4
  • It's bound to make a _huge_ difference! – Lightness Races in Orbit Mar 12 '15 at 12:39
  • @LightnessRacesinOrbit: CPU contention will make hardly any difference at all. Lack of I/O contention is a big deal, but this answer has already stated that seeking kills performance, and seeking, not lack of overall bandwidth, is the main issue with I/O contention. – Ben Voigt Mar 12 '15 at 15:44
  • @BenVoigt: Yes, I concur. And when you have 40 processes all trying to do stuff on disk, that's going to substantially increase disk seeking. (tl;dr I wasn't talking about CPU contention) – Lightness Races in Orbit Mar 12 '15 at 18:28
  • @LightnessRacesinOrbit: That seems... unusual even during normal operation (everything except entering and leaving hibernation). I know that when I catch a background task hitting the disk, I uninstall the sucker and replace it with something that only accesses the disk when I ask it for something. – Ben Voigt Mar 12 '15 at 18:35
  • @BenVoigt: That seems unlikely. Daemon logging is the most obvious counterexample, followed by such things as ntpd's drift file updates. I'm not claiming that either of those examples has a big effect here, but I don't think it's reasonable to expect no background tasks to touch the disk autonomously. – Lightness Races in Orbit Mar 12 '15 at 18:55
  • @LightnessRacesinOrbit: Small writes don't hit the disk, they go into cache and get written out whenever they don't cause contention... and the OS can allocate a block to reduce seeking (due to abstractions like logical block addressing, this usually means sequential allocation, causing fragmentation, but removing contention between multiple writers). Non-Windows systems have it even better -- with e.g. syslog, you don't have logging from multiple background processes. – Ben Voigt Mar 12 '15 at 19:43
  • @Ben Still not true; consider Apache, which writes to access_log and error_log and neither of those things are syslog. – Lightness Races in Orbit Mar 12 '15 at 20:03
  • @LightnessRacesinOrbit: I said "only accesses the disk when I ask it for something". Entries are made to `access_log` and `error_log` only when I send a (e.g. HTTP) request. Besides which, these small writes don't cause contention for the reasons already given. – Ben Voigt Mar 12 '15 at 20:11
  • @BenVoigt: Potentially anyone on the internet can make an HTTP request causing log entries! Besides which, as I said before, I'm not saying that these specific examples will cause high contention, only pointing out that one of your rationales doesn't seem to hold! – Lightness Races in Orbit Mar 12 '15 at 21:26
0

This is probably because RAM has much quicker input/output speeds than the hard disk, so the the RAM can output the stuff in it as quick as the hard disk can read in.

When copying files, you are also limited by various factors - the speed of the disk, if it has to read in and out onto the same disk it will take longer, the limited speed of the connection (if to external drive), checking it isn't overwriting anything etc

Wilf
  • 241
  • 2
  • 16
  • 9
    but still OS needs to write the 4GB RAM data on the Disk which is governed by the I/O bottleneck – coder Mar 09 '15 at 17:23
  • Also assuming favorable parameters it implies that during hibernating my Disk's write speed goes from 40MB/s to ~260 MB/s. Can it be the correct? – coder Mar 09 '15 at 17:31
  • 1
    Probably - there shouldn't be too much of a I/O bottleneck as it only needs to write the data (there is probably something in place so it know it won't overwrite stuff and where to put the data so it doesn't need to read the disk too much). On my (linux dual booted) laptop I can use [`dd if=/dev/zero of=/tmp/output.img bs=8k count=256k`](http://www.cyberciti.biz/tips/how-fast-is-linux-sata-hard-disk.html) and get `1862606848 bytes (1.9 GB) copied, 1.81605 s, 1.0 GB/s`, so it seems to be possible (I will add that windows copying files seems to take unnecessarily long anyway). – Wilf Mar 09 '15 at 17:39
  • You can also get a much quicker transfer when copying files over the local internet. Also it might not need to copy all the stuff in the RAM - some of the data in the RAM may just be cached and not needed to restore the system when waking from hibernation. – Wilf Mar 09 '15 at 17:43
  • I just tried the dd benchmark on my system. It never went more than 52 MB/s :/ (old machine) However I believe _"there is probably something in place so it know it won't overwrite stuff and where to put the data so it doesn't need to read the disk too much"_ Is the key for fast speed. – coder Mar 09 '15 at 17:45