58

Why do 7-zip / WinRAR unzip files to "temp" before moving them to their destination?

I do not see any advantage against just directly unzipping to destination. It is really irritating, especially for large files (as it is now!).

Edit:

Please explain why they are not done in place.

Stevoisiak
  • 13,555
  • 39
  • 101
  • 154
  • 3
    If you specify `\` as the temporary folder in WinRAR, it should extract directly to the destination. –  Jan 30 '13 at 12:59

4 Answers4

97

How exactly are you extracting the files? Are you using the command-line or the GUI? Are you dragging the files or selecting them and using the extract function? Are you using the shell-extension context-menu?

If you enter a destination folder and then select the extract function or use the shell-extension, then they do not extract to a temporary folder first, they extract directly to the destination.

If you select the files in the UI and drag them to the target folder, then it will extract to a temporary folder.

The reason is in how the destination is selected. If you enter the target folder or use the context-menu item, then the program knows exactly where it needs to extract to. However, if you merely drag the files, then due to how the drag-and-drop function of OLE works, the program does not know where the target folder is. In other words, it is Explorer that receives the target folder, not the archiving program. As a result, the program cannot know where to extract them, and so simply extracts them to the temp folder, then Explorer moves them once it’s done. You can see this clearly by extracting a large file using both methods. When you drag it out to a folder, it extracts, then you see Explorer’s standard file operation dialog moving it to the folder. If you specify the folder and click Extract, it extracts and no further processing is done.

Feel free to peruse the source-code for 7-Zip to see how extraction location is handled.


I learned this the hard way several years ago when I wanted to implement drag-and-drop in a program I was writing.

Synetech
  • 68,243
  • 36
  • 223
  • 356
  • 9
    Absolutely brilliant -- thanks for this! This has boggled me for years, but I've never taken the time to track down the "what" and why". The odd times I find my C: temp folder filled to the whazoo, I trudge through Winrar's options to find where it must be set to first extract to a temp folder on C:. But alas, I don't find it. I never connected the correlation between the two methods, and now appreciate seeing the light :) Thanks! – Coldblackice Nov 18 '12 at 23:10
  • 4
    One thing wrong here is that Explorer does not **move** the file but rather **copies** it. Which is *really irritating* because moving is really fast (except when moving from partition to partition) and should have been the ideal choice (it doesn't waste space), but the idiots at Microsoft thought it's a great idea to **copy** from the temporary folder then just leave the temporary folder data lying around. Not only is the extraction of a huge archive going to take double the space, it's also going to take almost twice as long! – ADTC Apr 01 '14 at 03:45
  • They probably had a good reason for that, and it may have been due to some kind of limitation of systems and/or DDE on which it was based, when they first designed OLE, back in the days of Windows 3; a limitation which may or may not apply today. – Synetech Apr 02 '14 at 00:51
  • 1
    Yeah, it's rediculous, especially on a virtual machine with limited space. Using "extract to" unzips an 8GB RAR archive with 6000+ files in about 1 minute. However, if you drag the folders out, it takes the same amount of time to unzip to a temp directory, but in addition, it freezes the target window and then COPIES the files for over 15 minutes at only 3mb/s. It's especially slow copying a file on the same drive (possibly due to the underlying virtual file system), even though I can copy a file from another virtual machine onto this machine over the network at hundreds of MB/s. – Triynko Jun 02 '15 at 19:31
  • 1
    I guess a potential work around would be to create a 0 byte file with a GUID name i.e. practically improbable to occur as a duplicate file name. Then monitor explorer.exe's file handles to see where it copies the file. Finally extract the archive to the destination. – Zv_oDD Nov 09 '15 at 18:02
  • Man, OLE and Explorer API is the worst. So bloated. So illogical, so "pattern" filled. It drove me so mad, I fully gave up on trying to fix Explorer and am now implementing my own file manager from scratch, using only low level API. – ScienceDiscoverer Aug 29 '23 at 16:28
1

It is done so memory requirements for decompression are kept to a minimum.

If they didn't use the filesystem, decompression would happen in memory. Under low memory conditions, or for large compressed files this would sooner or later exhaust available memory and start the process of memory paging.

Paging under these circumstances would be a lot slower than just using the filesystem because the file is still being decompressed (and page files keep being added), but also because as the file is being decompressed, it is being checked for errors and there's as such a lot of read/write operations. The worst thing that can happen to a page file.

EDIT: Regarding the use of a temporary directory, this is so to follow many operating system guidelines. If the decompression fails, there's no guarantee the program performing the operation cleans up after itself. It may have crashed for instance. As such, no residual file remains in your target directory and the operating system will dispose of the temporary file when it sees appropriate.

A Dwarf
  • 18,981
  • 2
  • 44
  • 66
  • 3
    While true, that doesn't explain why the files are compressed to a temporary directory first and *then* moved to the destination one. The operation could also be taking place right there. – slhck Jun 25 '11 at 15:23
  • 3
    Edited to better reflect your question – A Dwarf Jun 25 '11 at 15:29
  • A wired thing is that even the moving is from `C:\%system dir%\temp` to `C:\some where`, it is still as slow as cross region moving.... –  Jun 25 '11 at 15:39
  • Why do not the developers provide a choice for the self clean-up (system crashes rarely happen anyway), which can save half of the time in some cases. –  Jun 25 '11 at 15:40
  • @Dante, Developer don't usually give the option because the operating system itself controls this process. In code, the developer introduces the target directory. But the operating system IO services are usually the ones that use the temp directory. – A Dwarf Jun 25 '11 at 15:42
  • 4
    @Dante, that is not true. Moving a file on the same partition simply changes the index of the file in the file system to the new directory, it does not copy any data and takes only an instant. Don't believe me? Try it, cut a multi-GB file from one folder and paste it to another folder; it takes less than a second. They try the same thing to another drive or partition. Takes minutes. – Dour High Arch Jun 25 '11 at 15:56
  • @Dour High Arch, my mistake. –  Jun 25 '11 at 16:02
  • 4
    Re "the operating system will dispose of the temporary file when it sees appropriate": Does Windows *ever* do that? – u1686_grawity Jun 25 '11 at 16:17
  • @grawity, yes. However the windows temporary folder functions as a both a temp and recovery folder. If, for instance, you close your session and there's still a program opened with file handlers to the temp folder, those temp files will not be deleted, giving room for the program to use whatever recovery method (if one is needed) it decides to implement. Note that handlers can also exist (and often do) through DLLs associated with a program that may not be unloaded when the user closed the program because of the way Windows optimizes DLL loading. – A Dwarf Jun 25 '11 at 19:25
  • Any files left on the temp folder because of this will no longer be disposed by windows and the user should manually delete them. But that's the reason why you see your temp folder increase even though windows does, under normal operation, manage this folder for you. – A Dwarf Jun 25 '11 at 19:26
  • @A Dwarf: But all user's programs are killed before destroying the session, therefore having open files is impossible. Also, at least on desktop versions of Windows, there's only one directory per user, and it's never cleaned automatically. Does XP Pro have a setting for per-session tempdirs? – u1686_grawity Jun 25 '11 at 20:45
  • @grawity, a kill signal to a process is different from a normal application exit function. This is by design. – A Dwarf Jun 25 '11 at 21:02
  • I don’t see how decompressing to a temp folder has **any** impact on the amount of memory used. And there is no such guideline about the temp directory. What does the OS care if you decompress some files to say, an MP3 or game folder somewhere, but then run out? – Synetech Aug 27 '11 at 00:45
  • 1
    This completely false answer should be removed. 7zip only decompresses to a temp folder during drag-and-drop operations and this is due to limitations of Windows. See https://superuser.com/questions/197961/make-7-zip-extract-an-archive-directly-to-a-directory for more info. – dss539 Jun 05 '17 at 21:17
-1

The reason is simpler than you think: Many programs unpack files to %temp%, is that the destination filesystem might not have enough space.

Now, you might know that your filesystem might have enough space, however the application does not. What if that filesystem is in use by the OS or another application and gets filled up while decompressing?

Developers make the assumption that %temp% has "unlimited" space, while your destination does not.

Keltari
  • 71,875
  • 26
  • 179
  • 229
  • Not quite; that’s not a reason to use the temp directory. The application *does* (or at least can) know whether there is enough space or not. First, the archiver knows exactly how big the compressed files are when they are decompressed and can allocated them beforehand, and then fill them in as it decompresses, and second, it is quite simple to query the free space on a drive. Besides, if there is not enough space, then it just decompresses while it can, then throws an error when it runs out of space. – Synetech Aug 27 '11 at 00:36
  • Actually, this is quite the answer. Just because an application can preallocate the space on the target destination, does not mean it will have exclusive access to it. The application can query the destination and see if it has enough space, but while that query is happening, before its able to allocate space, another application can take some of that space. – Keltari Aug 27 '11 at 01:05
  • On top of that, a users temp space is tied to that user at the filesystem level. If there was a crash or another failure, another user could not see the temp files. The destination might not have such security restrictions, so its safer not to have possibly sensitive data hanging out on another filesystem. – Keltari Aug 27 '11 at 01:08
  • Like I said, if it runs out while decompressing, it simply gives an error message. How would extracting to a temp drive fix it? If there isn’t enough space, then there isn’t enough space after the files are extracted to the temp drive, and it wastes all those CPU cycles and drive reads/writes extracting to the temp drive for nothing. Like I said in my answer, feel free to peruse the source-code for 7-Zip, or [contact Alexander Roshal](http://rarsoft.com/feedback.htm) to ask him about WinRAR. Besides, like I also said, if you specify the target or use the context-menu, it *does not* use %temp%. – Synetech Aug 27 '11 at 01:12
-2

Not being a 7-zip or WinRAR developer, my comments here are purely speculation. That being said, using temp space to unpack to until everything is complete help to verify that all of the files are intact (i.e. that the zip is not corrupted).

There is nothing worse than unpacking a large compressed file, having started working on files at the beginning of the archive; only to find out that something is corrupted at the end of the archive. At that point you lose confidence in everything.

My last comment is that I don't recall seeing this behavior from 7-zip. When I right click and say extract here, I generally have access to the files as they are being unpacked. Have you double checked that it isn't a setting somewhere?

Kirk
  • 2,382
  • 17
  • 19
  • 3rd paragraph: I cannot find such settings, and it is a large bunch of files (4G) I was extracting just now, I am pretty sure all 4G was moved slowly at last. –  Jun 25 '11 at 15:22
  • 2nd paragraph: extract to "temp" does not prevent anything if the zipped file is already corrupted. –  Jun 25 '11 at 15:24
  • It prevents you from seeing files out of a corrupted zip. The logic is that you don't get to see anything until everything is verified. As far as the setting goes, I'm not sure what to tell you. If I tell 7-zip to extract a large tar file to my Desktop, I start seeing the files immediately. It could be because tar is not a compressed format. – Kirk Jun 25 '11 at 15:28
  • *> The logic is that you don't get to see anything until everything is verified.* That’s absurd. How then would you recover some files from a corrupt archive? The reason you don’t experience it is because you used the shell-extension instead of dragging the files from the GUI. – Synetech Aug 27 '11 at 00:47