2

Every now and then, I receive a download link for a 6 GB .zip data archive that I'd like to:

  • download

  • unzip

  • recompress with 7z, with a password

I'd like to avoid using temporary files for each step, because it would use ~ 18 GB of write operations on my SSD, whereas only 6 GB are necessary at the end.

On Linux, it would probably be wget ... | unzip ... | 7z ... with pipes.

How to do this on Windows?

Note: this question is linked to Make 7-Zip extract an archive directly to a directory but not a duplicate, because the linked question only covers the "7z temp" part of the question, whereas I'm speaking about a download+unzip+7z chain.
Also I'm looking for a command-line solution, and not a drag'n'drop GUI solution like Make 7-Zip extract an archive directly to a directory.

Basj
  • 1,489
  • 7
  • 47
  • 90
  • 1
    Why do you consider 18GB of write to be an issue? Space constraints or… wear-rate paranoia? – Tetsujin Dec 30 '20 at 18:36
  • 1
    @Tetsujin OPs mention of "write operations" indicates that is the case. – music2myear Dec 30 '20 at 19:03
  • @music2myear - I kind of guessed that, but I just wanted confirmation ;) – Tetsujin Dec 30 '20 at 19:31
  • It's not a duplicate, because the linked question only covers the "7z temp" part of the question, whereas I'm speaking about a download+unzip+7z chain :) – Basj Dec 30 '20 at 19:43
  • 1
    This operation is a bad fit for pipes or the like. You cannot really pipe multiple files around, because it’s not _just data_. They also have names. // This operation in general is very feasible. However, you need a purpose-made program to do it. – Daniel B Dec 30 '20 at 20:30
  • Using temp files would double only the Compress or Decompress stages. Assuming minimal compression and no temp files, downloading writes 6GB, decompressing writes a further 6GB, and compress writes a final 6GB, still totaling 18GB. Assuming a worst-case where both the decompress and compress stages use temp files, that total would balloon to 30GB. The only option to reduce writes to your SSD below 18GB would be to use a different disk to store the file, or to use a RAM disk of some sort. – music2myear Dec 30 '20 at 23:11
  • Further, the answer I flagged as duplicate notes the reason 7zip uses temp files is when the OS is handling the target portion of the process. Using the standard 7zip command line tools, the application would be handling the target portion of the process and so not temp file would be used. The duplicate is still the duplicate. – music2myear Dec 30 '20 at 23:13
  • You're right @DanielB, it would be complex with multiple files. Fortunately I only need 1 file from the original file (a Google Takeout archive with a `.mbox` file), see https://superuser.com/questions/1613860/unzip-and-7z-with-password-without-temporary-files/1613998#1613998 – Basj Dec 31 '20 at 08:50
  • @music2myear Yes it could easily go up to 30GB. I'm currently using a RAMdisk indeed to avoid this. – Basj Dec 31 '20 at 08:53

1 Answers1

2

The following command-line solution avoids temporary files for decompression/compression, and works on Windows. It uses -so and -si stdout/stdin flags of 7z.

7z e -so -r Takeout20201231_0941.zip *.mbox | 7z a -si -pSECRET -mhe=on -mx=2 Takeout20201231_0941.7z

Note that there is only 1 match for *.mbox in my original archive, so there is only one file to be decompressed / compressed.

With multiple files, it would more complex, as noted by @DanielB in his comment:

This operation is a bad fit for pipes or the like. You cannot really pipe multiple files around, because it’s not just data. They also have names.

Lastly, piping the download of the original ZIP directly to 7z e wouldn't be possible because ZIP files can't be stream-decompressed: they often have an index at the end of the file, thus requiring the whole file first before starting to decompress.

For this, we can quickly create a RAM disk X:\ with OSFMount:

osfmount.com -a -t vm -m X: -s 8G -o format:ntfs
Basj
  • 1,489
  • 7
  • 47
  • 90