zcat | head : write error from large gz file

Question

I was trying zcat to get only the first 1M lines of a compressed .gzip file into a new one.

However, I get the following error:

$ zcat path/to/R2.fq.gz | head -100 >R2.fqtest
head: write error: Input/output error

I was relieved when I found this thread and tried:

$ gunzip -c path/to/R2.fq.gz | head -n100 >R2_test.fq

But I get the same error again!

head: write error: Input/output error

Anyone know what might be going on and how to fix it?

thanks! Carmen

Edit >

As requested, I typed the following after getting the error

tail /var/log/syslog

and got

tail: cannot open `/var/log/syslog' for reading: No such file or directory

The command itself is correct. Right after the getting the error, execute `tail /var/log/syslog` and edit the result into your question. — Dennis, Apr 26 '13 at 19:34
Done! :) I got `tail: cannot open '/var/log/syslog' for reading: No such file or directory` — Carmen Sandoval, Apr 26 '13 at 20:56
Well, that didn't turn out to be particularly helpful... Which Linux distro are you using? — Dennis, Apr 26 '13 at 20:58

score 0 · Answer 1 · answered Apr 26 '13 at 18:52

0

it doesn't look like a problem with zcat/gunzip. Can you create the file and write into it?

Try this:

hexdump /dev/urandom | head -n 100 > R2_test.fq

answered Apr 26 '13 at 18:52

pqnet

508
3
10

`hexdump /dev/urandom | head -n 100 >R2_test.fq` I get this: `head: write error: Input/output error` – Carmen Sandoval Apr 26 '13 at 20:50
then you probably can't write in the file R2_test.fq . Try it from a directory where you have write access. – pqnet Apr 26 '13 at 23:28
pqnet, I do have access to the folder where I am trying to write the file. – Carmen Sandoval Apr 26 '13 at 23:36

score 0 · Answer 2 · edited Apr 29 '13 at 06:30

0

In order, please try and post results of:

head -n 10 /proc/cpuinfo

echo test | gzip -3 | zcat

echo test > R2.fqtest

The goal here is to first test for proper operation of head, then proper operation of zcat, then the ability to successfully write to R2.fqtest.

Since you state that everything works as expected until the attempt to write to R2.fqtest, which does not throw an error but also doesn't show the data you wrote, there's something hinky with that file or underlying filesystem. Does another process write, lock, or rotate that file? What does the fstab line for the filesystem in question look like? Can you unmount and fsck the filesystem in question? Do you see errors in the logs from the physical drive underlying that filesystem?

edited Apr 29 '13 at 06:30

Community

1

answered Apr 27 '13 at 03:38

Jim Salter

228
1
5

Thanks, Jim: The output of `head -n 10 /proc/cpuinfo` is: `processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 45 model name : Intel(R) Xeon(R) CPU E5-2665 0 @ 2.40GHz stepping : 7 cpu MHz : 1200.000 cache size : 20480 KB physical id : 0 siblings : 16` ; `echo test | gzip -3 | zcat` returns `test` as expected; and `echo test > R2.fqtest` does not return an error, but if I open R2.fqtest there is nothing there... – Carmen Sandoval Apr 28 '13 at 02:11
(Note - the new edit is me as well; didn't realize I wasn't logged in until I'd posted!) – Jim Salter Apr 29 '13 at 06:31

score 0 · Answer 3 · answered May 01 '13 at 23:18

0

So, it turns out this is a problem of not having enough disk space!

Thanks, eveyone!

Carmen

answered May 01 '13 at 23:18

Carmen Sandoval

145
1
7

zcat | head : write error from large gz file

3 Answers3