2

There is a concept in bash called process substitution. You can run a command and use the output as a file.

e.g.

$ cat -n <(seq 3|tac)
     1  3
     2  2
     3  1

I am curious why the following fails;

$ du -sk <(xzcat /var/log/mpd/scribble.log.xz )
0   /dev/fd/63

Similarly, we have this

$ file <(seq 1 2)
/dev/fd/63: broken symbolic link to pipe:[32560687]

The file is not empty.

$ xzcat /var/log/mpd/scribble.log.xz  | wc -c 
16877047

>/dev/null  pv  <(xzcat /var/log/mpd/scribble.log.xz ) 
16.1MiB 0:00:00 [ 232MiB/s] [  <=>                   ]

As a bonus question, do you know a technique in bash to masquarade fifos as regular files for a short duration?

Ярослав Рахматуллин
  • 11,096
  • 5
  • 42
  • 74

1 Answers1

2

Regular files consist of three main constituents:

  1. Name (hardlink)
  2. Inode (system info)
  3. Body (data)

Pipes (no matter named or anonymous, persistent or temporary) have no body. Therefore they always have zero size unlike regular files. They only have in-memory buffer. That's why anonymous named pipe /dev/fd/63 has zero length. You could use any other command within <(...) operator with the same result, say

du -sk <(cat somefile)

You can create named pipe with mkfifo command to see that its size is always equal to zero even if you fully fill its buffer. Actually, filesize of named (and any other) pipes has no real sense and is set to zero for uniformity of listing among other files.

Oleg Bolden
  • 1,687
  • 14
  • 14
  • So the *length of the pipe* is equal to the contents of the buffer which may be something or nothing, and the only way to know is if you access the buffer? – Ярослав Рахматуллин Sep 13 '17 at 16:06
  • I'd call it "capacity" rather then "size". FIFO's capacity is usually restricted in 64K though maximum buffer size in modern Linux systems is 1Mb (`cat /proc/sys/fs/pipe-max-size`). There are always two counterparts - reader from fifo and writer into it. Both writer and reader gets and receives information in portions until reader gets EOF mark in data flow. At this moment transfer of data is considered to be competed and pipe is deleted (unless it is named pipe). – Oleg Bolden Sep 13 '17 at 17:13
  • @OlegBolden If a FIFO resides in memory, how can it fill up, say, the var partition? I constantly see situations where /var is full and the culprit seems (at least to me) to be many FIFO files which were never closed properly. I see these situations where the output of `df -h` does not match the output of `du -h` – Timothy Pulliam Mar 24 '19 at 17:59