Shell unbuffering for CygWin: is it possible?

Question

I have tested on CygWin doing "ls -R" with a large directory and sending the output to a Test.txt file, so the process takes about 30 seconds to complete; the file is created, but it is empty; it will only be filled when the "ls -R" command finishes, and I need to see the file content meanwhile new data is entering.

The problem is supposed to come from the buffering that the operating system makes before writing into the file.

This is what I have tested in order to write into the file without buffering:

• The unbuffer command from the expect package: where is it? Modern expect version has no such unbuffer command (see below):

    ./unbuffer -p ls /cygdrive/y/Repositorio/ -R > Test.txt
    ./unbuffer ls /cygdrive/y/Repositorio/ -R | tee Test.txt
    ./unbuffer -p ls /cygdrive/y/Repositorio/ -R | tee Test.txt
    ./unbuffer ls /cygdrive/y/Repositorio/ -R | ./unbuffer -p egrep "" | tee Test.txt
    ./unbuffer -p ls /cygdrive/y/Repositorio/ -R | ./unbuffer -p egrep "" | tee Test.txt

• Installing the Expect package and naming "unbuffer" this script:

    #!/usr/bin/expect --
    # Description: unbuffer stdout of a program
    # Author: Don Libes, NIST
    eval spawn -noecho $argv
    set timeout -1
    expect

• Installing the Expect package and naming "unbuffer" this other script:

    #!/usr/bin/expect --
    # Description: unbuffer stdout of a program
    # Author: Don Libes, NIST
    if {[string compare [lindex $argv 0] "-p"] == 0} {
        # pipeline
        set stty_init "-echo"
        eval spawn -noecho [lrange $argv 1 end]
        close_on_eof -i $user_spawn_id 0
        interact {
        eof {
            # flush remaining output from child
            expect -timeout 1 -re .+
            return
        }
        }
    } else {
        set stty_init "-opost"
        set timeout -1
        eval spawn -noecho $argv
        expect
    }

• The script command:

    script -c "ls /cygdrive/y/Repositorio/ -R" | tee Test.txt
    script -c "ls /cygdrive/y/Repositorio/ -R" /dev/null | tee Test.txt
    script -q -c "ls /cygdrive/y/Repositorio/ -R" /dev/null | tee Test.txt
    script -q -c "ls /cygdrive/y/Repositorio/ -R" /dev/null | grep "" --line-buffered | tee Test.txt
    script -q -c "ls /cygdrive/y/Repositorio/ -R" /dev/null | egrep "" --line-buffered | tee Test.txt

• The egrep command with unbuffering option:

    cat BigFile.txt | egrep "" --line-buffered | tee Test.txt

• The stdbuf: it does not exist on modern CygWin, or so I think.

• Another methods:

    ls /cygdrive/y/Repositorio/ -R 1>&2 | tee Test.txt
    ls /cygdrive/y/Repositorio/ -R 1>&2 |& tee Test.txt

• Tests done with bash and mintty.

• Checked and tested everything in these threads:

  Writing "tail -f" output to another file
  https://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe
  https://serverfault.com/questions/294218/is-there-a-way-to-redirect-output-to-a-file-without-buffering-on-unix-linux

Results are always the same: the output file Test.txt is only filled at the end of the "ls -R" command.

Any more ideas, please?

Answer 1

Use stdbuf. It is on cygwin in the coreutils package.