Command to prepend string to each line?

Question

Looking for something like this? Any ideas?

cmd | prepend "[ERRORS] "

[ERROR] line1 text
[ERROR] line2 text
[ERROR] line3 text
... etc

Answer 1

Try this:

cmd | awk '{print "[ERROR] " $0}'

Cheers

Answer 2

cmd | while read line; do echo "[ERROR] $line"; done

has the advantage of only using bash builtins so fewer processes will be created/destroyed so it should be a touch faster than awk or sed.

@tzrik points out that it might also make a nice bash function. Defining it like:

function prepend() { while read line; do echo "${1}${line}"; done; }

would allow it to be used like:

cmd | prepend "[ERROR] "

Answer 3

With all due credit to @grawity, I'm submitting his comment as an answer, as it seems the best answer here to me.

sed 's/^/[ERROR] /' cmd

Answer 4

I created a GitHub repository to do some speed tests.

The result is:

• In the general case, awk is fastest. sed is a bit slower and perl is not much slower than sed. Apparently, all those are highly optimized languages for text processing.
• In very special situations, where forks dominate, running your script as a compiled ksh script (shcomp) can save even more processing time. In contrast, bash is dead slow compared to compiled ksh scripts.
• Creating a statically linked binary to beat awk seems not be worth the effort.

In contrast python is dead slow, but I have not tested a compiled case, because it is usually not what you would do in such a scripting case.

Following variants are tested:

while read line; do echo "[TEST] $line"; done
while read -r line; do echo "[TEST] $line"; done
while read -r line; do echo "[TEST]" $line; done
while read -r line; do echo "[TEST]" "$line"; done
sed 's/^/[TEST] /'
awk '{ print "[TEST] " $0 }'
awk -vT="[TEST] " '{ print T $0 }'
awk -vT="[TEST]" '{ print T " " $0 }'
awk -vT="[TEST]" 'BEGIN { T=T " "; } { print T $0 }'
T="[TEST] " awk '{ print ENVIRON["T"] $0 }'
T="[TEST]" awk '{ print ENVIRON["T"] " " $0 }'
T="[TEST]" awk 'BEGIN { T=ENVIRON["T"] " " } { print T $0 }'
perl -ne 'print "[TEST] $_"'

Two binary variants of one of my tools (it is not optimzed for speed, though):

./unbuffered.dynamic -cp'[TEST] ' -q ''
./unbuffered.static -cp'[TEST] ' -q ''

Python buffered:

python -uSc 'import sys
for line in sys.stdin: print "[TEST]",line,'

And Python unbuffered:

python -uSc 'import sys
while 1:
 line = sys.stdin.readline()
 if not line: break
 print "[TEST]",line,'

Answer 5

cmd | sed 's/.*/[ERROR] &/'

Answer 6

I wanted a solution that handled stdout and stderr, so I wrote prepend.sh and put it in my path:

#!/bin/bash

prepend_lines(){
  local prepended=$1
  while read line; do
    echo "$prepended" "$line"
  done
}

tag=$1

shift

"$@" > >(prepend_lines "$tag") 2> >(prepend_lines "$tag" 1>&2)

Now I can just run prepend.sh "[ERROR]" cmd ..., to prepend "[ERROR]" to the output of cmd, and still have stderr and stdout separate.

Answer 7

cmd | xargs -L 1 -i echo "prefix{}"

or even easier in case prefix is space-delimited from the line itself

cmd | xargs -L 1 echo prefix

This is not very efficient performance-wise, but short to write.

It works by running echo once per each line of input. xargs allows you to also process \0-delimited lines.

Answer 8

A solution without sed/awk and while loops:

cmd | xargs -n1 printf "$prefix%s\n"

Answer 9

Moreutils has the ts command, which can be abused to do what you need:

cmd | ts "[ERRORS]"

It's probably slower than the other solutions, but much shorter...