15

Using sed to make text that contains certain characters suitable for use in string literals is straightforward:

sed "s/\\\\/\\\\\\\\/g"
sed "s/\\\"/\\\\\\\"/g"

But how can I do something similar with a text file containing newline characters?

Melab
  • 983
  • 6
  • 18
  • 34
  • sed won't show you the new line character in the find portion, sed is old too, like a legacy program. You could use perl, it has equivalents to sed's find and replace. – barlop Jan 18 '18 at 04:08

7 Answers7

19

If you're dealing with a pipelined command, then I like @vossad01's answer. However, in a multi-line script, I prefer normal Bash parameter expansion, which is both more efficient (doesn't need to fork a new process for sed or create any pipes) and perhaps a bit easier to read:

${varName//$'\n'/\\n}

Reader's guide:

  • ${...} - Interpret the internal stuff using parameter expansion
  • varName - Name of the variable containing the content
  • // - Replace all instances of...
  • $'\n' - The literal newline character (we're turning the escape into the literal here)
  • / - Replace with...
  • \\n - The newline escape sequence ("\n"): note that we had to escape the backslash itself

Example:

$ foo="$(printf '%s\n' $(seq 1 3))"
$ echo "$foo"
1
2
3
$ echo "${foo//$'\n'/\\n}"
1\n2\n3
Community
  • 1
pieman72
  • 291
  • 2
  • 5
10

Use:

sed ':a;N;$!ba;s/\n/\\n/g'

Which uses the answer from How can I replace a newline (\n) using sed? substituting an escaped newline character as the replacement to match the question here.

Karu
  • 4,634
  • 7
  • 34
  • 54
vossad01
  • 201
  • 2
  • 5
  • This command appears to ignore the last linefeed, and insert an extra unescaped linefeed for those files that do not have a linefeed at the end of the line. – Michael C. Chen Dec 06 '21 at 22:20
  • This works in reverse also if you want to change escaped newlines to actual newlines. Just `sed ':a;N;$!ba;s/\\n/\n/g'` – flickerfly Jul 30 '22 at 00:53
4

I solved this one with:

sed 's/$/\\n/g' | tr -d'\n'

sed puts '\\n' at the end of every line, then tr removes the newlines.

Erick Brown
  • 151
  • 1
  • Nice! Of all the solutions here, this one is easiest to understand and the most straightforward IMO. – tytk May 19 '22 at 19:03
2

This awk solution worked for me:

$ awk 1 ORS='\\n' file.txt

Credit goes to this excellent answer.

Risadinha
  • 121
  • 2
2

sed is line based, and this can cause issues when trying to replace newline characters.

the official documentation for sed makes a specific reference to newline characters and states they are stripped off before being passed to sed.

I would suggest that 'tr' would probably be a better fit here.

as an example, to replace newline characters with spaces:

tr '\n' ' ' < inputfile
David Moylan
  • 147
  • 4
  • Won't `[\r\n]` be cleaner? – theoden8 Aug 12 '15 at 14:03
  • I guess that all depends on the source data and requirements. OP only referred to newline. I would recommend to try both and see which produces the best result. YMMV. – David Moylan Aug 12 '15 at 14:10
  • 1
    I tried using `tr "\n" "\\n"` before I posted this question and it just replaced the newline characters with backslashes. It seems to only turn a sequence into one character only. – Melab Aug 12 '15 at 20:12
  • @Melab correct. the 'tr' command will only do a single character replacement. – David Moylan Aug 15 '15 at 00:04
  • 7
    So then how does it help at all? – Melab Aug 17 '15 at 00:41
  • @Melab a '\n' character is a single character. So no problem there. I think perhaps what you are trying to get at is that tr will only replace with one character. And then you have a point, how will this help at all! – barlop Jan 18 '18 at 03:56
1

I had this problem too.

You can use the \a function to start every newline with the proper escape sequence, then pipe the resulting function into echo -n which suppresses newlines.

echo -n $(sed '\a
\\n');

Stick whatever you're sed-ing in the parentheses before the sed command.

RockPaperLz- Mask it or Casket
  • 7,741
  • 30
  • 73
  • 122
Kaeden Wile
  • 11
  • 3
1

I'd like to extend David Moytan's solution:

cat /etc/passwd | perl -e 'while(<>) { $_ =~ s/[\r\n]/__NEWLINE__/g; print "$_" }'
Community
  • 1
theoden8
  • 640
  • 7
  • 19