How to remove three consecutive line feeds in a Makefile rule?

Question

I need to search for and replace three consecutive newline characters in an input file and filter them out of an output file for a makefile rule on Centos 4. I am using GNU sed v4.1.2 and GNU make v3.82. I've tried variations on the following with no success thus far:

THREE_LINE_FEEDS := $(shell echo "\012\012\012")

SED_RULE := 's/'$(THREE_LINE_FEEDS)'//'

output.txt: input.txt
    sed -r $(SED_RULE) input.txt > output.txt

Using the proposed perl, I get this issue in the shell (adapted from my make rule):

> cat input.txt | perl -e '$/ = undef; _ = <>; s/\n{3}//g; print;' > output.txt
Can't modify constant item in scalar assignment at -e line 1, near "<>;"
Execution of -e aborted due to compilation errors.

Thor · Answer 1 · 2012-07-28T22:01:49.127

3

sed only sees one line at a time, so searching for consecutive line-feeds will not work. One way is to push all the data into the hold space, and then do the replace when all input has been gathered, e.g.:

sed -r -n '
  $! {             # When it is not the last line
    1  { h }       # Replace hold space with first line
    1! { H }       # Otherwise append to hold space
  }
  $  {             # When last line reached
    H              # Append it to hold space
    g              # Replace pattern space with hold space
    s/\n{3}//g     # Remove all occurrences of \n{3}
    p              # Print pattern space
  }
' input.txt

Here is a simpler option using perl, but works in the same manner:

perl -e '
  $/ = undef;      # Slurp mode, read whole file
  $_ = <>;         # Read file to $_
  s/\n{3}//g;      # Remove all occurrences of \n{3}
  print;           # Output $_
' input.txt

Edit

A shorter perl version suggested by Peter O.:

perl -0777 -pe 's/\n{3}//g' input.txt

-0777 enables slurp mode.
-p implicitly prints result.

edited Jul 28 '12 at 22:01

answered Jul 28 '12 at 14:23

Thor

6,419
1
36
42

The perl is not working when inserted into my makefile rules, please see above updated question. – WilliamKF Jul 28 '12 at 15:49
When using `$` in makefiles they need to be protected, i.e. add an extra `$` on there, so `perl -e '$$/ = undef; $$_ = <>; s/\n{3}//g; print' input.txt > output.txt` should work. – Thor Jul 28 '12 at 15:52
Yes, had figured that out, but still not working. Getting error: Can't modify constant item in scalar assignment at -e line 1, near "<>;" Execution of -e aborted due to compilation errors. Can you pipe into perl? Why is it executing -e? – WilliamKF Jul 28 '12 at 16:14
The error you're getting comes from this bit: `_ = <>;`, should be `$_ = <>;`. Either of `perl -e ... infile` and `cat infile | perl -e ...` works. `-e` specifies the command `perl` should run, see `perlrun(1)`. – Thor Jul 28 '12 at 17:02
Bingo, it works now, needed other $ doubled in the makefile. – WilliamKF Jul 28 '12 at 18:42
2

`perl -0777pe 's/\n{3}//g'` is exactly the same. `0777` puts Perl into file-slurp mode, and `p` makes a read loop which prints – Peter.O Jul 28 '12 at 21:32
Nice, didn't know about the `-0777` option. – Thor Jul 28 '12 at 22:02

WilliamKF · Accepted Answer · 2012-07-28T23:41:34.093

For those interested, here is the actual makefile rule (I couldn't get the newlines to work in the original, so used a comment to add the documentation):

# 0777  -  Enables slurp mode (i.e. $/ = undef;  -  Slurp mode, read whole file
#                                   $_ = <>;     -  Read file to $_)
# s/\n{3}//g;                                    -  Remove occurrences of \n{3}
# p     -  Implicitly print result (i.e. print;  -  Output $_)
output.txt: input.txt
    $(SED) -r $(SED_RULE) input.txt | perl -0777 -pe 's/\n{3}//g' > output.txt

How to remove three consecutive line feeds in a Makefile rule?

2 Answers2

Edit