grep to find files that contain ^M (Windows carriage return)

Question

I use Linux. There is a pesky ^M (Windows cariage return) somewhere hidden in thousands of configuration files, and I have to find it, because it makes the server fail.

How do I find ^M among a directories hierarchy full of configuration files?

I think I can not enter ^M on the bash command line. But I have it in a text file that I called m.txt

Related: [Remove carriage return in Unix](http://stackoverflow.com/q/800030/456814). — 40XUserNotFound, May 15 '14 at 21:17
"I can not enter ^M on the bash command line". Yes you can. Try control-V Control-M — Hennes, Aug 23 '16 at 11:06

score 115 · Accepted Answer · edited Sep 22 '20 at 19:18

115

grep -r $'\r' *

Use -r for recursive search and $'' for c-style escape in Bash.

Moreover, if you are sure it's a text file, then it should be safe to run

tr -d $'\r' < filename

to remove all \r in a file.

If you are using GNU sed, -i will perform an in-place edit, so you won't need to write the file back:

sed $'s/\r//' -i filename

edited Sep 22 '20 at 19:18

Community

1

answered Oct 01 '10 at 05:51

livibetter

1,867
1
14
15

11

@Nicolas: You can enter a ^M at the command line by pressing ^V^M, but it's better to use `$'\r'`. – Dennis Williamson Oct 01 '10 at 05:54
Great, it works! Thanks for the ^V^M trick too :-) – Nicolas Raoul Oct 01 '10 at 05:56
5

Under Cygwin, -U is needed to make this work. And -n will tell you the line number: grep -r -U -n -e $'\r' – Rainer Blome Jan 03 '13 at 16:16
5

Add an -l to the grep command to just view the filenames. Else you might be bombarded with matching lines. – SineSwiper Mar 25 '14 at 20:28
Note that using `\r` in `sed 's/\r//' -i filename` won't work with OS X's version of `sed` (I've read that GNU-sed will recognize it though). As a workaround, `CTRL-V + CTRL-M` seems to work in the place of `\r` instead. – 40XUserNotFound May 15 '14 at 21:46
¿Can `$'\r'` be somehow inserted inside a greater expression? – 174140 Jul 28 '15 at 06:47
@Cupcake I edited the answer, and use C-style, it should work as long as the shell is Bash. – livibetter Sep 20 '15 at 01:22
@uprego what do you mean by "greater expression?" Could you give a line of example code of what are you trying to do? – livibetter Sep 20 '15 at 01:23
@livibetter I was confused by not knowning (currently don't know yet) the meaning of the dollah glyph in `sed $'s/\r//' -i filename`, use a dollar before a substitution expresion. Never seen it. And I was confused by the uses in `grep -r $'\r' *` and `tr -d $'\r' < filename`, I was matching them to the idiom `join file1 file12 -t $'\t'`, like used in [https://stackoverflow.com/questions/1722353](https://stackoverflow.com/questions/1722353). – 174140 Sep 21 '15 at 18:48
1

@uprego not sure if you understand them now, but fyi and other's, search `$'` read the first hit in manpage `bash(1)`, basically, you can see it as if you were writing C literal string. As for the `command < filename`, the use of `<` or `>` is called *redirection*, this is first time I have seen anyone called it *greater expression*. Search `REDIRECTION` in `bash(1)`. – livibetter Sep 23 '15 at 02:03
For those who happen to be using SunOS 5.10 and met with the error grep: illegal option -r , you should use `grep $'\r' *` – Nasri Najib Oct 25 '17 at 11:51
how to exclude searching for \r\n? – machinarium Jul 17 '19 at 14:48

score 14 · Answer 2 · edited Jul 25 '12 at 23:25

14

When I tried, I could tell it was sort-of working, but the lines were printing blank. Add in the option:

--color=never

If you get this issue, I think it's the escape characters for color highlighting interfering with the \r character.

edited Jul 25 '12 at 23:25

Gaff

18,569
15
57
68

answered Jul 25 '12 at 22:58

Judson Wilson

141
1
2

Thanks! I did not know why I was getting blank lines. – KolaB Sep 07 '20 at 16:07

mivk · Answer 3 · 2020-09-07T12:13:01.453

3

If I understand your question correctly, what you really want is to normalize all line-endings to the Unix LF (\x0a) standard. That is not the same as just blindly removing CRs (\x0d).

If you happen to have some Mac files around which use just CR for newlines, you will destroy those files. (Yes, Macs are supposed to use LF since almost 20 years, but there are still (in 2019) many Mac apps which use just CR).

You could use Perl's \R linebreak escape to replace any sort of newline with \n.

perl -i.bak -pe 's/\R/\n/g' $your_file

This would replace in-place any sort of linebreak with \n in $your_file, keeping a backup of the original file in ${your_file}.bak.

edited Sep 07 '20 at 12:13

answered Aug 16 '19 at 19:59

mivk

3,441
36
33

`dos2unix` is the proper command. – pbies Apr 28 '22 at 09:16
@pbies: Unfortunately, no. `dos2unix` doesn't seem to work in cases of mixed line endings (just re-tested on files with mixed CRLF and CR-only). That was the original reason why I started to use Perl instead. Besides, `dos2unix` needs to be installed, while Perl is already available on any decent Unix system. – mivk Apr 29 '22 at 15:12
Have you tried binary mode? – pbies Apr 29 '22 at 19:11

score 3 · Answer 4 · edited Jul 09 '15 at 05:55

If your server does not have a bash shell, an alternative is to use the -f option on grep, in combination with a prepared file containing \r.

To create the file:

$ echo -ne '\r' > /tmp/cr                    --or--                   $ printf '\r' > /tmp/cr

$ od -c /tmp/cr
0000000  \r
0000001

To actually do the search

$ grep -f /tmp/cr *.html *.php *.asp *.whatever

or you can be a little lazy and just type *,

$ grep -f /tmp/cr *

The -f filename option on grep is used to specify a file that contains patterns to match, one per line. In this case there's only one pattern.

score 1 · Answer 5 · edited Nov 20 '20 at 02:40

1

To use grep on end-of-line characters, I guess you have to tell grep the file is binary.

-l (letter L) is for printing only the filename
-P is for perl regexp (so \x0d is transformed to \r or ^M)

grep -l --binary -P '\x0d' *

edited Nov 20 '20 at 02:40

phuclv

26,555
15
113
235

answered Aug 23 '16 at 11:00

Vouze

216
1
7

score 0 · Answer 6 · answered Apr 28 '22 at 09:00

Other answers require Bash, this one should not:

grep -a -r "$(printf '\r')"

Explanation

printf '\r' prints a literal carriage return character
The wrapping "$(..)" puts the CR into an argument to the grep command.
-a tells grep to act on binary files, too, so that it actually prints matching lines even if the file is considered binary.

score 0 · Answer 7 · edited Jan 19 '12 at 09:23

0

In regular expression style, various newlines:

Windows (CR LF)
\r\n

Unix (LF)
\n

Since the \r\n sequence is fairly unique, I think you should be able to search for it that way?

To make things worse Macs used to have just '\r' in place of newline. I cannot verify this, but I don't think MacOSX generations does that any more.

Older Macs (CR)
\r

edited Jan 19 '12 at 09:23

Manwe

888
4
12

answered Oct 01 '10 at 05:48

Jeff Atwood

23,972
30
99
120

In the directory that contains m.txt, `grep "\r\n" *` gives no result. No result either for `egrep -e "\r\n" *` nor `grep -E "\r\n" *` – Nicolas Raoul Oct 01 '10 at 05:54
@nicolas ah, I misunderstood.. you meant CR only `\r` my bad. A full windows newline is indeed `\r\n` or CRLF – Jeff Atwood Oct 01 '10 at 05:58

score 0 · Answer 8 · answered Oct 10 '11 at 05:11

0

If you are on a Mac and use homebrew, you can do:

brew install tofrodos
fromdos file.txt

to remove all the Windows carriage returns from file.txt

To switch back to Windows carriage returns,

todos file.txt

answered Oct 10 '11 at 05:11

kortina

101
3

to search in a folder and clean all files coming from dos, run this command : find . -type f -name "*.java" | xargs fromdos – Taiko May 19 '14 at 03:38

score 0 · Answer 9 · edited Nov 20 '20 at 02:39

0

Following up on previous answers, the tr method is good:

533$ if [[ -n "`tr -cd "\r" <~/.bashrc`" ]]; then echo "DOS"; else echo "UNIX"; fi
UNIX

534$ if [[ -n "`tr -cd "\r" <dosfile.txt`" ]]; then echo "DOS"; else echo "UNIX"; fi
DOS

edited Nov 20 '20 at 02:39

phuclv

26,555
15
113
235

answered Nov 21 '14 at 06:23

Malcolm Boekhoff

97

grep to find files that contain ^M (Windows carriage return)

9 Answers9

Explanation