121

I need to substitute some text inside a text file with a replacement. Usually I would do something like

sed -i 's/text/replacement/g' path/to/the/file

The problem is that both text and replacement are complex strings containing dashes, slashes, blackslashes, quotes and so on. If I escape all necessary characters inside text the thing becomes quickly unreadable. On the other hand I do not need the power of regular expressions: I just need to substitute the text literally.

Is there a way to do text substitution without using regular expressions with some bash command?

It would be rather trivial to write a script that does this, but I figure there should exist something already.

Hennes
  • 64,768
  • 7
  • 111
  • 168
Andrea
  • 1,415
  • 3
  • 10
  • 8
  • Necessary to do it through bash? A simplistic solution would be to open in Word and do a `find and replace all` – Akash May 09 '12 at 15:04
  • 27
    @akash Because systems that have `bash` always ship with Microsoft Word? ;) No.. Just kidding. The OP might want to do this on a remote machine or for a batch of files though. – slhck May 09 '12 at 15:07
  • @slhck :) Well, I guess gedit should have a similar option – Akash May 09 '12 at 15:09
  • An option would be to *somehow* correctly escape everything before passing it to `sed`, which is probably a futile effort considering all the switches and platform differences. – l0b0 May 09 '12 at 15:11
  • related: https://stackoverflow.com/questions/29613304/is-it-possible-to-escape-regex-metacharacters-reliably-with-sed – Ciro Santilli OurBigBook.com Mar 28 '18 at 13:07
  • [Related](https://superuser.com/a/1368785/500826). tl;dr: use `tr`, `sed` with `y/`. Take a look to [`sponge`](https://linux.die.net/man/1/sponge). – Pablo A Feb 11 '20 at 15:44
  • 3
    Also have this problem and I am puzzled that a software that was written in 1974, nobody ever bothered to implement a simple string replacement. Isn't this a thing you would have even done before implementing regex? I'm speechless. – user136036 Apr 11 '21 at 18:15

18 Answers18

21

When you don't need the power of regular expressions, don't use it. That is fine.
But, this is not really a regular expression.

sed 's|literal_pattern|replacement_string|g'

So, if / is your problem, use | and you don't need to escape the former.

PS: About the comments, also see this Stackoverflow answer on Escape a string for sed search pattern.


Update: If you are fine using Perl try it with \Q and \E like this,

 perl -pe 's|\Qliteral_pattern\E|replacement_string|g'

@RedGrittyBrick has also suggested a similar trick with stronger Perl syntax in a comment here or here

peterh
  • 2,553
  • 10
  • 31
  • 49
nik
  • 55,788
  • 10
  • 98
  • 140
  • Thank you, I did not know about the difference between / and | – Andrea May 09 '12 at 15:18
  • 105
    I'm not sure this answer is useful... The only difference between `s|||` and `s///` is that the seperator character is different and so that **one** character doesn't need escaping. You could equally do `s###`. The real issue here is that the OP doesn't want to have to worry about escaping the contents of `literal_pattern` (which is not literal at all and will be interpreted as a regex). – Benj May 09 '12 at 15:31
  • 31
    This will not avoid the interpretation of other special characters. What if search for `1234.*aaa` with your solution it match much more than the intended `1234\.\*aaa`. – Matteo May 09 '12 at 15:47
  • 47
    This answer should *not* be accepted – Steven Lu Aug 04 '16 at 18:01
  • 9
    This misses the point entirely. The text to be matched could contain any wierdness. In my case it's a random password. You know how those go – Christian Bongiorno Jun 11 '18 at 18:01
  • 1
    Use `str =~ s/\Q$replace_this\E/$with_this/;` in case of variables – Jithin Pavithran Sep 30 '18 at 14:26
  • 2
    I agree with Steven Lu, because it is simply wrong. The first character after 's' in "s/aaa/bbb/" is just an arbitrary delimiter and can be anything, as long as it doesn't occur in aaa or bbb. These three give exactly the same result, and 'aaa' will *always* be interpreted as a regular expression by sed: 's/aaa/bbb/', 's|aaa|bbb|', and even 'scaaacbbbc'. https://www.gnu.org/software/sed/manual/sed.html#The-_0022s_0022-Command – Luc VdV Feb 04 '20 at 11:59
  • This problem was so annoying that I made a CLI tool to do it: https://github.com/paralin/git-find-replace – Christian Stewart May 26 '23 at 00:57
21
export FIND='find this'
export REPLACE='replace with this'
ruby -p -i -e "gsub(ENV['FIND'], ENV['REPLACE'])" path/to/file

This is the only 100% safe solution here, because:

  • It's a static substition, not a regexp, no need to escape anything (thus, superior to using sed)
  • It won't break if your string contains } char (thus, superior to a submitted Perl solution)
  • It won't break with any character, because ENV['FIND'] is used, not $FIND. With $FIND or your text inlined in Ruby code, you could hit a syntax error if your string contained an unescaped '.
Nowaker
  • 1,575
  • 13
  • 13
  • 1
    I had to use `export FIND='find this; export REPLACE='replace with this';` in my bash script so that `ENV['FIND']` and `ENV['replace']` had the expected values. I was replacing some really long encrypted strings in a file. This was just the ticket. – dmmfll Jun 24 '16 at 12:45
  • This is a good answer answer because it's reliable and ruby is ubiquitous. Based on this answer I now use [this shell script](https://gist.github.com/pesterhazy/72f98f600313e1a9ce1036dffb7ca8b1). – loevborg Jul 09 '18 at 08:49
  • Unfortunately doesn't work when FIND contains multiple lines. – adrelanos Jan 26 '19 at 17:09
  • There's nothing that would prevent it from working with multiple lines in FIND. Use double quoted \n. – Nowaker Jan 27 '19 at 19:20
  • 1
    you dont need `export`, you can pass variables to subshells in sh by declaring them before the command on the same line; `FIND='find this' REPLACE='replace with this' ruby ....` with no newlines. Especially handy if you were never working with variables in the first place: `FIND=\`echo command\` REPLACE=\`cat replace.txt\` ruby ...`. Use `\\` if you still want newlines in the actual script anyway ;) – Hashbrown Jan 23 '20 at 00:05
13

The replace command will do this.

https://linux.die.net/man/1/replace

Change in place:

replace text replacement -- path/to/the/file

To stdout:

replace text replacement < path/to/the/file

Example:

$ replace '.*' '[^a-z ]{1,3}' <<EOF
> r1: /.*/g
> r2: /.*/gi
> EOF
r1: /[^a-z ]{1,3}/g
r2: /[^a-z ]{1,3}/gi

The replace command comes with MySQL or MariaDB.

Derek Veit
  • 238
  • 2
  • 4
7

You can do it converting the patterns to their escaped form automatically. Like this:

keyword_raw=$'1\n2\n3'
keyword_regexp="$(printf '%s' "$keyword_raw" | sed -e 's/[]\/$*.^|[]/\\&/g' | sed ':a;N;$!ba;s,\n,\\n,g')"
# keyword_regexp is now '1\/2\/3'

replacement_raw=$'2\n3\n4'
replacement_regexp="$(printf '%s' "$replacement_raw" | sed -e 's/[\/&]/\\&/g' | sed ':a;N;$!ba;s,\n,\\n,g')"
# replacement_regexp is now '2\/3\/4'

echo $'a/b/c/1\n2\n3/d/e/f' | sed -e "s/$keyword_regexp/$replacement_regexp/"
# the last command will print 'a/b/c/2\n3\n4/d/e/f'

Credits for this solutions goes here: https://stackoverflow.com/questions/407523/escape-a-string-for-a-sed-replace-pattern

Note1: this only works for non-empty keywords. Empty keywords are not accepted by sed (sed -e 's//replacement/').

Note2: unfortunately, I don't know a popular tool that would NOT use regexp-s to solve the problem. You can write such a tool in Rust or C, but it's not there by default.

VasyaNovikov
  • 3,366
  • 3
  • 25
  • 33
  • 6
    This completely misses the OP's point. Obviously you can escape the pattern, but for some patterns this is tedious. – icecreamsword Oct 02 '18 at 19:35
  • 1
    @icecreamsword did you read my answer below the first line? The script does escaping **automatically**. – VasyaNovikov Oct 03 '18 at 04:24
  • Does not work for me, it prints `a/b/c/123/d/e/f`, unless I remove the `$` prefix from strings in variable assignments. Would you mind explaining the script `':a;N;$!ba'`? My understanding was that the second sed expression is unnecessary. Did you really intend to use actual line breaks in the strings? – MayeulC Jun 01 '22 at 12:56
  • Works for me except on bash this emits a bunch of warning to stderr when used on macOS (but still works) such as `sed: 1: ":a;N;$!ba;s,\n,\\n,g": unused label 'a;N;$!ba;s,\n,\\n,g'`. – Allon Guralnek Aug 12 '22 at 05:38
4

This is an enhancement of Hashbrown’s answer (and wef’s answer to a very similar question).

We can remove the issue of the special meaning of various special characters and strings (^, ., [, *, $, \(, \), \{, \}, \+, \?, &, \1, …, whatever, and the / delimiter) by removing the special characters. Specifically, we can convert everything to hex; then we have only 0-9 and a-f to deal with.  This example demonstrates the principle:

$ echo -n '3.14' | xxd
0000000: 332e 3134                                3.14

$ echo -n 'pi'   | xxd
0000000: 7069                                     pi

$ echo '3.14 is a transcendental number.  3614 is an integer.'     | xxd
0000000: 332e 3134 2069 7320 6120 7472 616e 7363  3.14 is a transc
0000010: 656e 6465 6e74 616c 206e 756d 6265 722e  endental number.
0000020: 2020 3336 3134 2069 7320 616e 2069 6e74    3614 is an int
0000030: 6567 6572 2e0a                           eger..

$ echo "3.14 is a transcendental number.  3614 is an integer." | xxd -p |
                                                       sed 's/332e3134/7069/g' | xxd -p -r
pi is a transcendental number.  3614 is an integer.

whereas, of course, sed 's/3.14/pi/g' would also change 3614.

The above is a slight oversimplification; it doesn’t account for boundaries.  Consider this (somewhat contrived) example:

$ echo -n 'E' | xxd
0000000: 45                                       E

$ echo -n 'g' | xxd
0000000: 67                                       g

$ echo '$Q Eak!' | xxd
0000000: 2451 2045 616b 210a                      $Q Eak!.

$ echo '$Q Eak!' | xxd -p | sed 's/45/67/g' | xxd -p -r
&q gak!

Because $ (24) and Q (51) combine to form 2451, the s/45/67/g command rips it apart from the inside.  It changes 2451 to 2671, which is &q (26 + 71).  We can prevent that by separating the bytes of data in the search text, the replacement text and the file with spaces.  Here’s a stylized solution:

encode() {
        xxd -p    -- "$@" | sed 's/../& /g' | tr -d '\n'
}
decode() {
        xxd -p -r -- "$@"
}
left=$( printf '%s' "$search"      | encode)
right=$(printf '%s' "$replacement" | encode)
encode path/to/the/file | sed "s/$left/$right/g" | decode

I defined an encode function because I used that functionality three times, and then I defined decode for symmetry.  If you don’t want to define a decode function, just change the last line to

encode path/to/the/file | sed "s/$left/$right/g" | xxd -p –r

Note that the encode function triples the size of the data (text) in the file, and then sends it through sed as a single line — without even having a newline at the end.  GNU sed seems to be able to handle this; other versions might not be able to.  Also, this does not modify the file in place; you’ll need to write the output to a temporary file and then copy it over the original file (or one of the other tricks for doing that).

As an added bonus, this solution handles multi-line search and replace (in other words, search and replacement strings that contain newline(s)).

4

You could also use perl's \Q mechanism to "quote (disable) pattern metacharacters"

perl -pe 'BEGIN {$text = q{your */text/?goes"here"}} s/\Q$text\E/replacement/g'
glenn jackman
  • 25,463
  • 6
  • 46
  • 69
4

check out my Perl script. it do exactly what you need without implicit or explicit use of regular expression :

https://github.com/Samer-Al-iraqi/Linux-str_replace

str_replace Search Replace File # replace in File in place

STDIN | str_replace Search Replace # to STDOUT

very handy right? I had to learn Perl to do it. because I really really need it.

Samer Ata
  • 141
  • 1
2

You can use php's str_replace:

php -R 'echo str_replace("\|!£$%&/()=?^\"'\''","replace",$argn),PHP_EOL;'<input.txt >output.txt

Note: You would still need to escape single quotes ' and double quotes ", though.

simlev
  • 3,782
  • 3
  • 14
  • 33
2

You can do this in sh without any script (though putting this "one-liner" into a script would be better) or non-standard external program (I reeeally liked @Nowaker's answer thanks to it's safety against injection, but this old CentOS box I needed this on didn't have ruby!). as long as perl isn't non-standard for you

Without attempting to escape the string (and account for issues with doing it correctly syntactically, knowing all the special characters, et cetera), we can just blanket encode all the strings so that nothing has the possibility of being special.

cat path/to/the/file | xxd -p | tr -d '\n' \
| perl -pe "s/$(printf '%s' 'text' | xxd -p | tr -d '\n')(?=(?:.{2})*\$)/$(printf '%s' 'replacement' | xxd -p | tr -d '\n')/g" \
| xxd -p -r

This was just to match the asker's example, other users can obviously replace 'text' with "$text" if using a variable, or cat path/to/the/file with printf '%s' "$input" if not using a file.

You can even replace the /g with / to make it replace-once, or otherwise edit the regex outside the $() to "escape" only portions of the matcher (say, add a ^ after s/ to make it match only the start of the file).
If in the above you need ^/$ to match ends-of-lines again you'll need unencode those:

cat path/to/the/file | xxd -p | tr -d '\n' | sed 's/0a/\n/g'\
| perl -pe "s/^$(printf '%s' 'text' | xxd -p | tr -d '\n')(?=(?:.{2})*\$)/$(printf '%s' 'replacement' | xxd -p | tr -d '\n')/g" \
| sed 's/\n/0a/g' | xxd -p -r

Which'll replace all lines in the file begining with 'text' to instead start with 'replacement'


Test:

Within ^/.[a]|$0\\{7}!!^/.[a]|$0\\{7}!!^/.[a]|$0\\{7}, replace literal ^/.[a]|$0\\{7} with literally $0\\

printf '%s' '^/.[a]|$0\\{7}!!^/.[a]|$0\\{7}!!^/.[a]|$0\\{7}' \
| xxd -p | tr -d '\n' \
| perl -pe "s/$(printf '%s' '^/.[a]|$0\\{7}' | xxd -p | tr -d '\n')(?=(?:.{2})*\$)/$(printf '%s' '$0\\' | xxd -p | tr -d '\n')/g" \
| xxd -p -r

Output:

$0\\!!$0\\!!$0\\
Hashbrown
  • 2,782
  • 4
  • 32
  • 47
  • (1) This looks *a **LOT*** like [wef’s answer](https://stackoverflow.com/q/54059656/3960947#54059658 "How to search & replace arbitrary literal strings in sed and awk (and perl)") to a very similar question.  I’m not accusing you of copying it — it’s certainly possible for people to discover / devise the same solution independently — just advising you that somebody else posted this before you.  … (Cont’d) – G-Man Says 'Reinstate Monica' Apr 01 '20 at 04:28
  • 1
    (Cont’d) …  (2) When I first saw this answer, I thought it was brilliant.  A few minutes later, I realized that, like a diamond, it was flawed.  For example, if you try to change `E` to `g` in a file that contains `$Q`, it will change to `&q`.  This is because `E` is `45`, `g` is `67`, and `$Q` is `2451`, so, when you do ```s/45/67/```, you change `2451` to `2671`, which is `&q` (`26` + `71`).  … I have posted an answer that addresses this issue. – G-Man Says 'Reinstate Monica' Apr 01 '20 at 04:28
  • thanks for that @G-ManSays'ReinstateMonica' not too hard to fix, it'll work now – Hashbrown Apr 01 '20 at 06:33
2

Working on an alpine docker container, i wasn't keen to install python / pearl / ruby / python just to do the very simple operation of a find and replace. All these solutions are horribly complex!!

There are two viable solutions to this:

  1. Use a different find + replace from elsewhere (e.g. python/pearl/etc)
  2. Escape all the regex metacharacters. We can use sed for this purpose.

I cannot do the first in my case as I am working in a minimal docker container.
This solution can be used for the second

In my case i had a known string in my file: {{replace_me}} and a user input. Lets call it $replace_text.

sed -i "s/{{replace_me}}/$(sed 's/[&/\]/\\&/g' <<<"$replace_text")/g" path/to/file

How does it work?

We use sed -i for an inplace conversion. Here i use the \ as a delimiter as I am specifically escaping this in my replacement line. This protects against the user putting down my\string.

The $(sed 's/[&/\]/\\&/g' <<<"$replace_text") bit is explained in detail here in a great answer where this solution is derived from. In this case I am using it as a one liner

In answer to OPs initial question, here is a sed one liner that should do the trick:

sed -i "s/$(sed 's/[^^]/[&]/g; s/\^/\\^/g' <<<"$search_text")/$(sed 's/[&/\]/\\&/g' <<<"$replace_text")/g" path/to/file

but i guess he probably no longer cares given that it has been 7 years!

  • This is complicated. (1) When I started to read this, I thought “This is only trying to solve the second half of the problem — the replacement string.  That’s not ideal, but a partial answer is better than none.”  But then I saw that your answer was trying to solve the whole problem, and it’s wrong.  If `$initial_text` is `3.14`, then the `printf` outputs `3.14`.  But **`.`** is a special character in regular expressions, so `s/3.14/foo/` will find and replace `3614`, `3q14`, `3%14`, etc. … (Cont’d) – G-Man Says 'Reinstate Monica' Mar 27 '20 at 04:33
  • (Cont’d) …  (2) You say that one can use “anything” as the delimiter for the **```s```** command.  That’s misleading; you can use *any **single character**.*  There are, indeed, times when doing text manipulation that we assume that we can choose a *string* that’s so “obscure” that it will not appear naturally in the data.  But you can’t do that with a single character; the initial\_text and the replace\_text can, between them, contain every valid ASCII character.  So what do you use as the delimiter then? … (Cont’d) – G-Man Says 'Reinstate Monica' Mar 27 '20 at 04:33
  • (Cont’d) …  (3) Gotcha!  It seems like you don’t fully understand how your answer works.  `printf "%q" "my|string"` outputs `my\|string`, and so it will work even if you use `|` as the delimiter!  (4) In your answer, you put the `$(…)` command substitutions outside quotes.  This will cause it to break if initial\_text or replace\_text contains space(s). … (Cont’d) – G-Man Says 'Reinstate Monica' Mar 27 '20 at 04:33
  • (Cont’d) …  (5) Your fourth paragraph (the third one after the first code block) uses sloppy terminology.  You say ``"$(printf "%q" "$replace_text")"`` converts a regex string to an escaped string.  But replace\_text isn’t a regex — the replacement string in an **`s`** command is a specially formatted string that needs special handling, but it’s not a regex.  Please try to use correct terminology. (6) Your initial example is misleading.  In the context you show, `{` and `}` aren’t special characters and don’t need to be escaped. – G-Man Says 'Reinstate Monica' Mar 27 '20 at 04:34
  • I concede that the "replace" and the python options are simpler, but as stated my solution is for a minimal docker container. There isn't really another solution here for that use case. Okay onto your points: (1) - valid, my understanding of printf mistook "ready for shell interpretation" as "regex safe" - clearly not the same thing. Will look for a replacement and update the answer (and my own code!). (2/3) fair, people not knowledgeable in sed wouldn't get that i meant any single char. Will update. Your insight is interesting. Any Regexp char works, if we know that others are escaped. – ThatGuyCalledRob Mar 27 '20 at 16:12
  • Cont.. (4) beginner mistake, will update. (5) fair will update wording here, alongside new regex escaping solution. (6) ok i use a different string in my case and i adapted it for this answer, will change to be a little simpler – ThatGuyCalledRob Mar 27 '20 at 16:14
  • Update: [this solution](https://stackoverflow.com/questions/29613304/is-it-possible-to-escape-regex-metacharacters-reliably-with-sed) seems to fit my purpooses - will be swapping out the printf for this and i have a working solution – ThatGuyCalledRob Mar 27 '20 at 16:21
2

I pieced together a few other answers and came up with this:

function unregex {
   # This is a function because dealing with quotes is a pain.
   # http://stackoverflow.com/a/2705678/120999
   sed -e 's/[]\/()$*.^|[]/\\&/g' <<< "$1"
}
function fsed {
   local find=$(unregex "$1")
   local replace=$(unregex "$2")
   shift 2
   # sed -i is only supported in GNU sed.
   #sed -i "s/$find/$replace/g" "$@"
   perl -p -i -e "s/$find/$replace/g" "$@"
}
Xiong Chiamiov
  • 966
  • 8
  • 11
  • Doesn't work with newlines. Also doesn't help to escape newlines with `\n`. Any solution? – adrelanos Mar 29 '19 at 08:44
  • find and replace need different escaping, see the answer by `VasyaNovikov`. `\n` can be escaped with `sed ':a;N;$!ba;s,\n,\\n,g'` – milahu Oct 01 '21 at 14:50
1

Node.JS equivalent of @Nowaker:

export FNAME='moo.txt'
export FIND='search'
export REPLACE='rpl'
node -e 'fs=require("fs");fs.readFile(process.env.FNAME,"utf8",(err,data)=>{if(err!=null)throw err;fs.writeFile(process.env.FNAME,data.replace(process.env.FIND,process.env.REPLACE),"utf8",e=>{if(e!=null)throw e;});});'
A T
  • 781
  • 1
  • 11
  • 26
1

Heres one more "almost" working way.

Use vi or vim.

Create a textfile with your substitution in it:

:%sno/my search string \\"-:#2;g('.j');\\">/my replacestring=\\"bac)(o:#46;\\">/
:x

then execute vi or vim from the commandline:

vi -S commandfile.txt path/to/the/file

:%sno is the vi command to do search and replace without magic.

/ is my chosen separator.

:x saves and exits vi.

You need to escape backslashes '\' the forwardslash '/' may be replaced with e.g. a questionmark '?' or something else that is not in your search or replace-string, pipe '|' did not work for me tho.

ref: https://stackoverflow.com/questions/6254820/perform-a-non-regex-search-replace-in-vim https://vim.fandom.com/wiki/Search_without_need_to_escape_slash http://linuxcommand.org/lc3_man_pages/vim1.html

1

Using a Simple Python Script

Most systems have python ready to go these days. So here's a simple script that'll work for ya:

# replace.py
# USAGE: python replace.py bad-word good-word target-file.txt
#
import sys

search_term = sys.argv[1]
replace_term = sys.argv[2]
target_file = sys.argv[3]

with open(target_file, 'r') as file:
        content = file.read()

content = content.replace(sys.argv[1], sys.argv[2])

with open(target_file, 'w') as file:
        file.write(content)

One Caveat: This works great if your good/bad words are already in system/environment variables. Just make sure you use double-quotes to wrap the variables when passing to the script.

For example:

python replace.py "$BAD_WORD" "$GOOD_WORD" target-file.txt

However, these will not work:

# This breaks on $ or " characters
BAD_WORD="your-artibrary-string"

# This breaks on ' characters
BAD_WORD='your-artibrary-string'

# This breaks on spaces plus a variety of characters
BAD_WORD=your-artibrary-string

Handling Arbitrary Literal Characters

1. Write the Chars to Disk

If I need to provide a arbitrary literal value to a script (skipping any escaping), I generally write it to disk using this method:

head -c -1 << 'CRAZY_LONG_EOF_MARKER' | tee /path/to/file > /dev/null
arbitrary-one-line-string
CRAZY_LONG_EOF_MARKER

... where:

  • We're employing the Here Document mechanism to write literal text
  • We're using head and tee to delete the trailing newline that Here Docs create
  • We're preventing evalution of variables inside the Here Doc by quoting the EOL marker string

Here's a quick demo with tricky chars:

head -c -1 << 'CRAZY_LONG_EOF_MARKER' | tee /path/to/file > /dev/null
1"2<3>4&5'6$7 # 8
CRAZY_LONG_EOF_MARKER

2. Use Modified Python Script

Here's an updated script that reads from word files:

# replace.py
# USAGE: python replace.py bad-word.txt good-word.txt target-file.txt
#
import sys

search_term_file = sys.argv[1]
replace_term_file = sys.argv[2]
target_file = sys.argv[3]

print [search_term_file, replace_term_file, target_file]

with open(search_term_file, 'r') as file:
        search_term = file.read()
with open(replace_term_file, 'r') as file:
        replace_term = file.read()
with open(target_file, 'r') as file:
        content = file.read()

print [search_term, replace_term]
content = content.replace(search_term, replace_term)

with open(target_file, 'w') as file:
        file.write(content)
Ryan
  • 121
  • 3
1

here is an fsed implementation with grep and dd (via)

limitation: the pattern can match only single lines = the pattern cannot contain \n

this version will replace only the first match. to replace more matches, remove -m 1 and loop all matches

fixedReplaceFirst(){ # aka fsed (fixed string editor)
  input="$1"
  pattern="$2"
  replace="$3"
  match="$(echo "$input" | grep -b -m 1 -o -E "$pattern")"
  offset1=$(echo "$match" | cut -d: -f1)
  match="$(echo "$match" | cut -d: -f2-)"
  matchLength=${#match}
  offset2=$(expr $offset1 + $matchLength)
  echo "$input" | dd bs=1 status=none count=$offset1
  echo -n "$replace"
  echo "$input" | dd bs=1 status=none skip=$offset2
}

read -d '' input <<EOF
#%replace_me

here,
#%replace_me
is not replaced
EOF

read -d '' replace <<EOF
a=1
b=2
EOF

fixedReplaceFirst "$input" "^#%replace_me$" "$replace"

there is rpl (python), but it requires a regular file, so it does not work to replace stdin to stdout

$ echo $'[(\n.*' >temp.txt
$ rpl $'[(\n.*' 'yep'
$ cat temp.txt
yep

there is replace (C, nixpkgs), but it fails on newlines

$ replace-literal '[(.*' 'yep' <<< '[(.*'
yep

$ replace-literal $'[(\n.*' 'yep' <<< $'[(\n.*'
[(
.*
milahu
  • 150
  • 9
1

just in case I will need this in the future again

subject=' http://a\tbc/ef.png?&;wert="3$3*4%5"#eft def '
search='http://a\tbc/ef.png?&;wert="3$3*4%5"#eft'
replace='e&*$%\tf.png'
echo "$subject" | awk -v srch=$(printf "%s" "$search"| sed -e 's/\\/\\\\\\\\/g' -e 's/[?&*.$]/\\\\\0/g' ) -v replc=$(printf "%s" "$replace"| sed -e 's/\\/\\\\/g'  -e 's/[?&]/\\\\\0/g'  ) '{gsub(srch, replc, $0); print}' 2> /dev/null

srch and replc are escaped in as subshell not sure if it actually adds value here, but this should get around mostly all special chars

Summer-Sky
  • 181
  • 1
  • 6
1

After reading through the many answers here and not finding a straightforward way to do find + replace with string literals (not regular expressions) with sed / git grep:

I wrote a small CLI tool to do this:

go install -v github.com/paralin/git-find-replace@main
cd ./my/git/repo
git find-replace 'SearchString' 'ReplaceString'

The source code for the tool is available on GitHub.

Christian Stewart
  • 704
  • 2
  • 5
  • 15
0

Bash can do it!

zInput='string .* to .* search'
zMatch='.*'
zResult=''
while :; do
    [[ "$zInput" =~ (.*)("$zMatch")(.*) ]] || break
    zResult="replacement${BASH_REMATCH[3]}$zResult"
    zInput="${BASH_REMATCH[1]}"
done
zResult="$zInput$zResult"
echo "$zResult"

outputs

string replacement to replacement search

Quoted strings (no matter how you quote them ('' "" $'')) on the right side of the =~ are not interpreted as regex, but instead taken literally.

Result is built backwards because leftmost .* is greedy.

Paul
  • 103
  • 7