13

I was wondering if using rm $(ls) to delete files(or rm -r $(ls) to delete directories as well) was safe? Because in all the websites, people give other ways to do this even though this command seems much easier than other commands.

Sergiy Kolodyazhnyy
  • 103,293
  • 19
  • 273
  • 492
posixKing
  • 1,123
  • 2
  • 11
  • 20
  • 3
    Basic answer, no. ls can't handle special characters. I can write an answer in a bit to explain this in more detail – Sergiy Kolodyazhnyy Sep 03 '16 at 01:42
  • 5
    `touch 'foo -r .. bar'` ; `rm $(ls)`; where'd my parent directory go? Also, what kind of alternatives are you seeing that are even more complicated than this? `rm *` is far easier to type and think about, and safer (but not perfectly safe; see Dennis's answer). – Peter Cordes Sep 03 '16 at 09:15
  • 1
    In addition to the excellent answers, be aware that `ls` can vary between implementations, and therefore is non-standard. Depending on what you need, consider alternatives such as `find` and `stat`. You should use `ls` only for human consumption, never for use by other commands or in scripts. – Paddy Landau Sep 06 '16 at 10:35

2 Answers2

26

No, it is not safe, and the commonly used alternative rm * isn't a lot safer.

There are many problems with rm $(ls). As others have already covered in their answers, the output of ls will be split at characters present in the internal field separator.

Best case scenario, it simply doesn't work. Worst case scenario, you intended to remove only files (but not directories) – or selectively remove some files with -i – but there's a file with the name c -rf in the current directory. Let's see what happens.

$ mkdir a
$ touch b
$ touch 'c -rf'
$ rm -i $(ls)
$ ls
c -rf

The command rm -i $(ls) was supposed to remove only files and ask before removing each one, but the command that was ultimately executed read

rm -i a b c -rf

so it did something else entirely.

Note that rm * is only marginally better. With the directory structure as before, it will behave as intended here, but if you have a file called -rf, you're still out of luck.

$ mkdir a
$ touch b
$ touch ./-rf
$ rm -i *
$ ls
-rf

There are several better alternatives. The easiest ones involve only rm and globbing.

  • The command

    rm -- *
    

    will work exactly as intended, where -- signals that everything after it should not be interpreted as an option.

    This has been part of the POSIX utility syntax guidelines for over two decades now. It's widespread, but you shouldn't expect it to be present everywhere.

  • The command

    rm ./*
    

    makes the glob expand differently and thus requires no support from the called utility.

    For my example from above, you can see the command that will ultimately be executed by prepending echo.

    $ echo rm ./*
    rm ./a ./b ./-rf
    

    The leading ./ prevents rm from accidentally treating any of the filenames like options.

Dennis
  • 1,829
  • 1
  • 12
  • 15
  • 1
    Very good point , filenames with - after expansion become flags to `rm`. +1 – Sergiy Kolodyazhnyy Sep 03 '16 at 04:43
  • 1
    Even nastier: `touch 'foo -rf .. bar`. I don't think an attacker can get any higher than the parent directory, unless we can produce a path separator in `ls`'s output. – Peter Cordes Sep 03 '16 at 09:23
  • @Peter attacker would have to have write permissions to remove patent directory in the first place, no ? – Sergiy Kolodyazhnyy Sep 03 '16 at 16:36
  • 1
    @PeterCordes I'm not sure if all versions of *rm* have this failsafe, but on Ubuntu, openSUSE, and Fedora, it says `rm: refusing to remove '.' or '..' directory: skipping '..'` or something similar when trying to remove the parent directory. – Dennis Sep 03 '16 at 17:11
  • @Serg: the attacker just sends you a .zip with that filename in it and lets you shoot yourself in the foot by extracting it and then trying to delete the contents. Or by creating that filename in /var/tmp or something. – Peter Cordes Sep 03 '16 at 18:56
  • @Dennis correct, Just tried that: `$ rm -rf .. rm: refusing to remove '.' or '..' directory: skipping '..'` – Sergiy Kolodyazhnyy Sep 03 '16 at 19:00
7

What this is intended to do?

  • ls lists files in current directory
  • $(ls) substitutes output of ls places that as argument for rm
  • Essentially rm $(ls) is intended to delete all files in current directory

What's wrong with this picture ?

ls cannot properly handle special characters in filename. Unix users generally advised to use different approaches. I've also showed that in a related question about counting filenames. For instance:

$ touch file$'\n'name                                                                                                    
$ ls                                                                                                                     
file?name
$ rm $(ls)
rm: cannot remove 'file': No such file or directory
rm: cannot remove 'name': No such file or directory
$ 

Also, as properly mentioned in Denis's answer, a filename with leading dashes, could be interpreted as argument to rm after substitution, which defeats the purpose of removing filename.

What works

You want to delete files in current directory. So use glob rm *:

$ ls                                                                                                                     
file?name
$ rm $(ls)
rm: cannot remove 'file': No such file or directory
rm: cannot remove 'name': No such file or directory
$ rm *
$ ls
$ 

You can use find command. This tool is frequently recommended for more than just current directory - it can recursively traverse entire directory tree, and operate on files via -exec . . .{} \;

$ touch "file name"                                
$ find . -maxdepth 1 -mindepth 1                                                                                         
./file name
$ find . -maxdepth 1 -mindepth 1 -exec rm {} \;                                                                          
$ ls
$ 

Python doesn't have issue with special characters in filenames, so we could employ that as well(note that this one is for files only, you will need to use os.rmdir() and os.path.isdir() if you want to operate on directories):

python -c 'import os; [ os.remove(i) for i in os.listdir(".") if os.path.isfile(i) ]'

In fact, the command above could be turned into function or alias in ~/.bashrc for brevity. For example,

rm_stuff()
{
    # Clears all files in the current working directory
    python -c 'import os; [ os.remove(i) for i in os.listdir(".") if os.path.isfile(i) ]'

}

Perl version of that would be

perl -e 'use Cwd;my $d=cwd();opendir(DIR,$d); while ( my $f = readdir(DIR)){ unlink $f;}; closedir(DIR)'
Sergiy Kolodyazhnyy
  • 103,293
  • 19
  • 273
  • 492
  • 1
    Your `"$(ls)"` example only works if there's only one file in the directory. You might as well just use tab-completion to expand the filename since it's the only completion. – Peter Cordes Sep 03 '16 at 09:20
  • @PeterCordes indeed, for an odd reason it only works with one file. That's one more argument against using `ls` then :) I'll edit it out – Sergiy Kolodyazhnyy Sep 03 '16 at 15:26
  • I wouldn't call it an odd reason: you either quote `$(ls)` to disable word splitting, or you let word-splitting happen (with disastrous results). The only clean way to pass around a list of multiple strings without treating data as code is with array variables, or with `\0` as a separator, but the shell itself can't do that. Still `IFS=$'\n'` is the least dangerous, but can't match `find -print0 | xargs -0`. Or `grep -l --null`. Or you avoid the whole issue with things like `find -exec rm {} +`. (Note the `+` to pass multiple args to each invocation of rm; WAY more efficient). – Peter Cordes Sep 03 '16 at 15:32
  • @PeterCordes Yup, totally agreed there. But `IFS=$'\n'` will fail in this case ,too, since I've newline in filename, so wordsplitting will treat it as two filenames instead of one. The odd reason , however, is the fact that with default `IFS` which is space,tab,newline , original `rm "$(ls)"` should also fail, it should treat like i said the filename as two separate ones, but it didn't. Typically I use `find` with `-exec` , or `find . . .-print0 | while IFS= read -d'' FILENAME ; do . . . done` structure to deal with filenames. Or one could use `python` , I've added example of that. – Sergiy Kolodyazhnyy Sep 03 '16 at 15:52
  • `"$(ls)"` always expands to one arg because the quotes protect the expansion of `$(ls)` from word-splitting. Just like they protect `"$foo"`. – Peter Cordes Sep 03 '16 at 16:09
  • @PeterCordes so you're saying `rm` would treat `"$(ls)"` as one argument , even if there were multiple filenames ? I know quoting protects from word splitting , not the first time using it , but `"$(ls)"` should be expanding to multiple arguments. What you originally said is correct though - there occurs word splitting, and for filenames with newlines, shell will do word splitting – Sergiy Kolodyazhnyy Sep 03 '16 at 16:21
  • The output of `ls` is a string, command substitution doesn't change that. `$(ls)` is just a flat string. And it's the shell that chooses whether to treat `"$(echo foo bar)"` as one arg or not, not `rm`. – Peter Cordes Sep 03 '16 at 16:25
  • @PeterCordes ah , I see what you're saying. Yup, exactly right. – Sergiy Kolodyazhnyy Sep 03 '16 at 16:26
  • The Bash FAQ (linked in the accepted answer to this question) explains it pretty well. – Peter Cordes Sep 03 '16 at 16:26
  • Oh, there's plenty material on that online. I usually refer to the article I've linked in my answer, since even that wiki that accepted answer has linked references it. Personally i just avoid dealing with filenames in bash. Other programs like find address bash pitfalls a bit better – Sergiy Kolodyazhnyy Sep 03 '16 at 16:41