108

When is #!/bin/bash more appropriate than #!/bin/sh in a shell script?

jpaugh
  • 1,378
  • 10
  • 20
Hendré
  • 833
  • 3
  • 8
  • 13
  • 56
    When you are using `bash` functions and syntax rather than `sh` functions and syntax. – Mokubai Oct 10 '16 at 07:06
  • 6
    See http://stackoverflow.com/questions/5725296/difference-between-sh-and-bash/5725402#5725402 – SPRBRN Oct 10 '16 at 07:18
  • 3
    If it helps anyone, I've noticed that `vim` will highlight `bash`-isms if your script has the `#!/bin/sh` shebang. I only change it to `bash` if things get hairy enough to start needing bash-features. – Seldom 'Where's Monica' Needy Oct 11 '16 at 03:39
  • @SeldomNeedy Some of the things it highlights by default work fine in any POSIX shell though. `$(...)` is particularly obnoxious. Also, some of them are subtle [`<(...)` and `cmd >& file` don't get any error highlighting, for example, they just don't have special highlighting for what they mean, with or without `g:is_bash`] – Random832 Oct 14 '16 at 04:03
  • @Random832 Looks like there's been some activity on this topic recently in [vim's issue-tracker](https://github.com/vim/vim/issues/1147). – Seldom 'Where's Monica' Needy Oct 14 '16 at 04:24
  • @Random832 If the second two idioms you listed are not valid in Bourne shells (which vim asininely assumes under `#!/bin/sh` unless `g:is_bash` or `g:is_posix` are set), would it be a good idea submitting issues about those? – Seldom 'Where's Monica' Needy Oct 14 '16 at 04:43
  • @SeldomNeedy These days you can go a long, long time without tripping over a legacy proprietary Unix (Solaris, HPUX, AIX, etc) but most of those have a `/bin/sh` (and default-$PATH utilities) whose behavior was frozen circa *1992*, in order that none of their customers' dusty shell scripts would be broken by the changes in POSIX.1-1996 (let's not even talk about -2001 or -2008!) I don't recommend writing *new* shell scripts to cope with that anymore - good enough to deal with it if/when it comes up - but as someone who *used to* deal with that all day every day, Vim's behavior has gut appeal. – zwol Oct 14 '16 at 21:19

6 Answers6

153

In short:

  • There are several shells which implement a superset of the POSIX sh specification. On different systems, /bin/sh might be a link to ash, bash, dash, ksh, zsh, &c. (It will always be sh-compatible though – never csh or fish.)

  • As long as you stick to sh features only, you can (and probably even should) use #!/bin/sh and the script should work fine, no matter which shell it is.

  • If you start using bash-specific features (e.g. arrays), you should specifically request bash – because, even if /bin/sh already invokes bash on your system, it might not on everyone else's system, and your script will not run there. (The same of course applies to zsh and ksh.) You can use shellcheck to identify bashisms.

  • Even if the script is for personal use only, you might notice that some OSes change /bin/sh during upgrades – e.g. on Debian it used to be bash, but later was replaced with very minimal dash. Scripts which used bashisms but had #!/bin/sh suddenly broke.

However:

  • Even #!/bin/bash is not very correct. On different systems, bash might live in /usr/bin or /usr/pkg/bin or /usr/local/bin.

  • A more reliable option is #!/usr/bin/env bash, which uses $PATH. (Although the env tool itself isn't strictly guaranteed either, /usr/bin/env still works on more systems than /bin/bash does.)

Toote
  • 101
  • 2
u1686_grawity
  • 426,297
  • 64
  • 894
  • 966
  • 10
    Nice answer. Did you mean to make it CW? – Mokubai Oct 10 '16 at 08:37
  • 3
    Just a remark if bash is run from `/bin/sh` then it will try to mimic `sh` features (see [man page](http://manpages.ubuntu.com/bash.1)), not sure bash specific features are available in this mode. [`env` tool is a posix tool](http://www.unix.com/man-page/posix/1posix/env/) it should be found in most distribution, but that I'm unsure as some may not respect posix. – bric3 Oct 10 '16 at 12:51
  • @Brice: Does POSIX require it to be in `/usr/bin`? – u1686_grawity Oct 10 '16 at 13:05
  • @grawity Unfortunately I don't think one can assume that `sh` lives either under `/bin` or under `/usr/bin`, I'm not sure POSIX standardised path layout so no assumption can be made on the binaries path. In practice though it's very rare to have these kind of variants. As you wrote `#!/usr/bin/env` is probably the most safe way but not bullet proof. One thing to consider when using this `env` trick that it is no longer possible pass arguments to the interpreter, in practice again most of the interpreters have a way to set options at a later point (like `set +x` for bash). – bric3 Oct 10 '16 at 13:40
  • 8
    No, in fact POSIX [specifically states](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html) it can't be assumed to be in `/bin`: "*Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh, and should be determined by interrogation of the PATH returned by getconf PATH, ensuring that the returned pathname is an absolute pathname and not a shell built-in*". Also see [this question](http://unix.stackexchange.com/q/29608/22222) and, specifically [this answer](http://unix.stackexchange.com/a/77586/22222). – terdon Oct 10 '16 at 15:24
  • 12
    Hmm, well, so that means POSIX doesn't actually define a portable way for having `#!` scripts work? – u1686_grawity Oct 10 '16 at 15:30
  • 2
    @grawity: The way POSIX defines is having no `#!` line at all. Per POSIX, a file which is not identified as a binary executable at execution time is invoked via the standard shell interpreter. – R.. GitHub STOP HELPING ICE Oct 10 '16 at 16:45
  • 4
    POSIX sh is **not** Bourne: it's more a derivative of early ksh than it is a direct descendant of Bourne. Distinguishing them is easy: `echo foo ^ cat` emits `foo ^ cat` in POSIX sh, and emits only `foo` in Bourne (as `^` is a pipe character there). – Charles Duffy Oct 10 '16 at 20:15
  • 2
    @grawity The recommendation is to update the `#!` line dynamically at install time: "Furthermore, on systems that support executable scripts (the `"#!"` construct), it is recommended that applications using executable scripts install them using `getconf PATH` to determine the shell pathname and update the `"#!"` script appropriately as it is being installed (for example, with *sed*). [...]" (from the "sh" page) – hvd Oct 11 '16 at 14:22
  • 2
    /usr/bin/env requires `/usr` to be mounted. – Paul Draper Oct 11 '16 at 19:04
  • @Brice Correct, `bash` does behave differently when run as `sh`. IIRC, it disables only the conflicting features, not any which are strictly orthogonal to POSIX sh. – jpaugh Oct 11 '16 at 21:26
18

Use the shebang corresponding to the shell you actually used to develop and debug your script. I.e. if your login shell is bash, and you run your script as executable in your terminal, use #!/bin/bash. Don't just assume that since you haven't used arrays (or whatever bash feature you're aware of), you're safe to pick whatever shell you like. There are many subtle differences between shells (echo, functions, loops, you name it) which cannot be discovered without proper testing.

Consider this: if you leave #!/bin/bash and your users don't have it, they will see a clear error message, something like 

Error: /bin/bash not found

Most users can fix this under one minute by installing the appropriate package. On the other hand, if you replace the shebang by #!/bin/sh and test it on a system where /bin/sh is a symlink to /bin/bash, your users which don't have bash will be in trouble. They'll most likely see a cryptic error message like:

Error in script.sh line 123: error parsing token xyz

This may take hours to fix, and there will be no clue about which shell they should have used.

There aren't many reasons why you'd want to use a different shell in the shebang. One reason is when the shell you have used is not widespread. Another one is to gain performance with sh which is significantly faster on some systems, AND your script will be a performance bottleneck. In that case, test your script thoroughly with the target shell, then change the shebang.

Dmitry Grigoryev
  • 9,151
  • 4
  • 43
  • 77
4

Generally if time is more important than functionality you will use the faster shell. sh is often aliased to dash and tends to be used for root's cron tasks or batch operations where every (nano) second counts.

mckenzm
  • 899
  • 4
  • 14
  • 2
    Loading an extra shell interpreter from the filesystem can negate such an advantage, and anything where nanoseconds (which are in the same order of magnitude as an actual machine cycle) count is the domain of C and assembly language programmers. – rackandboneman Oct 11 '16 at 09:26
  • Nanoseconds only make a difference in cron jobs if you have billions of them, and cron won't be able to handle so many anyway. – Dmitry Grigoryev Oct 11 '16 at 10:16
  • 1
    Even if mckenzm exaggerated a bit, there's no denying that having more performance is better! Don't get hung up on the "nano" part. (Nanoseconds don't even count in C unless it's an inner loop on a real-time system or such!) – jpaugh Oct 11 '16 at 21:30
  • 1
    @jpaugh Unless you're working with a (legacy) system that assumes certain operations will take at least a specific period of time. It happens! – JAB Oct 12 '16 at 18:12
4

You should only ever use #! /bin/sh.

You should not use bash (or zsh, or fish, or ...) extensions in a shell script, ever.

You should only ever write shell scripts that work with any implementation of the shell language (including all the "utility" programs that go along with the shell itself). These days you can probably take POSIX.1-2001 (not -2008) as authoritative for what the shell and utilities are capable of, but be aware that you may one day be called upon to port your script to a legacy system (e.g. Solaris or AIX) whose shell and utilities were frozen circa 1992.

What, seriously?!

Yes, seriously.

Here's the thing: Shell is a terrible programming language. The only thing it has going for it is that /bin/sh is the one and only script interpreter that every Unix installation is guaranteed to have.

Here's the other thing: some iteration of the core Perl 5 interpreter (/usr/bin/perl) is more likely to be available on a randomly selected Unix installation than (/(usr|opt)(/(local|sfw|pkg)?)?/bin/bash is. Other good scripting languages (Python, Ruby, node.js, etc. — I'll even include PHP and Tcl in that category when comparing to shell) are also roughly as available as bash and other extended shells.

Therefore, if you have the option of writing a bash script, you have the option of using a programming language that isn't terrible, instead.

Now, simple shell scripts, the kind that just run a few programs in a sequence from a cron job or something, there's nothing wrong with leaving them as shell scripts. But simple shell scripts don't need arrays or functions or [[ even. And you should only write complicated shell scripts when you don't have any other choice. Autoconf scripts, for instance, are properly still shell scripts. But those scripts have to run on every incarnation of /bin/sh that's relevant to the program being configured. and that means they cannot use any extensions. You probably don't have to care about old proprietary Unixes these days, but you probably should care about current open-source BSDs, some of which don't install bash by default, and embedded environments that give you only a minimal shell and busybox.

In conclusion, the moment you find yourself wanting a feature that's not available in the portable shell language, that is a sign that the script has become too complicated to stay a shell script. Rewrite it in a better language instead.

zwol
  • 1,258
  • 8
  • 23
  • 4
    Sorry but that's a bit silly. Yes, if you're writing something that will be i) distributed and ii) to different environments, you should stick to basic `sh`. That, however, is a far cry from stating that you should *never* use other shells. There are thousands (probably many more) scripts written every day and the vast majority of those will only ever run on a single machine. Your advice makes sense on production code, but not for the daily "sysdaminy" jobs that most scripts are written for. If I know that my script will only ever be used by me and on a Linux machine, bash is fine. – terdon Oct 15 '16 at 11:11
  • 1
    @terdon The point is that if you have the option of writing a bash script, then you also have the option of writing a perl (or whatever) script, and this is *always* the better choice. – zwol Oct 15 '16 at 16:33
  • @terdon The answer is surely not silly, your comment instead is simply naive. Shell scripts are terrible to debug compared to Perl and such, for which one can easily use complete IDEs with graphical debugging and all. Additionally, most of the scripts written every day for some small thing to do tend to be changed and changed all over again, grow, are copied as examples to colleagues or the net etc. There's most likely no reason to take such a risk. – Thorsten Schöning Oct 16 '16 at 17:08
  • @ThorstenSchöning I said *a little* silly. If this were [Unix.se] or even [so] I might even agree. If we're talking about general best practice, or professional programmers, of course it's best to stick to basic POSIX sh. However, this is [su], a site aimed at end users and telling people to never use bash or zsh when writing shell scripts is, in my opinion, extreme. – terdon Oct 16 '16 at 18:05
  • @terdon It is actually *more* important, in my opinion, to discourage end users from writing complex shell scripts. Professionals have a higher skill level on average (thus they are more likely to be able to cope with the pitfalls of complex shell scripting), should know with some precision which environments they need to be portable to, and are being paid not to give up in frustration. – zwol Oct 16 '16 at 18:10
  • (I would still give exactly the same advice to professionals, but with the understanding that they will know the exceptions when they trip over them.) – zwol Oct 16 '16 at 18:10
  • @zwol sure, writing complete shell scripts is almost always a good sign that you should probably be using a real language instead. However, the vast majority of Linux (as opposed to general *nix) users don't know any programming languages. They just know a tiny bit of bash, if that. And that tiny bit probably includes basic things like `[[` or here strings and whatnot that should never ever be used in serious scripts but are fine for the everyday tasks of a single user machine. As I said above, had you posted this on U&L I wouldn't disagree, but the audience here is different. – terdon Oct 16 '16 at 18:15
3

To be even more brief, use sh if portability across the most systems is most important and bash if you'd like to use some of its specific features, such as arrays, if the version of bash supports it.

Spencer Williams
  • 161
  • 3
1
  • sh (for most cases), bash if specifically required (or ksh, or whatever)

The safest path. when hard coded, would be /usr/bin/shellOfChoice but a new convention I am trying to use always now - as 'default locations' via a change PATH may change is:

#!/usr/bin/env sh or
#!/usr/bin/env bash
or for e.g., perl scripts
#!/usr/bin/env perl -w

Of course, when you have a reason for a script to NEVER be automatically taking a new PATH then continue to have it hard-coded - and then /usr/bin/something should be the most likely path to use.

Michael Felt
  • 121
  • 3