198
$ echo -n "apfjxkic-omyuobwd339805ak:60a06cd2ddfad610b9490d359d605407" | base64
YXBmanhraWMtb215dW9id2QzMzk4MDVhazo2MGEwNmNkMmRkZmFkNjEwYjk0OTBkMzU5ZDYwNTQw
Nw==

The output has a return before Nw==. What is the correct way to generate base64 in Linux?

terminal screenshot

psmears
  • 496
  • 3
  • 8
Tiina
  • 2,920
  • 3
  • 17
  • 26
  • 6
    Are you certain the output contains a newline, and it's not just your window wrapping? That command worked fine for me on mac. What OS are you using? – Ian Jul 03 '17 at 05:10
  • @lan you can also see that from http://www.motobit.com/util/base64-decoder-encoder.asp – Tiina Jul 03 '17 at 05:11
  • 55
    RFC 2045, which defined Base64, REQUIRES a newline after 76 characters (max). What makes you think your example is _not_ the correct way? – MSalters Jul 03 '17 at 08:40
  • 35
    @MSalters [RFC 4648](https://tools.ietf.org/html/rfc4648) specifically addresses that issue. *Implementations MUST NOT add line feeds to base-encoded data unless the specification referring to this document explicitly directs base encoders to add line feeds after a specific number of characters.* => this implementation is incorrect according to RFC 4648, as long as it claims to produce 'plain' base64-encoded output. More interestingly, GNU base64 (in question?) manpages specifically refers to RFC 3548, which also specifies no wrapping by default, and which RFC 4648 obsoletes. – Bob Jul 03 '17 at 10:46
  • @MSalters Because I believed "base 64 encoding does not contain white space/tab/\n", and that is also how `java.util.Base64` implements it. – Tiina Jul 04 '17 at 06:23
  • 4
    @Bob: RFC's have a bit less respect for API stability; a base64 tool can't just change its output format without breaking scripts. – MSalters Jul 04 '17 at 06:44
  • 3
    @MSalters I cannot be certain an older version does not exist, but GNU base64 was written in 2004 and AFAICT always claimed to follow RFC 3548. RFC 3548 contains the same "MUST NOT add line feeds" clause. So even the original implementation was "wrong". At the very least, its implementation does not match its documentation. Anyway, you asked for why OP's example is correct and referenced an RFC; my response is the correct RFC that actually defines base64 in isolation. If your answer is "for historical reasons", so be it, but OP isn't wrong here. – Bob Jul 04 '17 at 07:05
  • 2
    It seems to be a mess: https://en.wikipedia.org/wiki/Base64#Implementations_and_history So bacially, base64 is incompatible with itself. Great. – Oskar Skog Jul 04 '17 at 13:53
  • Are there base64 decode implementations that do not silently accept `\n`s in the input? (I hope not, but who knows...) – marcelm Jul 04 '17 at 21:19
  • 2
    @marcelm in java if a base64 string contains `\n`, decoder throws `java.lang.IllegalArgumentException: Illegal base64 character a` – Tiina Jul 05 '17 at 00:48
  • btw, in your example the purpose of base64 is unclear, since input is already ascii text, so base64 would only needlessly inflate data size. – Display Name Jul 05 '17 at 11:44
  • While the example might have been ASCII, it's not wholly safe to assume that the content the OP is concerned about will always be ASCII. – kayleeFrye_onDeck Jul 05 '17 at 21:16
  • Necroed, but: in Java 8 up (since 2014) `java.util.Base64.getMime{Encoder,Decoder}()` respectively adds linebreaks (CRLF) and accepts/removes them – dave_thompson_085 Nov 18 '22 at 04:08

4 Answers4

337

Try:

echo -n "apfjxkic-omyuobwd339805ak:60a06cd2ddfad610b9490d359d605407" | base64 -w 0

From man base64:

-w, --wrap=COLS
Wrap encoded lines after COLS character (default 76). Use 0 to disable line wrapping.

A likely reason for 76 being the default is that Base64 encoding was to provide a way to include binary files in e-mails and Usenet postings which was intended for humans using monitors with 80 characters width. Having a 76-character width as default made that usecase easier.

Kamil Maciorowski
  • 69,815
  • 22
  • 136
  • 202
  • 29
    Oh man, I always just piped this through `tr`. Good to know there's a "proper way". – Score_Under Jul 03 '17 at 17:05
  • The explanation about why the default value is not zero is a mistery for me. – Dherik Jul 25 '19 at 21:01
  • 4
    @Dherik I *guess* it's courtesy towards text processing tools. `base64` encodes arbitrary binary data as text. Tools that expect text usually read one line at a time and [may not deal with very long lines well](https://askubuntu.com/a/716738/693277). If `-w 0` was the default, you would get by default just one line of text; an enormously long line if the input was large. It's better to wrap by default. I think `76` was chosen because it's little less than `80` which is [a sort of de-facto standard for terminals](https://stackoverflow.com/q/6862968/10765659). – Kamil Maciorowski Jul 25 '19 at 21:32
  • @KamilMaciorowski thank you for the information. Every time that I used the `base64` command I needed to pass the `-w 0` (and when I forgot, weird things can happen...), so this default behaviour was very strange to me. – Dherik Jul 25 '19 at 21:35
  • 3
    @Dherik A likely reason is that base64 encoding was to provide a way to include binary files in emails and usenet postings which was intended for humans using monitors with 80 characters width. Having a 76-character width as default made that usecase easier. – Thorbjørn Ravn Andersen Jan 20 '20 at 06:19
  • @ThorbjørnRavnAndersen The answer is now community wiki and your comment has been assimilated. Thank you. – Kamil Maciorowski Jan 21 '20 at 15:48
  • Note that MacOS doesn't have -W. It has -b though. – Thorbjørn Ravn Andersen Jan 22 '20 at 09:11
  • @ThorbjørnRavnAndersen Note you can [edit] the answer and improve it, [*especially* if it's community wiki](https://meta.stackexchange.com/q/11740/355310#comment953392_11741). – Kamil Maciorowski Jan 22 '20 at 11:47
  • @KamilMaciorowski Yes. And I can leave a comment. At my own digression. Feel free to edit your answer as much as you like based on that. – Thorbjørn Ravn Andersen Jan 22 '20 at 14:26
  • Perfect -n fixed my issue. The culprit for me was the echo command appending a newline character BEFORE piping to base64. – Phil_1984_ Jun 23 '20 at 10:14
83

On systems where the -w option to base64 is not available (e.g. Alpine Linux, an Arch Linux initramfs hook, etc.), you can manually process the output of base64:

base64 some_file.txt | tr -d \\n

This is the brute-force approach; instead of getting the program to co-operate, I am using tr to indiscriminately strip every newline on stdout.

Score_Under
  • 931
  • 5
  • 4
  • 3
    Always use the hammer when available. – dna Sep 16 '20 at 11:56
  • 1
    The following was useful for "correcting" a base64-encoded value: `echo 'dGVzdAo=' | base64 -d | tr -d \\n | base64` – Aubrey Lavigne Mar 16 '21 at 00:32
  • Not inferior, just a different and often only approach. As you said there are plenty of systems out there without the `-w` option, including MacOSX.. – nnsense Nov 01 '21 at 20:05
3

For anyone using openssl base64, you can use the -A flag:

 -A                 Process base64 data on one line (requires -a)
 -a                 Perform base64 encoding/decoding (alias -base64)

The following worked for me:

echo -n '{string}' | openssl base64 -A
Redtama
  • 131
  • 3
0

So please use echo -n to remove the line break before redirecting to base64; and use base64 -w 0 to prevent base64 itself to add line break into the output.

me@host:~ 
$ echo -n mypassword | base64 -w 0
bXlwYXNzd29yZA==me@host:~ # <<<<<<<<<<<<<<< notice that no line break added after "==" due to '-w 0'; so me@host is on the same line
$ echo -n 'mypassword' | base64 -w 0
bXlwYXNzd29yZA==me@host:~ # <<<<<<<<<<<<<<<<<< notice adding single quotes does not affect output, so you can use values containing spaces freely

A good way to verify is using od -c to show the actual chars.

me@host:~ 
$ echo -n bXlwYXNzd29yZA== | base64 -d | od -c
0000000   m   y   p   a   s   s   w   o   r   d
0000012

You see no "\n" is added. But if you use "echo" without "-n", od will show "\n":

me@host:~ 
$ echo mypassword | base64 -w 0
bXlwYXNzd29yZAo=me@host:~ 
$ echo bXlwYXNzd29yZAo= | base64 -d | od -c
0000000   m   y   p   a   s   s   w   o   r   d  \n
0000013

At last, create a function will help you in the future:

base64-encode() {
    if [ -z "$@" ]; then
        echo "Encode string with base64; echoing without line break, and base64 does not print line break neither, to not introducing extra chars while redirecting. Provide the string to encode. "
        return 1
    fi
    echo -n "$@" | base64 -w 0  # here I suppose string if containing space is already quoted
}
WesternGun
  • 592
  • 3
  • 12