Bash: How to eliminate a comma at the end of a variable generated by a loop

Question

I have the following bash script which outputs two random numbers on the same line.

#!/bin/bash

for i in 1 2; do
   unset var
   until [ "$var" -lt 10000 ] 2>/dev/null; do
      var="$RANDOM"
done
printf "%s," "${var/%,/}"
done

The output is:

5751,2129,

I am trying to get rid of the comma at the end of $var which I have tried with "${var/%,/}" so that the output of $var = 5751,2129 and can be used. Can anyone help with this?

Answer 1

After you assign like this: var="$RANDOM", the var variable holds a string from the expansion of $RANDOM. In Bash $RANDOM expands to a decimal number from the range from 0 to 32767. There is no , character there, so there is no point in trying to remove , from the expansion of $var later.

The comma characters you observed in the output came from this part of the code:

printf "%s," "${var/%,/}"
# here    ^

This command was called in each iteration of the loop, so each iteration added one comma character to the output.

What has been printed cannot be unprinted. There are nuances:

• You can pipe the output to a filter and the filter may remove some parts of it. The filter may be outside of the script (e.g. you call the_script | the_filter); or you can pipe the output of some part of the script to a filter inside the script. In the latter case the output of the entire script will be filtered; still what has been printed by the part of the script is not unprinted; I mean it does get to the filter. The filter removes it later.
• If you print to a terminal, it's possible to make it overwrite some part(s) of the previous output with new data. There are characters and sequences to move the cursor around; still they all get to the terminal. You can visually hide previous output almost immediately, but if you redirect the output to a file (or to a terminal that doesn't understand the sequences used) then you will find it's all there.

The right way to get rid of the unwanted comma is not to print it in the first place; or to filter it inside the script. There are several ways to do it and it's not my intention to find all of them. I will discuss some of them. I assume you want to know possible approaches also for loops with more than two iterations; maybe for loops where the number of iterations is not known in advance; maybe for loops not enumerated by numbers; maybe for loops that may never finish (e.g. while true instead of for).

Note: you used printf "%s," "${var/%,/}", it prints no trailing newline character. I will try to replicate this behavior if possible.

Few possible approaches:

1. The inside of your loop does not depend on $i. You can get rid of the loop and use two separate variables:

   unset var1
   until [ "$var1" -lt 10000 ] 2>/dev/null; do
      var1="$RANDOM"
   done
   unset var2
   until [ "$var2" -lt 10000 ] 2>/dev/null; do
      var2="$RANDOM"
   done
   printf '%s,%s' "$var1" "$var2"
   

   Notes:

   • It's not DRY.
   • It doesn't scale well. (What if you had for i in {1..100}?)
   • It gets cumbersome if the number of iterations is not known in advance.

2. You can put your current code in a pipe and filter out the trailing comma. Example:

   for i in 1 2; do
      unset var
      until [ "$var" -lt 10000 ] 2>/dev/null; do
         var="$RANDOM"
      done
   printf "%s," "$var"
   done | sed 's/,$//'
   

   Notes:

   • sed (or whatever filter you use) may or may not handle the incomplete line (a line not terminated by the newline character) the loop produces. In case of sed it depends on the implementation.
   • If sed does handle it, it may still terminate its output with a newline.
   • sed (or whatever filter you use) may not handle a line that is too long. The above particular code generates a reasonably short line, but in general (imagine many iterations) the length may be a problem.
   • sed, as a line-oriented tool, must read an entire line before it processes it. In this case it must read its entire input. You won't get anything from it until after all the iterations finish.
   • The loop is run in a subshell. In a general case you may want it to act in the main shell, for whatever reason.

3. You can capture the output from your current code to a variable. At the end you remove the trailing comma while expanding the variable:

   capture="$(for i in 1 2; do
      unset var
      until [ "$var" -lt 10000 ] 2>/dev/null; do
         var="$RANDOM"
      done
   printf "%s," "$var"
   done)"
   printf "%s" "${capture%,}"
   

   Notes:

   • The loop is run in a subshell. In a general case you may want it to act in the main shell, for whatever reason.
   • The code is silent until it reaches the last printf (outside of the loop). You won't get anything until after all the iterations finish.
   • In general a loop can output any number of bytes. I think Bash can store significant amount of data in a variable; then, because printf is a builtin, it can probably handle printf "%s" "${capture%,}" without hitting the limit for the length of a command line. I haven't tested this thoroughly because IMO storing large amount of data in a shell variable is not the best practice anyway. Still the practice may be justified if you know the output is limited in length. (For the record: the above code generates very short output for sure.)
   • Bash cannot store NUL character(s) in a variable (most shells cannot; zsh can). Additionally $() removes all trailing newlines. This means you cannot use a variable to store an arbitrary output and reproduce it later accurately. (For the record: in the above code the fragment inside $() does not generate NULs or trailing newlines.)

4. Instead of capturing output you can make each iteration append to some variable:

   capture=''
   for i in 1 2; do
      unset var
      until [ "$var" -lt 10000 ] 2>/dev/null; do
         var="$RANDOM"
      done
   capture="$capture$var,"
   done
   printf "%s" "${capture%,}"
   

   Notes:

   • The code is run in the main shell (not in a subshell).
   • Limitations of storing data in a variable (see the previous method) still apply.
   • In Bash you can append to the variable with capture+="$var,". (Note: if the integer attribute has been set for the variable then =+ means "add", not "append".)

5. You can detect the last iteration and use a format without ,:

   # this example is more educative with more than two iterations
   for i in {1..5}; do
      unset var
      until [ "$var" -lt 10000 ] 2>/dev/null; do
         var="$RANDOM"
      done
      if [ "$i" -eq 5 ]; then
         printf "%s" "$var"
      else 
         printf "%s," "$var"
      fi
   done
   

   Notes:

   • No subshell.
   • Detecting the last iteration is harder if you don't know the number in advance.
   • It's even harder if you iterate over an array (e.g. for i in "${arr[@]}").
   • Each iteration prints immediately, you get output sequentially. This would work even if the loop was infinite.

6. You can detect the first iteration and use a format without ,. Note you could have used ,%s instead of %s, in your original code; then you would get ,5751,2129 instead of 5751,2129,. With this change any of the above methods that avoids or removes the trailing comma can be converted to a method that avoids or removes the leading comma. The very last method becomes:

   # this example is more educative with more than two iterations
   for i in {1..5}; do
      unset var
      until [ "$var" -lt 10000 ] 2>/dev/null; do
         var="$RANDOM"
      done
      if [ "$i" -eq 1 ]; then
         printf "%s" "$var"
      else 
         printf ",%s" "$var"
      fi
   done
   

   Notes:

   • No subshell.
   • Detecting the first iteration is easy if you always start with 1 (or a fixed unique string in general).
   • But it's harder if you iterate over an array, e.g. for i in "${arr[@]}". You shouldn't check if [ "$i" = "${arr[1]}" ] because there may be an element identical to "${arr[1]}" later in the array. A straightforward way to deal with this is to keep an index for the loop (index=1 before the loop, then increment by one at the end of every iteration) and test its value against 1; I would find such code somewhat cumbersome though.
   • Each iteration prints immediately, you get output sequentially. This would work even if the loop was infinite.

7. You can make , itself come from a variable. Enter the loop with the variable being empty and set it to , at the end of each iteration. This will effectively change the value just once at the end of the first iteration. Example:

   # this example is more educative with more than two iterations
   separator=''
   for i in {1..5}; do
      unset var
      until [ "$var" -lt 10000 ] 2>/dev/null; do
         var="$RANDOM"
      done
      printf "%s%s" "$separator" "$var"
      separator=,
   done
   

   Notes:

   • No subshell.
   • Works well even if iterating over an array.
   • Each iteration prints immediately, you get output sequentially. This would work even if the loop was infinite.

   Personally I find this method quite elegant.

General notes:

• Each snippet generates output without a trailing newline (with a possible exception of the one with sed or another filter). If you need the entire output to form a properly terminated line of text, run printf '\n' (or sole echo) after the loop.
• You want , to be a separator, not a terminator. This means a loop with exactly zero iterations will generate the same empty output as a loop with exactly one iteration, if $var in the iteration expands to an empty string. In our case $var expands to a non-empty string each time and we know we have more than zero iterations; but in a general case using a separator instead of a terminator may lead to ambiguity.