Index range of array doesn't allow you to iterate over a new line in bash












2














I'm working on a simple bash script that fixes a duplicate naming issue on a retropie.



The script itself grabs any name that is mentioned more than once in a gameslist.xml file, then stores those in an array for later use.



I end up looping over this array in indexes like so:



pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done


which pulls the 1st to the 10th element (i.e. ${game_array[9]})
however the output gets joined to one line:



pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
R.B.I. Baseball '94 World Series Baseball '95 Mega Games 1 Bill Walsh College Football T2: The Arcade Game Sonic & Knuckles + Sonic the Hedgehog Sega Top Five Pyramid Magic Tecmo Super Baseball Super Chinese Tycoon


But if I loop over the entire array, it outputs on new lines as expected:



pi@retropie:~ $ for game in ${game_array[@]}; do echo $game; done | head -10
R.B.I. Baseball '94
World Series Baseball '95
Mega Games 1
Bill Walsh College Football
T2: The Arcade Game
Sonic & Knuckles + Sonic the Hedgehog
Sega Top Five
Pyramid Magic
Tecmo Super Baseball
Super Chinese Tycoon


the field seperator has been set to new line with IFS='$n' which is why the second one works, but I can't for the life of me figure out why it isn't with the first one?



here's the full test script for context:



#!/bin/bash

user_input=$1
while [ -z "$user_input" ]; do
echo "please enter the name of the system you want to fix the game list for"
echo "(as it is labelled in /home/pi/RetroPie/roms)"
read -r user_input
done

ls "/home/pi/RetroPie/roms/$user_input" >/dev/null 2>&1

if [ "$?" -ne 0 ]; then
echo "this doesn't appear to be a system installed here. exiting."
exit 1
fi

games_to_fix()
{
IFS=$'n'
console=$1
filepath="/opt/retropie/configs/all/emulationstation/gamelists/$console/gamelist.xml"
game_array=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $0}'| cut -d ">" -f 2 | cut -d "<" -f 1))
number_to_fix=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $1}'))
}

get_new_name()
{
mYpath=$1
new_name=$(echo $mYpath | cut -d ">" -f 2 | cut -d "<" -f 1 | sed -e 's/.///g' | sed -e 's/.7z//g')
}

games_to_fix $user_input

IFS=$'n'
index=0
for i in ${number_to_fix[@]}; do
loop=1
for game in ${game_array[@]:$index:$i}; do
# for ind in $(eval echo {1..$i}); do
line_number=$(fgrep -n "<name>$game</name>" $filepath | awk '{print $1}' | cut -d : -f 1 | sed -e "${loop}q;d")
path_line_number=$(expr $line_number - 1 )
path=$(sed "${path_line_number}q;d" $filepath | cut -d : -f 2)
get_new_name "$path"
sed -i "${line_number}s/$game/$new_name/g" $filepath
((loop++))
done
index=$(expr index + $i);
done









share|improve this question
























  • I'm sceptical. What's in the array? declare -p game_array
    – glenn jackman
    Jul 31 at 17:47










  • Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for example arr=("foo bar" "baz" "bam foo" "bar") ; for item in "${arr[@]:1:2}"; do echo "$item"; done
    – steeldriver
    Jul 31 at 17:56
















2














I'm working on a simple bash script that fixes a duplicate naming issue on a retropie.



The script itself grabs any name that is mentioned more than once in a gameslist.xml file, then stores those in an array for later use.



I end up looping over this array in indexes like so:



pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done


which pulls the 1st to the 10th element (i.e. ${game_array[9]})
however the output gets joined to one line:



pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
R.B.I. Baseball '94 World Series Baseball '95 Mega Games 1 Bill Walsh College Football T2: The Arcade Game Sonic &amp; Knuckles + Sonic the Hedgehog Sega Top Five Pyramid Magic Tecmo Super Baseball Super Chinese Tycoon


But if I loop over the entire array, it outputs on new lines as expected:



pi@retropie:~ $ for game in ${game_array[@]}; do echo $game; done | head -10
R.B.I. Baseball '94
World Series Baseball '95
Mega Games 1
Bill Walsh College Football
T2: The Arcade Game
Sonic &amp; Knuckles + Sonic the Hedgehog
Sega Top Five
Pyramid Magic
Tecmo Super Baseball
Super Chinese Tycoon


the field seperator has been set to new line with IFS='$n' which is why the second one works, but I can't for the life of me figure out why it isn't with the first one?



here's the full test script for context:



#!/bin/bash

user_input=$1
while [ -z "$user_input" ]; do
echo "please enter the name of the system you want to fix the game list for"
echo "(as it is labelled in /home/pi/RetroPie/roms)"
read -r user_input
done

ls "/home/pi/RetroPie/roms/$user_input" >/dev/null 2>&1

if [ "$?" -ne 0 ]; then
echo "this doesn't appear to be a system installed here. exiting."
exit 1
fi

games_to_fix()
{
IFS=$'n'
console=$1
filepath="/opt/retropie/configs/all/emulationstation/gamelists/$console/gamelist.xml"
game_array=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $0}'| cut -d ">" -f 2 | cut -d "<" -f 1))
number_to_fix=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $1}'))
}

get_new_name()
{
mYpath=$1
new_name=$(echo $mYpath | cut -d ">" -f 2 | cut -d "<" -f 1 | sed -e 's/.///g' | sed -e 's/.7z//g')
}

games_to_fix $user_input

IFS=$'n'
index=0
for i in ${number_to_fix[@]}; do
loop=1
for game in ${game_array[@]:$index:$i}; do
# for ind in $(eval echo {1..$i}); do
line_number=$(fgrep -n "<name>$game</name>" $filepath | awk '{print $1}' | cut -d : -f 1 | sed -e "${loop}q;d")
path_line_number=$(expr $line_number - 1 )
path=$(sed "${path_line_number}q;d" $filepath | cut -d : -f 2)
get_new_name "$path"
sed -i "${line_number}s/$game/$new_name/g" $filepath
((loop++))
done
index=$(expr index + $i);
done









share|improve this question
























  • I'm sceptical. What's in the array? declare -p game_array
    – glenn jackman
    Jul 31 at 17:47










  • Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for example arr=("foo bar" "baz" "bam foo" "bar") ; for item in "${arr[@]:1:2}"; do echo "$item"; done
    – steeldriver
    Jul 31 at 17:56














2












2








2







I'm working on a simple bash script that fixes a duplicate naming issue on a retropie.



The script itself grabs any name that is mentioned more than once in a gameslist.xml file, then stores those in an array for later use.



I end up looping over this array in indexes like so:



pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done


which pulls the 1st to the 10th element (i.e. ${game_array[9]})
however the output gets joined to one line:



pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
R.B.I. Baseball '94 World Series Baseball '95 Mega Games 1 Bill Walsh College Football T2: The Arcade Game Sonic &amp; Knuckles + Sonic the Hedgehog Sega Top Five Pyramid Magic Tecmo Super Baseball Super Chinese Tycoon


But if I loop over the entire array, it outputs on new lines as expected:



pi@retropie:~ $ for game in ${game_array[@]}; do echo $game; done | head -10
R.B.I. Baseball '94
World Series Baseball '95
Mega Games 1
Bill Walsh College Football
T2: The Arcade Game
Sonic &amp; Knuckles + Sonic the Hedgehog
Sega Top Five
Pyramid Magic
Tecmo Super Baseball
Super Chinese Tycoon


the field seperator has been set to new line with IFS='$n' which is why the second one works, but I can't for the life of me figure out why it isn't with the first one?



here's the full test script for context:



#!/bin/bash

user_input=$1
while [ -z "$user_input" ]; do
echo "please enter the name of the system you want to fix the game list for"
echo "(as it is labelled in /home/pi/RetroPie/roms)"
read -r user_input
done

ls "/home/pi/RetroPie/roms/$user_input" >/dev/null 2>&1

if [ "$?" -ne 0 ]; then
echo "this doesn't appear to be a system installed here. exiting."
exit 1
fi

games_to_fix()
{
IFS=$'n'
console=$1
filepath="/opt/retropie/configs/all/emulationstation/gamelists/$console/gamelist.xml"
game_array=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $0}'| cut -d ">" -f 2 | cut -d "<" -f 1))
number_to_fix=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $1}'))
}

get_new_name()
{
mYpath=$1
new_name=$(echo $mYpath | cut -d ">" -f 2 | cut -d "<" -f 1 | sed -e 's/.///g' | sed -e 's/.7z//g')
}

games_to_fix $user_input

IFS=$'n'
index=0
for i in ${number_to_fix[@]}; do
loop=1
for game in ${game_array[@]:$index:$i}; do
# for ind in $(eval echo {1..$i}); do
line_number=$(fgrep -n "<name>$game</name>" $filepath | awk '{print $1}' | cut -d : -f 1 | sed -e "${loop}q;d")
path_line_number=$(expr $line_number - 1 )
path=$(sed "${path_line_number}q;d" $filepath | cut -d : -f 2)
get_new_name "$path"
sed -i "${line_number}s/$game/$new_name/g" $filepath
((loop++))
done
index=$(expr index + $i);
done









share|improve this question















I'm working on a simple bash script that fixes a duplicate naming issue on a retropie.



The script itself grabs any name that is mentioned more than once in a gameslist.xml file, then stores those in an array for later use.



I end up looping over this array in indexes like so:



pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done


which pulls the 1st to the 10th element (i.e. ${game_array[9]})
however the output gets joined to one line:



pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
R.B.I. Baseball '94 World Series Baseball '95 Mega Games 1 Bill Walsh College Football T2: The Arcade Game Sonic &amp; Knuckles + Sonic the Hedgehog Sega Top Five Pyramid Magic Tecmo Super Baseball Super Chinese Tycoon


But if I loop over the entire array, it outputs on new lines as expected:



pi@retropie:~ $ for game in ${game_array[@]}; do echo $game; done | head -10
R.B.I. Baseball '94
World Series Baseball '95
Mega Games 1
Bill Walsh College Football
T2: The Arcade Game
Sonic &amp; Knuckles + Sonic the Hedgehog
Sega Top Five
Pyramid Magic
Tecmo Super Baseball
Super Chinese Tycoon


the field seperator has been set to new line with IFS='$n' which is why the second one works, but I can't for the life of me figure out why it isn't with the first one?



here's the full test script for context:



#!/bin/bash

user_input=$1
while [ -z "$user_input" ]; do
echo "please enter the name of the system you want to fix the game list for"
echo "(as it is labelled in /home/pi/RetroPie/roms)"
read -r user_input
done

ls "/home/pi/RetroPie/roms/$user_input" >/dev/null 2>&1

if [ "$?" -ne 0 ]; then
echo "this doesn't appear to be a system installed here. exiting."
exit 1
fi

games_to_fix()
{
IFS=$'n'
console=$1
filepath="/opt/retropie/configs/all/emulationstation/gamelists/$console/gamelist.xml"
game_array=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $0}'| cut -d ">" -f 2 | cut -d "<" -f 1))
number_to_fix=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $1}'))
}

get_new_name()
{
mYpath=$1
new_name=$(echo $mYpath | cut -d ">" -f 2 | cut -d "<" -f 1 | sed -e 's/.///g' | sed -e 's/.7z//g')
}

games_to_fix $user_input

IFS=$'n'
index=0
for i in ${number_to_fix[@]}; do
loop=1
for game in ${game_array[@]:$index:$i}; do
# for ind in $(eval echo {1..$i}); do
line_number=$(fgrep -n "<name>$game</name>" $filepath | awk '{print $1}' | cut -d : -f 1 | sed -e "${loop}q;d")
path_line_number=$(expr $line_number - 1 )
path=$(sed "${path_line_number}q;d" $filepath | cut -d : -f 2)
get_new_name "$path"
sed -i "${line_number}s/$game/$new_name/g" $filepath
((loop++))
done
index=$(expr index + $i);
done






bash shell quoting array






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Dec 16 at 21:48









ilkkachu

55.5k783151




55.5k783151










asked Jul 31 at 17:26









RobotJohnny

776316




776316












  • I'm sceptical. What's in the array? declare -p game_array
    – glenn jackman
    Jul 31 at 17:47










  • Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for example arr=("foo bar" "baz" "bam foo" "bar") ; for item in "${arr[@]:1:2}"; do echo "$item"; done
    – steeldriver
    Jul 31 at 17:56


















  • I'm sceptical. What's in the array? declare -p game_array
    – glenn jackman
    Jul 31 at 17:47










  • Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for example arr=("foo bar" "baz" "bam foo" "bar") ; for item in "${arr[@]:1:2}"; do echo "$item"; done
    – steeldriver
    Jul 31 at 17:56
















I'm sceptical. What's in the array? declare -p game_array
– glenn jackman
Jul 31 at 17:47




I'm sceptical. What's in the array? declare -p game_array
– glenn jackman
Jul 31 at 17:47












Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for example arr=("foo bar" "baz" "bam foo" "bar") ; for item in "${arr[@]:1:2}"; do echo "$item"; done
– steeldriver
Jul 31 at 17:56




Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for example arr=("foo bar" "baz" "bam foo" "bar") ; for item in "${arr[@]:1:2}"; do echo "$item"; done
– steeldriver
Jul 31 at 17:56










2 Answers
2






active

oldest

votes


















3














In brief: you should use quotes around array expansions like that, unless you explicitly want field/word splitting. "$@" expands each positional parameter to a separate word, and similarly so does "${a[@]}". By extension, that should work in the same way for "${a[@]:0:2}".





That said, it still seems like an inconsistency in Bash, and what you used should work in your case (since there are no glob characters in the values, and field splitting is taken care of by IFS being set properly).



Taking the full array works:



$ IFS=$'n'
$ a=("foo bar" "asdf ghjk")
$ printf "%sn" ${a[@]}
foo bar
asdf ghjk


Slicing it doesn't work for an array, but works for $@:



$ printf "%sn" ${a[@]:0:2}
foo bar asdf ghjk

$ set -- "aa bb" "cc dd"
$ printf "%sn" ${@:1:2}
aa bb
cc dd


It does work in ksh and zsh, which sort of underlines Bash being inconsistent here (zsh of course would have its own syntax for the equivalent):



$ ifs=$'n' ksh -c 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
foo bar
asdf ghjk
$ ifs=$'n' zsh -yc 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
foo bar
asdf ghjk


The quoted version works in Bash, too, and is better when you just want the values as-is, since you don't need to depend on IFS. The default IFS works fine here even though the array elements have spaces:



$ unset IFS                         # uses default of $' tn'
$ printf "%sn" "${a[@]:0:2}"
foo bar
asdf ghjk




It looks as though the unquoted ${a[@]:0:2} joins the elements with spaces, a bit like what happens in Bash in contexts where word-splitting doesn't happen (e.g. str=${a[@]}). And then it tries to split the result with IFS, as usual. E.g. here, it splits on the newline in the middle of the second array element:



$ IFS=$'n'
$ a=("foo bar" $'newnline' "asdf ghjk");
$ printf ":%sn" ${a[@]:0:3}
:foo bar new
:line asdf ghjk


As said above, you really should use the quotes around array expansions in most cases, though still, one would assume ${a[@]:n:m} would result in multiple words, just like ${a[@]} does.



The behaviour here seems present in Bash 4.4.12(1)-release and 5.0.0(1)-alpha. I posted a bug report about it.






share|improve this answer























  • In zsh, you'd rather use $a[1,2] (or "${(@)a[1,2]}" or "${a[@][1,2]}" to preserve empty elements).
    – Stéphane Chazelas
    Aug 1 at 12:11












  • @StéphaneChazelas, doesn't "${a[@]:1:2}" work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)
    – ilkkachu
    Aug 1 at 13:23










  • Yes, I'm just saying there are more idiomatic/native/sensible ways to do it in zsh (${var:x:y} was added much later in zsh and only for compatibility with ksh/bash as it had its own better way to do it before them). bash copied it from ksh93, you may want to check the status there.
    – Stéphane Chazelas
    Aug 1 at 13:26












  • Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
    – Isaac
    Aug 2 at 7:56










  • @Isaac, no, they shouldn't. Bash's reference puts it in plain text re $*: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for $@): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."
    – ilkkachu
    Aug 2 at 8:19



















2














quote
quote



Quote



Quote



Quote!!



This works:



$ game_array=("foo bar" "baz" "bam foo" "bar")

$ for game in "${game_array[@]:0:10}" ; do echo "$game"; done
foo bar
baz
bam foo
bar





share|improve this answer





















    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "106"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f459664%2findex-range-of-array-doesnt-allow-you-to-iterate-over-a-new-line-in-bash%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    3














    In brief: you should use quotes around array expansions like that, unless you explicitly want field/word splitting. "$@" expands each positional parameter to a separate word, and similarly so does "${a[@]}". By extension, that should work in the same way for "${a[@]:0:2}".





    That said, it still seems like an inconsistency in Bash, and what you used should work in your case (since there are no glob characters in the values, and field splitting is taken care of by IFS being set properly).



    Taking the full array works:



    $ IFS=$'n'
    $ a=("foo bar" "asdf ghjk")
    $ printf "%sn" ${a[@]}
    foo bar
    asdf ghjk


    Slicing it doesn't work for an array, but works for $@:



    $ printf "%sn" ${a[@]:0:2}
    foo bar asdf ghjk

    $ set -- "aa bb" "cc dd"
    $ printf "%sn" ${@:1:2}
    aa bb
    cc dd


    It does work in ksh and zsh, which sort of underlines Bash being inconsistent here (zsh of course would have its own syntax for the equivalent):



    $ ifs=$'n' ksh -c 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
    foo bar
    asdf ghjk
    $ ifs=$'n' zsh -yc 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
    foo bar
    asdf ghjk


    The quoted version works in Bash, too, and is better when you just want the values as-is, since you don't need to depend on IFS. The default IFS works fine here even though the array elements have spaces:



    $ unset IFS                         # uses default of $' tn'
    $ printf "%sn" "${a[@]:0:2}"
    foo bar
    asdf ghjk




    It looks as though the unquoted ${a[@]:0:2} joins the elements with spaces, a bit like what happens in Bash in contexts where word-splitting doesn't happen (e.g. str=${a[@]}). And then it tries to split the result with IFS, as usual. E.g. here, it splits on the newline in the middle of the second array element:



    $ IFS=$'n'
    $ a=("foo bar" $'newnline' "asdf ghjk");
    $ printf ":%sn" ${a[@]:0:3}
    :foo bar new
    :line asdf ghjk


    As said above, you really should use the quotes around array expansions in most cases, though still, one would assume ${a[@]:n:m} would result in multiple words, just like ${a[@]} does.



    The behaviour here seems present in Bash 4.4.12(1)-release and 5.0.0(1)-alpha. I posted a bug report about it.






    share|improve this answer























    • In zsh, you'd rather use $a[1,2] (or "${(@)a[1,2]}" or "${a[@][1,2]}" to preserve empty elements).
      – Stéphane Chazelas
      Aug 1 at 12:11












    • @StéphaneChazelas, doesn't "${a[@]:1:2}" work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)
      – ilkkachu
      Aug 1 at 13:23










    • Yes, I'm just saying there are more idiomatic/native/sensible ways to do it in zsh (${var:x:y} was added much later in zsh and only for compatibility with ksh/bash as it had its own better way to do it before them). bash copied it from ksh93, you may want to check the status there.
      – Stéphane Chazelas
      Aug 1 at 13:26












    • Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
      – Isaac
      Aug 2 at 7:56










    • @Isaac, no, they shouldn't. Bash's reference puts it in plain text re $*: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for $@): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."
      – ilkkachu
      Aug 2 at 8:19
















    3














    In brief: you should use quotes around array expansions like that, unless you explicitly want field/word splitting. "$@" expands each positional parameter to a separate word, and similarly so does "${a[@]}". By extension, that should work in the same way for "${a[@]:0:2}".





    That said, it still seems like an inconsistency in Bash, and what you used should work in your case (since there are no glob characters in the values, and field splitting is taken care of by IFS being set properly).



    Taking the full array works:



    $ IFS=$'n'
    $ a=("foo bar" "asdf ghjk")
    $ printf "%sn" ${a[@]}
    foo bar
    asdf ghjk


    Slicing it doesn't work for an array, but works for $@:



    $ printf "%sn" ${a[@]:0:2}
    foo bar asdf ghjk

    $ set -- "aa bb" "cc dd"
    $ printf "%sn" ${@:1:2}
    aa bb
    cc dd


    It does work in ksh and zsh, which sort of underlines Bash being inconsistent here (zsh of course would have its own syntax for the equivalent):



    $ ifs=$'n' ksh -c 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
    foo bar
    asdf ghjk
    $ ifs=$'n' zsh -yc 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
    foo bar
    asdf ghjk


    The quoted version works in Bash, too, and is better when you just want the values as-is, since you don't need to depend on IFS. The default IFS works fine here even though the array elements have spaces:



    $ unset IFS                         # uses default of $' tn'
    $ printf "%sn" "${a[@]:0:2}"
    foo bar
    asdf ghjk




    It looks as though the unquoted ${a[@]:0:2} joins the elements with spaces, a bit like what happens in Bash in contexts where word-splitting doesn't happen (e.g. str=${a[@]}). And then it tries to split the result with IFS, as usual. E.g. here, it splits on the newline in the middle of the second array element:



    $ IFS=$'n'
    $ a=("foo bar" $'newnline' "asdf ghjk");
    $ printf ":%sn" ${a[@]:0:3}
    :foo bar new
    :line asdf ghjk


    As said above, you really should use the quotes around array expansions in most cases, though still, one would assume ${a[@]:n:m} would result in multiple words, just like ${a[@]} does.



    The behaviour here seems present in Bash 4.4.12(1)-release and 5.0.0(1)-alpha. I posted a bug report about it.






    share|improve this answer























    • In zsh, you'd rather use $a[1,2] (or "${(@)a[1,2]}" or "${a[@][1,2]}" to preserve empty elements).
      – Stéphane Chazelas
      Aug 1 at 12:11












    • @StéphaneChazelas, doesn't "${a[@]:1:2}" work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)
      – ilkkachu
      Aug 1 at 13:23










    • Yes, I'm just saying there are more idiomatic/native/sensible ways to do it in zsh (${var:x:y} was added much later in zsh and only for compatibility with ksh/bash as it had its own better way to do it before them). bash copied it from ksh93, you may want to check the status there.
      – Stéphane Chazelas
      Aug 1 at 13:26












    • Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
      – Isaac
      Aug 2 at 7:56










    • @Isaac, no, they shouldn't. Bash's reference puts it in plain text re $*: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for $@): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."
      – ilkkachu
      Aug 2 at 8:19














    3












    3








    3






    In brief: you should use quotes around array expansions like that, unless you explicitly want field/word splitting. "$@" expands each positional parameter to a separate word, and similarly so does "${a[@]}". By extension, that should work in the same way for "${a[@]:0:2}".





    That said, it still seems like an inconsistency in Bash, and what you used should work in your case (since there are no glob characters in the values, and field splitting is taken care of by IFS being set properly).



    Taking the full array works:



    $ IFS=$'n'
    $ a=("foo bar" "asdf ghjk")
    $ printf "%sn" ${a[@]}
    foo bar
    asdf ghjk


    Slicing it doesn't work for an array, but works for $@:



    $ printf "%sn" ${a[@]:0:2}
    foo bar asdf ghjk

    $ set -- "aa bb" "cc dd"
    $ printf "%sn" ${@:1:2}
    aa bb
    cc dd


    It does work in ksh and zsh, which sort of underlines Bash being inconsistent here (zsh of course would have its own syntax for the equivalent):



    $ ifs=$'n' ksh -c 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
    foo bar
    asdf ghjk
    $ ifs=$'n' zsh -yc 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
    foo bar
    asdf ghjk


    The quoted version works in Bash, too, and is better when you just want the values as-is, since you don't need to depend on IFS. The default IFS works fine here even though the array elements have spaces:



    $ unset IFS                         # uses default of $' tn'
    $ printf "%sn" "${a[@]:0:2}"
    foo bar
    asdf ghjk




    It looks as though the unquoted ${a[@]:0:2} joins the elements with spaces, a bit like what happens in Bash in contexts where word-splitting doesn't happen (e.g. str=${a[@]}). And then it tries to split the result with IFS, as usual. E.g. here, it splits on the newline in the middle of the second array element:



    $ IFS=$'n'
    $ a=("foo bar" $'newnline' "asdf ghjk");
    $ printf ":%sn" ${a[@]:0:3}
    :foo bar new
    :line asdf ghjk


    As said above, you really should use the quotes around array expansions in most cases, though still, one would assume ${a[@]:n:m} would result in multiple words, just like ${a[@]} does.



    The behaviour here seems present in Bash 4.4.12(1)-release and 5.0.0(1)-alpha. I posted a bug report about it.






    share|improve this answer














    In brief: you should use quotes around array expansions like that, unless you explicitly want field/word splitting. "$@" expands each positional parameter to a separate word, and similarly so does "${a[@]}". By extension, that should work in the same way for "${a[@]:0:2}".





    That said, it still seems like an inconsistency in Bash, and what you used should work in your case (since there are no glob characters in the values, and field splitting is taken care of by IFS being set properly).



    Taking the full array works:



    $ IFS=$'n'
    $ a=("foo bar" "asdf ghjk")
    $ printf "%sn" ${a[@]}
    foo bar
    asdf ghjk


    Slicing it doesn't work for an array, but works for $@:



    $ printf "%sn" ${a[@]:0:2}
    foo bar asdf ghjk

    $ set -- "aa bb" "cc dd"
    $ printf "%sn" ${@:1:2}
    aa bb
    cc dd


    It does work in ksh and zsh, which sort of underlines Bash being inconsistent here (zsh of course would have its own syntax for the equivalent):



    $ ifs=$'n' ksh -c 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
    foo bar
    asdf ghjk
    $ ifs=$'n' zsh -yc 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
    foo bar
    asdf ghjk


    The quoted version works in Bash, too, and is better when you just want the values as-is, since you don't need to depend on IFS. The default IFS works fine here even though the array elements have spaces:



    $ unset IFS                         # uses default of $' tn'
    $ printf "%sn" "${a[@]:0:2}"
    foo bar
    asdf ghjk




    It looks as though the unquoted ${a[@]:0:2} joins the elements with spaces, a bit like what happens in Bash in contexts where word-splitting doesn't happen (e.g. str=${a[@]}). And then it tries to split the result with IFS, as usual. E.g. here, it splits on the newline in the middle of the second array element:



    $ IFS=$'n'
    $ a=("foo bar" $'newnline' "asdf ghjk");
    $ printf ":%sn" ${a[@]:0:3}
    :foo bar new
    :line asdf ghjk


    As said above, you really should use the quotes around array expansions in most cases, though still, one would assume ${a[@]:n:m} would result in multiple words, just like ${a[@]} does.



    The behaviour here seems present in Bash 4.4.12(1)-release and 5.0.0(1)-alpha. I posted a bug report about it.







    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Aug 2 at 8:59

























    answered Jul 31 at 19:01









    ilkkachu

    55.5k783151




    55.5k783151












    • In zsh, you'd rather use $a[1,2] (or "${(@)a[1,2]}" or "${a[@][1,2]}" to preserve empty elements).
      – Stéphane Chazelas
      Aug 1 at 12:11












    • @StéphaneChazelas, doesn't "${a[@]:1:2}" work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)
      – ilkkachu
      Aug 1 at 13:23










    • Yes, I'm just saying there are more idiomatic/native/sensible ways to do it in zsh (${var:x:y} was added much later in zsh and only for compatibility with ksh/bash as it had its own better way to do it before them). bash copied it from ksh93, you may want to check the status there.
      – Stéphane Chazelas
      Aug 1 at 13:26












    • Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
      – Isaac
      Aug 2 at 7:56










    • @Isaac, no, they shouldn't. Bash's reference puts it in plain text re $*: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for $@): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."
      – ilkkachu
      Aug 2 at 8:19


















    • In zsh, you'd rather use $a[1,2] (or "${(@)a[1,2]}" or "${a[@][1,2]}" to preserve empty elements).
      – Stéphane Chazelas
      Aug 1 at 12:11












    • @StéphaneChazelas, doesn't "${a[@]:1:2}" work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)
      – ilkkachu
      Aug 1 at 13:23










    • Yes, I'm just saying there are more idiomatic/native/sensible ways to do it in zsh (${var:x:y} was added much later in zsh and only for compatibility with ksh/bash as it had its own better way to do it before them). bash copied it from ksh93, you may want to check the status there.
      – Stéphane Chazelas
      Aug 1 at 13:26












    • Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
      – Isaac
      Aug 2 at 7:56










    • @Isaac, no, they shouldn't. Bash's reference puts it in plain text re $*: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for $@): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."
      – ilkkachu
      Aug 2 at 8:19
















    In zsh, you'd rather use $a[1,2] (or "${(@)a[1,2]}" or "${a[@][1,2]}" to preserve empty elements).
    – Stéphane Chazelas
    Aug 1 at 12:11






    In zsh, you'd rather use $a[1,2] (or "${(@)a[1,2]}" or "${a[@][1,2]}" to preserve empty elements).
    – Stéphane Chazelas
    Aug 1 at 12:11














    @StéphaneChazelas, doesn't "${a[@]:1:2}" work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)
    – ilkkachu
    Aug 1 at 13:23




    @StéphaneChazelas, doesn't "${a[@]:1:2}" work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)
    – ilkkachu
    Aug 1 at 13:23












    Yes, I'm just saying there are more idiomatic/native/sensible ways to do it in zsh (${var:x:y} was added much later in zsh and only for compatibility with ksh/bash as it had its own better way to do it before them). bash copied it from ksh93, you may want to check the status there.
    – Stéphane Chazelas
    Aug 1 at 13:26






    Yes, I'm just saying there are more idiomatic/native/sensible ways to do it in zsh (${var:x:y} was added much later in zsh and only for compatibility with ksh/bash as it had its own better way to do it before them). bash copied it from ksh93, you may want to check the status there.
    – Stéphane Chazelas
    Aug 1 at 13:26














    Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
    – Isaac
    Aug 2 at 7:56




    Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
    – Isaac
    Aug 2 at 7:56












    @Isaac, no, they shouldn't. Bash's reference puts it in plain text re $*: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for $@): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."
    – ilkkachu
    Aug 2 at 8:19




    @Isaac, no, they shouldn't. Bash's reference puts it in plain text re $*: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for $@): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."
    – ilkkachu
    Aug 2 at 8:19













    2














    quote
    quote



    Quote



    Quote



    Quote!!



    This works:



    $ game_array=("foo bar" "baz" "bam foo" "bar")

    $ for game in "${game_array[@]:0:10}" ; do echo "$game"; done
    foo bar
    baz
    bam foo
    bar





    share|improve this answer


























      2














      quote
      quote



      Quote



      Quote



      Quote!!



      This works:



      $ game_array=("foo bar" "baz" "bam foo" "bar")

      $ for game in "${game_array[@]:0:10}" ; do echo "$game"; done
      foo bar
      baz
      bam foo
      bar





      share|improve this answer
























        2












        2








        2






        quote
        quote



        Quote



        Quote



        Quote!!



        This works:



        $ game_array=("foo bar" "baz" "bam foo" "bar")

        $ for game in "${game_array[@]:0:10}" ; do echo "$game"; done
        foo bar
        baz
        bam foo
        bar





        share|improve this answer












        quote
        quote



        Quote



        Quote



        Quote!!



        This works:



        $ game_array=("foo bar" "baz" "bam foo" "bar")

        $ for game in "${game_array[@]:0:10}" ; do echo "$game"; done
        foo bar
        baz
        bam foo
        bar






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Aug 2 at 7:56









        Isaac

        11.1k11648




        11.1k11648






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Unix & Linux Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f459664%2findex-range-of-array-doesnt-allow-you-to-iterate-over-a-new-line-in-bash%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Morgemoulin

            Scott Moir

            Souastre