Index range of array doesn't allow you to iterate over a new line in bash
I'm working on a simple bash script that fixes a duplicate naming issue on a retropie.
The script itself grabs any name that is mentioned more than once in a gameslist.xml file, then stores those in an array for later use.
I end up looping over this array in indexes like so:
pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
which pulls the 1st to the 10th element (i.e. ${game_array[9]}
)
however the output gets joined to one line:
pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
R.B.I. Baseball '94 World Series Baseball '95 Mega Games 1 Bill Walsh College Football T2: The Arcade Game Sonic & Knuckles + Sonic the Hedgehog Sega Top Five Pyramid Magic Tecmo Super Baseball Super Chinese Tycoon
But if I loop over the entire array, it outputs on new lines as expected:
pi@retropie:~ $ for game in ${game_array[@]}; do echo $game; done | head -10
R.B.I. Baseball '94
World Series Baseball '95
Mega Games 1
Bill Walsh College Football
T2: The Arcade Game
Sonic & Knuckles + Sonic the Hedgehog
Sega Top Five
Pyramid Magic
Tecmo Super Baseball
Super Chinese Tycoon
the field seperator has been set to new line with IFS='$n'
which is why the second one works, but I can't for the life of me figure out why it isn't with the first one?
here's the full test script for context:
#!/bin/bash
user_input=$1
while [ -z "$user_input" ]; do
echo "please enter the name of the system you want to fix the game list for"
echo "(as it is labelled in /home/pi/RetroPie/roms)"
read -r user_input
done
ls "/home/pi/RetroPie/roms/$user_input" >/dev/null 2>&1
if [ "$?" -ne 0 ]; then
echo "this doesn't appear to be a system installed here. exiting."
exit 1
fi
games_to_fix()
{
IFS=$'n'
console=$1
filepath="/opt/retropie/configs/all/emulationstation/gamelists/$console/gamelist.xml"
game_array=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $0}'| cut -d ">" -f 2 | cut -d "<" -f 1))
number_to_fix=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $1}'))
}
get_new_name()
{
mYpath=$1
new_name=$(echo $mYpath | cut -d ">" -f 2 | cut -d "<" -f 1 | sed -e 's/.///g' | sed -e 's/.7z//g')
}
games_to_fix $user_input
IFS=$'n'
index=0
for i in ${number_to_fix[@]}; do
loop=1
for game in ${game_array[@]:$index:$i}; do
# for ind in $(eval echo {1..$i}); do
line_number=$(fgrep -n "<name>$game</name>" $filepath | awk '{print $1}' | cut -d : -f 1 | sed -e "${loop}q;d")
path_line_number=$(expr $line_number - 1 )
path=$(sed "${path_line_number}q;d" $filepath | cut -d : -f 2)
get_new_name "$path"
sed -i "${line_number}s/$game/$new_name/g" $filepath
((loop++))
done
index=$(expr index + $i);
done
bash shell quoting array
add a comment |
I'm working on a simple bash script that fixes a duplicate naming issue on a retropie.
The script itself grabs any name that is mentioned more than once in a gameslist.xml file, then stores those in an array for later use.
I end up looping over this array in indexes like so:
pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
which pulls the 1st to the 10th element (i.e. ${game_array[9]}
)
however the output gets joined to one line:
pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
R.B.I. Baseball '94 World Series Baseball '95 Mega Games 1 Bill Walsh College Football T2: The Arcade Game Sonic & Knuckles + Sonic the Hedgehog Sega Top Five Pyramid Magic Tecmo Super Baseball Super Chinese Tycoon
But if I loop over the entire array, it outputs on new lines as expected:
pi@retropie:~ $ for game in ${game_array[@]}; do echo $game; done | head -10
R.B.I. Baseball '94
World Series Baseball '95
Mega Games 1
Bill Walsh College Football
T2: The Arcade Game
Sonic & Knuckles + Sonic the Hedgehog
Sega Top Five
Pyramid Magic
Tecmo Super Baseball
Super Chinese Tycoon
the field seperator has been set to new line with IFS='$n'
which is why the second one works, but I can't for the life of me figure out why it isn't with the first one?
here's the full test script for context:
#!/bin/bash
user_input=$1
while [ -z "$user_input" ]; do
echo "please enter the name of the system you want to fix the game list for"
echo "(as it is labelled in /home/pi/RetroPie/roms)"
read -r user_input
done
ls "/home/pi/RetroPie/roms/$user_input" >/dev/null 2>&1
if [ "$?" -ne 0 ]; then
echo "this doesn't appear to be a system installed here. exiting."
exit 1
fi
games_to_fix()
{
IFS=$'n'
console=$1
filepath="/opt/retropie/configs/all/emulationstation/gamelists/$console/gamelist.xml"
game_array=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $0}'| cut -d ">" -f 2 | cut -d "<" -f 1))
number_to_fix=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $1}'))
}
get_new_name()
{
mYpath=$1
new_name=$(echo $mYpath | cut -d ">" -f 2 | cut -d "<" -f 1 | sed -e 's/.///g' | sed -e 's/.7z//g')
}
games_to_fix $user_input
IFS=$'n'
index=0
for i in ${number_to_fix[@]}; do
loop=1
for game in ${game_array[@]:$index:$i}; do
# for ind in $(eval echo {1..$i}); do
line_number=$(fgrep -n "<name>$game</name>" $filepath | awk '{print $1}' | cut -d : -f 1 | sed -e "${loop}q;d")
path_line_number=$(expr $line_number - 1 )
path=$(sed "${path_line_number}q;d" $filepath | cut -d : -f 2)
get_new_name "$path"
sed -i "${line_number}s/$game/$new_name/g" $filepath
((loop++))
done
index=$(expr index + $i);
done
bash shell quoting array
I'm sceptical. What's in the array?declare -p game_array
– glenn jackman
Jul 31 at 17:47
Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for examplearr=("foo bar" "baz" "bam foo" "bar")
;for item in "${arr[@]:1:2}"; do echo "$item"; done
– steeldriver
Jul 31 at 17:56
add a comment |
I'm working on a simple bash script that fixes a duplicate naming issue on a retropie.
The script itself grabs any name that is mentioned more than once in a gameslist.xml file, then stores those in an array for later use.
I end up looping over this array in indexes like so:
pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
which pulls the 1st to the 10th element (i.e. ${game_array[9]}
)
however the output gets joined to one line:
pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
R.B.I. Baseball '94 World Series Baseball '95 Mega Games 1 Bill Walsh College Football T2: The Arcade Game Sonic & Knuckles + Sonic the Hedgehog Sega Top Five Pyramid Magic Tecmo Super Baseball Super Chinese Tycoon
But if I loop over the entire array, it outputs on new lines as expected:
pi@retropie:~ $ for game in ${game_array[@]}; do echo $game; done | head -10
R.B.I. Baseball '94
World Series Baseball '95
Mega Games 1
Bill Walsh College Football
T2: The Arcade Game
Sonic & Knuckles + Sonic the Hedgehog
Sega Top Five
Pyramid Magic
Tecmo Super Baseball
Super Chinese Tycoon
the field seperator has been set to new line with IFS='$n'
which is why the second one works, but I can't for the life of me figure out why it isn't with the first one?
here's the full test script for context:
#!/bin/bash
user_input=$1
while [ -z "$user_input" ]; do
echo "please enter the name of the system you want to fix the game list for"
echo "(as it is labelled in /home/pi/RetroPie/roms)"
read -r user_input
done
ls "/home/pi/RetroPie/roms/$user_input" >/dev/null 2>&1
if [ "$?" -ne 0 ]; then
echo "this doesn't appear to be a system installed here. exiting."
exit 1
fi
games_to_fix()
{
IFS=$'n'
console=$1
filepath="/opt/retropie/configs/all/emulationstation/gamelists/$console/gamelist.xml"
game_array=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $0}'| cut -d ">" -f 2 | cut -d "<" -f 1))
number_to_fix=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $1}'))
}
get_new_name()
{
mYpath=$1
new_name=$(echo $mYpath | cut -d ">" -f 2 | cut -d "<" -f 1 | sed -e 's/.///g' | sed -e 's/.7z//g')
}
games_to_fix $user_input
IFS=$'n'
index=0
for i in ${number_to_fix[@]}; do
loop=1
for game in ${game_array[@]:$index:$i}; do
# for ind in $(eval echo {1..$i}); do
line_number=$(fgrep -n "<name>$game</name>" $filepath | awk '{print $1}' | cut -d : -f 1 | sed -e "${loop}q;d")
path_line_number=$(expr $line_number - 1 )
path=$(sed "${path_line_number}q;d" $filepath | cut -d : -f 2)
get_new_name "$path"
sed -i "${line_number}s/$game/$new_name/g" $filepath
((loop++))
done
index=$(expr index + $i);
done
bash shell quoting array
I'm working on a simple bash script that fixes a duplicate naming issue on a retropie.
The script itself grabs any name that is mentioned more than once in a gameslist.xml file, then stores those in an array for later use.
I end up looping over this array in indexes like so:
pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
which pulls the 1st to the 10th element (i.e. ${game_array[9]}
)
however the output gets joined to one line:
pi@retropie:~ $ for game in ${game_array[@]:0:10} ; do echo $game; done
R.B.I. Baseball '94 World Series Baseball '95 Mega Games 1 Bill Walsh College Football T2: The Arcade Game Sonic & Knuckles + Sonic the Hedgehog Sega Top Five Pyramid Magic Tecmo Super Baseball Super Chinese Tycoon
But if I loop over the entire array, it outputs on new lines as expected:
pi@retropie:~ $ for game in ${game_array[@]}; do echo $game; done | head -10
R.B.I. Baseball '94
World Series Baseball '95
Mega Games 1
Bill Walsh College Football
T2: The Arcade Game
Sonic & Knuckles + Sonic the Hedgehog
Sega Top Five
Pyramid Magic
Tecmo Super Baseball
Super Chinese Tycoon
the field seperator has been set to new line with IFS='$n'
which is why the second one works, but I can't for the life of me figure out why it isn't with the first one?
here's the full test script for context:
#!/bin/bash
user_input=$1
while [ -z "$user_input" ]; do
echo "please enter the name of the system you want to fix the game list for"
echo "(as it is labelled in /home/pi/RetroPie/roms)"
read -r user_input
done
ls "/home/pi/RetroPie/roms/$user_input" >/dev/null 2>&1
if [ "$?" -ne 0 ]; then
echo "this doesn't appear to be a system installed here. exiting."
exit 1
fi
games_to_fix()
{
IFS=$'n'
console=$1
filepath="/opt/retropie/configs/all/emulationstation/gamelists/$console/gamelist.xml"
game_array=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $0}'| cut -d ">" -f 2 | cut -d "<" -f 1))
number_to_fix=($(fgrep "<name>" "$filepath" | sort | uniq -c | sort -rn | awk '$1 > 1 {print $1}'))
}
get_new_name()
{
mYpath=$1
new_name=$(echo $mYpath | cut -d ">" -f 2 | cut -d "<" -f 1 | sed -e 's/.///g' | sed -e 's/.7z//g')
}
games_to_fix $user_input
IFS=$'n'
index=0
for i in ${number_to_fix[@]}; do
loop=1
for game in ${game_array[@]:$index:$i}; do
# for ind in $(eval echo {1..$i}); do
line_number=$(fgrep -n "<name>$game</name>" $filepath | awk '{print $1}' | cut -d : -f 1 | sed -e "${loop}q;d")
path_line_number=$(expr $line_number - 1 )
path=$(sed "${path_line_number}q;d" $filepath | cut -d : -f 2)
get_new_name "$path"
sed -i "${line_number}s/$game/$new_name/g" $filepath
((loop++))
done
index=$(expr index + $i);
done
bash shell quoting array
bash shell quoting array
edited Dec 16 at 21:48
ilkkachu
55.5k783151
55.5k783151
asked Jul 31 at 17:26
RobotJohnny
776316
776316
I'm sceptical. What's in the array?declare -p game_array
– glenn jackman
Jul 31 at 17:47
Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for examplearr=("foo bar" "baz" "bam foo" "bar")
;for item in "${arr[@]:1:2}"; do echo "$item"; done
– steeldriver
Jul 31 at 17:56
add a comment |
I'm sceptical. What's in the array?declare -p game_array
– glenn jackman
Jul 31 at 17:47
Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for examplearr=("foo bar" "baz" "bam foo" "bar")
;for item in "${arr[@]:1:2}"; do echo "$item"; done
– steeldriver
Jul 31 at 17:56
I'm sceptical. What's in the array?
declare -p game_array
– glenn jackman
Jul 31 at 17:47
I'm sceptical. What's in the array?
declare -p game_array
– glenn jackman
Jul 31 at 17:47
Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for example
arr=("foo bar" "baz" "bam foo" "bar")
; for item in "${arr[@]:1:2}"; do echo "$item"; done
– steeldriver
Jul 31 at 17:56
Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for example
arr=("foo bar" "baz" "bam foo" "bar")
; for item in "${arr[@]:1:2}"; do echo "$item"; done
– steeldriver
Jul 31 at 17:56
add a comment |
2 Answers
2
active
oldest
votes
In brief: you should use quotes around array expansions like that, unless you explicitly want field/word splitting. "$@"
expands each positional parameter to a separate word, and similarly so does "${a[@]}"
. By extension, that should work in the same way for "${a[@]:0:2}"
.
That said, it still seems like an inconsistency in Bash, and what you used should work in your case (since there are no glob characters in the values, and field splitting is taken care of by IFS
being set properly).
Taking the full array works:
$ IFS=$'n'
$ a=("foo bar" "asdf ghjk")
$ printf "%sn" ${a[@]}
foo bar
asdf ghjk
Slicing it doesn't work for an array, but works for $@
:
$ printf "%sn" ${a[@]:0:2}
foo bar asdf ghjk
$ set -- "aa bb" "cc dd"
$ printf "%sn" ${@:1:2}
aa bb
cc dd
It does work in ksh and zsh, which sort of underlines Bash being inconsistent here (zsh of course would have its own syntax for the equivalent):
$ ifs=$'n' ksh -c 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
foo bar
asdf ghjk
$ ifs=$'n' zsh -yc 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
foo bar
asdf ghjk
The quoted version works in Bash, too, and is better when you just want the values as-is, since you don't need to depend on IFS
. The default IFS
works fine here even though the array elements have spaces:
$ unset IFS # uses default of $' tn'
$ printf "%sn" "${a[@]:0:2}"
foo bar
asdf ghjk
It looks as though the unquoted ${a[@]:0:2}
joins the elements with spaces, a bit like what happens in Bash in contexts where word-splitting doesn't happen (e.g. str=${a[@]}
). And then it tries to split the result with IFS
, as usual. E.g. here, it splits on the newline in the middle of the second array element:
$ IFS=$'n'
$ a=("foo bar" $'newnline' "asdf ghjk");
$ printf ":%sn" ${a[@]:0:3}
:foo bar new
:line asdf ghjk
As said above, you really should use the quotes around array expansions in most cases, though still, one would assume ${a[@]:n:m}
would result in multiple words, just like ${a[@]}
does.
The behaviour here seems present in Bash 4.4.12(1)-release
and 5.0.0(1)-alpha
. I posted a bug report about it.
Inzsh
, you'd rather use$a[1,2]
(or"${(@)a[1,2]}"
or"${a[@][1,2]}"
to preserve empty elements).
– Stéphane Chazelas
Aug 1 at 12:11
@StéphaneChazelas, doesn't"${a[@]:1:2}"
work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)
– ilkkachu
Aug 1 at 13:23
Yes, I'm just saying there are more idiomatic/native/sensible ways to do it inzsh
(${var:x:y}
was added much later inzsh
and only for compatibility with ksh/bash as it had its own better way to do it before them).bash
copied it fromksh93
, you may want to check the status there.
– Stéphane Chazelas
Aug 1 at 13:26
Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
– Isaac
Aug 2 at 7:56
@Isaac, no, they shouldn't. Bash's reference puts it in plain text re$*
: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for$@
): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."
– ilkkachu
Aug 2 at 8:19
|
show 5 more comments
quote
quote
Quote
Quote
Quote!!
This works:
$ game_array=("foo bar" "baz" "bam foo" "bar")
$ for game in "${game_array[@]:0:10}" ; do echo "$game"; done
foo bar
baz
bam foo
bar
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f459664%2findex-range-of-array-doesnt-allow-you-to-iterate-over-a-new-line-in-bash%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
In brief: you should use quotes around array expansions like that, unless you explicitly want field/word splitting. "$@"
expands each positional parameter to a separate word, and similarly so does "${a[@]}"
. By extension, that should work in the same way for "${a[@]:0:2}"
.
That said, it still seems like an inconsistency in Bash, and what you used should work in your case (since there are no glob characters in the values, and field splitting is taken care of by IFS
being set properly).
Taking the full array works:
$ IFS=$'n'
$ a=("foo bar" "asdf ghjk")
$ printf "%sn" ${a[@]}
foo bar
asdf ghjk
Slicing it doesn't work for an array, but works for $@
:
$ printf "%sn" ${a[@]:0:2}
foo bar asdf ghjk
$ set -- "aa bb" "cc dd"
$ printf "%sn" ${@:1:2}
aa bb
cc dd
It does work in ksh and zsh, which sort of underlines Bash being inconsistent here (zsh of course would have its own syntax for the equivalent):
$ ifs=$'n' ksh -c 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
foo bar
asdf ghjk
$ ifs=$'n' zsh -yc 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
foo bar
asdf ghjk
The quoted version works in Bash, too, and is better when you just want the values as-is, since you don't need to depend on IFS
. The default IFS
works fine here even though the array elements have spaces:
$ unset IFS # uses default of $' tn'
$ printf "%sn" "${a[@]:0:2}"
foo bar
asdf ghjk
It looks as though the unquoted ${a[@]:0:2}
joins the elements with spaces, a bit like what happens in Bash in contexts where word-splitting doesn't happen (e.g. str=${a[@]}
). And then it tries to split the result with IFS
, as usual. E.g. here, it splits on the newline in the middle of the second array element:
$ IFS=$'n'
$ a=("foo bar" $'newnline' "asdf ghjk");
$ printf ":%sn" ${a[@]:0:3}
:foo bar new
:line asdf ghjk
As said above, you really should use the quotes around array expansions in most cases, though still, one would assume ${a[@]:n:m}
would result in multiple words, just like ${a[@]}
does.
The behaviour here seems present in Bash 4.4.12(1)-release
and 5.0.0(1)-alpha
. I posted a bug report about it.
Inzsh
, you'd rather use$a[1,2]
(or"${(@)a[1,2]}"
or"${a[@][1,2]}"
to preserve empty elements).
– Stéphane Chazelas
Aug 1 at 12:11
@StéphaneChazelas, doesn't"${a[@]:1:2}"
work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)
– ilkkachu
Aug 1 at 13:23
Yes, I'm just saying there are more idiomatic/native/sensible ways to do it inzsh
(${var:x:y}
was added much later inzsh
and only for compatibility with ksh/bash as it had its own better way to do it before them).bash
copied it fromksh93
, you may want to check the status there.
– Stéphane Chazelas
Aug 1 at 13:26
Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
– Isaac
Aug 2 at 7:56
@Isaac, no, they shouldn't. Bash's reference puts it in plain text re$*
: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for$@
): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."
– ilkkachu
Aug 2 at 8:19
|
show 5 more comments
In brief: you should use quotes around array expansions like that, unless you explicitly want field/word splitting. "$@"
expands each positional parameter to a separate word, and similarly so does "${a[@]}"
. By extension, that should work in the same way for "${a[@]:0:2}"
.
That said, it still seems like an inconsistency in Bash, and what you used should work in your case (since there are no glob characters in the values, and field splitting is taken care of by IFS
being set properly).
Taking the full array works:
$ IFS=$'n'
$ a=("foo bar" "asdf ghjk")
$ printf "%sn" ${a[@]}
foo bar
asdf ghjk
Slicing it doesn't work for an array, but works for $@
:
$ printf "%sn" ${a[@]:0:2}
foo bar asdf ghjk
$ set -- "aa bb" "cc dd"
$ printf "%sn" ${@:1:2}
aa bb
cc dd
It does work in ksh and zsh, which sort of underlines Bash being inconsistent here (zsh of course would have its own syntax for the equivalent):
$ ifs=$'n' ksh -c 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
foo bar
asdf ghjk
$ ifs=$'n' zsh -yc 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
foo bar
asdf ghjk
The quoted version works in Bash, too, and is better when you just want the values as-is, since you don't need to depend on IFS
. The default IFS
works fine here even though the array elements have spaces:
$ unset IFS # uses default of $' tn'
$ printf "%sn" "${a[@]:0:2}"
foo bar
asdf ghjk
It looks as though the unquoted ${a[@]:0:2}
joins the elements with spaces, a bit like what happens in Bash in contexts where word-splitting doesn't happen (e.g. str=${a[@]}
). And then it tries to split the result with IFS
, as usual. E.g. here, it splits on the newline in the middle of the second array element:
$ IFS=$'n'
$ a=("foo bar" $'newnline' "asdf ghjk");
$ printf ":%sn" ${a[@]:0:3}
:foo bar new
:line asdf ghjk
As said above, you really should use the quotes around array expansions in most cases, though still, one would assume ${a[@]:n:m}
would result in multiple words, just like ${a[@]}
does.
The behaviour here seems present in Bash 4.4.12(1)-release
and 5.0.0(1)-alpha
. I posted a bug report about it.
Inzsh
, you'd rather use$a[1,2]
(or"${(@)a[1,2]}"
or"${a[@][1,2]}"
to preserve empty elements).
– Stéphane Chazelas
Aug 1 at 12:11
@StéphaneChazelas, doesn't"${a[@]:1:2}"
work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)
– ilkkachu
Aug 1 at 13:23
Yes, I'm just saying there are more idiomatic/native/sensible ways to do it inzsh
(${var:x:y}
was added much later inzsh
and only for compatibility with ksh/bash as it had its own better way to do it before them).bash
copied it fromksh93
, you may want to check the status there.
– Stéphane Chazelas
Aug 1 at 13:26
Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
– Isaac
Aug 2 at 7:56
@Isaac, no, they shouldn't. Bash's reference puts it in plain text re$*
: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for$@
): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."
– ilkkachu
Aug 2 at 8:19
|
show 5 more comments
In brief: you should use quotes around array expansions like that, unless you explicitly want field/word splitting. "$@"
expands each positional parameter to a separate word, and similarly so does "${a[@]}"
. By extension, that should work in the same way for "${a[@]:0:2}"
.
That said, it still seems like an inconsistency in Bash, and what you used should work in your case (since there are no glob characters in the values, and field splitting is taken care of by IFS
being set properly).
Taking the full array works:
$ IFS=$'n'
$ a=("foo bar" "asdf ghjk")
$ printf "%sn" ${a[@]}
foo bar
asdf ghjk
Slicing it doesn't work for an array, but works for $@
:
$ printf "%sn" ${a[@]:0:2}
foo bar asdf ghjk
$ set -- "aa bb" "cc dd"
$ printf "%sn" ${@:1:2}
aa bb
cc dd
It does work in ksh and zsh, which sort of underlines Bash being inconsistent here (zsh of course would have its own syntax for the equivalent):
$ ifs=$'n' ksh -c 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
foo bar
asdf ghjk
$ ifs=$'n' zsh -yc 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
foo bar
asdf ghjk
The quoted version works in Bash, too, and is better when you just want the values as-is, since you don't need to depend on IFS
. The default IFS
works fine here even though the array elements have spaces:
$ unset IFS # uses default of $' tn'
$ printf "%sn" "${a[@]:0:2}"
foo bar
asdf ghjk
It looks as though the unquoted ${a[@]:0:2}
joins the elements with spaces, a bit like what happens in Bash in contexts where word-splitting doesn't happen (e.g. str=${a[@]}
). And then it tries to split the result with IFS
, as usual. E.g. here, it splits on the newline in the middle of the second array element:
$ IFS=$'n'
$ a=("foo bar" $'newnline' "asdf ghjk");
$ printf ":%sn" ${a[@]:0:3}
:foo bar new
:line asdf ghjk
As said above, you really should use the quotes around array expansions in most cases, though still, one would assume ${a[@]:n:m}
would result in multiple words, just like ${a[@]}
does.
The behaviour here seems present in Bash 4.4.12(1)-release
and 5.0.0(1)-alpha
. I posted a bug report about it.
In brief: you should use quotes around array expansions like that, unless you explicitly want field/word splitting. "$@"
expands each positional parameter to a separate word, and similarly so does "${a[@]}"
. By extension, that should work in the same way for "${a[@]:0:2}"
.
That said, it still seems like an inconsistency in Bash, and what you used should work in your case (since there are no glob characters in the values, and field splitting is taken care of by IFS
being set properly).
Taking the full array works:
$ IFS=$'n'
$ a=("foo bar" "asdf ghjk")
$ printf "%sn" ${a[@]}
foo bar
asdf ghjk
Slicing it doesn't work for an array, but works for $@
:
$ printf "%sn" ${a[@]:0:2}
foo bar asdf ghjk
$ set -- "aa bb" "cc dd"
$ printf "%sn" ${@:1:2}
aa bb
cc dd
It does work in ksh and zsh, which sort of underlines Bash being inconsistent here (zsh of course would have its own syntax for the equivalent):
$ ifs=$'n' ksh -c 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
foo bar
asdf ghjk
$ ifs=$'n' zsh -yc 'IFS="$ifs"; a=("foo bar" "asdf ghjk"); printf "%sn" ${a[@]:0:2}'
foo bar
asdf ghjk
The quoted version works in Bash, too, and is better when you just want the values as-is, since you don't need to depend on IFS
. The default IFS
works fine here even though the array elements have spaces:
$ unset IFS # uses default of $' tn'
$ printf "%sn" "${a[@]:0:2}"
foo bar
asdf ghjk
It looks as though the unquoted ${a[@]:0:2}
joins the elements with spaces, a bit like what happens in Bash in contexts where word-splitting doesn't happen (e.g. str=${a[@]}
). And then it tries to split the result with IFS
, as usual. E.g. here, it splits on the newline in the middle of the second array element:
$ IFS=$'n'
$ a=("foo bar" $'newnline' "asdf ghjk");
$ printf ":%sn" ${a[@]:0:3}
:foo bar new
:line asdf ghjk
As said above, you really should use the quotes around array expansions in most cases, though still, one would assume ${a[@]:n:m}
would result in multiple words, just like ${a[@]}
does.
The behaviour here seems present in Bash 4.4.12(1)-release
and 5.0.0(1)-alpha
. I posted a bug report about it.
edited Aug 2 at 8:59
answered Jul 31 at 19:01
ilkkachu
55.5k783151
55.5k783151
Inzsh
, you'd rather use$a[1,2]
(or"${(@)a[1,2]}"
or"${a[@][1,2]}"
to preserve empty elements).
– Stéphane Chazelas
Aug 1 at 12:11
@StéphaneChazelas, doesn't"${a[@]:1:2}"
work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)
– ilkkachu
Aug 1 at 13:23
Yes, I'm just saying there are more idiomatic/native/sensible ways to do it inzsh
(${var:x:y}
was added much later inzsh
and only for compatibility with ksh/bash as it had its own better way to do it before them).bash
copied it fromksh93
, you may want to check the status there.
– Stéphane Chazelas
Aug 1 at 13:26
Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
– Isaac
Aug 2 at 7:56
@Isaac, no, they shouldn't. Bash's reference puts it in plain text re$*
: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for$@
): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."
– ilkkachu
Aug 2 at 8:19
|
show 5 more comments
Inzsh
, you'd rather use$a[1,2]
(or"${(@)a[1,2]}"
or"${a[@][1,2]}"
to preserve empty elements).
– Stéphane Chazelas
Aug 1 at 12:11
@StéphaneChazelas, doesn't"${a[@]:1:2}"
work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)
– ilkkachu
Aug 1 at 13:23
Yes, I'm just saying there are more idiomatic/native/sensible ways to do it inzsh
(${var:x:y}
was added much later inzsh
and only for compatibility with ksh/bash as it had its own better way to do it before them).bash
copied it fromksh93
, you may want to check the status there.
– Stéphane Chazelas
Aug 1 at 13:26
Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
– Isaac
Aug 2 at 7:56
@Isaac, no, they shouldn't. Bash's reference puts it in plain text re$*
: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for$@
): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."
– ilkkachu
Aug 2 at 8:19
In
zsh
, you'd rather use $a[1,2]
(or "${(@)a[1,2]}"
or "${a[@][1,2]}"
to preserve empty elements).– Stéphane Chazelas
Aug 1 at 12:11
In
zsh
, you'd rather use $a[1,2]
(or "${(@)a[1,2]}"
or "${a[@][1,2]}"
to preserve empty elements).– Stéphane Chazelas
Aug 1 at 12:11
@StéphaneChazelas, doesn't
"${a[@]:1:2}"
work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)– ilkkachu
Aug 1 at 13:23
@StéphaneChazelas, doesn't
"${a[@]:1:2}"
work fine in zsh too? Of course zsh has its own way of doing that, I'm not arguing against that. Just using zsh as a point of comparison for something I see as a bug in Bash (even though the whole thing is something that probably shouldn't usually be used)– ilkkachu
Aug 1 at 13:23
Yes, I'm just saying there are more idiomatic/native/sensible ways to do it in
zsh
(${var:x:y}
was added much later in zsh
and only for compatibility with ksh/bash as it had its own better way to do it before them). bash
copied it from ksh93
, you may want to check the status there.– Stéphane Chazelas
Aug 1 at 13:26
Yes, I'm just saying there are more idiomatic/native/sensible ways to do it in
zsh
(${var:x:y}
was added much later in zsh
and only for compatibility with ksh/bash as it had its own better way to do it before them). bash
copied it from ksh93
, you may want to check the status there.– Stéphane Chazelas
Aug 1 at 13:26
Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
– Isaac
Aug 2 at 7:56
Using an unquoted ${a[@]} causes the problem, and the problem is not a bug. For example: from man ksh The meaning of $ and $@ is identical when not quoted*. Both unquoted expansions should become one string separated by spaces.
– Isaac
Aug 2 at 7:56
@Isaac, no, they shouldn't. Bash's reference puts it in plain text re
$*
: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for $@
): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."– ilkkachu
Aug 2 at 8:19
@Isaac, no, they shouldn't. Bash's reference puts it in plain text re
$*
: "($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. " and the text in POSIX is similar (for $@
): "Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set. When the expansion occurs in a context where field splitting will be performed, ... each of the non-empty fields shall be further split as described in Field Splitting."– ilkkachu
Aug 2 at 8:19
|
show 5 more comments
quote
quote
Quote
Quote
Quote!!
This works:
$ game_array=("foo bar" "baz" "bam foo" "bar")
$ for game in "${game_array[@]:0:10}" ; do echo "$game"; done
foo bar
baz
bam foo
bar
add a comment |
quote
quote
Quote
Quote
Quote!!
This works:
$ game_array=("foo bar" "baz" "bam foo" "bar")
$ for game in "${game_array[@]:0:10}" ; do echo "$game"; done
foo bar
baz
bam foo
bar
add a comment |
quote
quote
Quote
Quote
Quote!!
This works:
$ game_array=("foo bar" "baz" "bam foo" "bar")
$ for game in "${game_array[@]:0:10}" ; do echo "$game"; done
foo bar
baz
bam foo
bar
quote
quote
Quote
Quote
Quote!!
This works:
$ game_array=("foo bar" "baz" "bam foo" "bar")
$ for game in "${game_array[@]:0:10}" ; do echo "$game"; done
foo bar
baz
bam foo
bar
answered Aug 2 at 7:56
Isaac
11.1k11648
11.1k11648
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f459664%2findex-range-of-array-doesnt-allow-you-to-iterate-over-a-new-line-in-bash%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I'm sceptical. What's in the array?
declare -p game_array
– glenn jackman
Jul 31 at 17:47
Can you try to reduce your code to a minimal testable example that demonstrates the issue? as it stands, it's rather hard to understand exactly what you are asking. Are you sure the problem cannot be solved by proper quoting - for example
arr=("foo bar" "baz" "bam foo" "bar")
;for item in "${arr[@]:1:2}"; do echo "$item"; done
– steeldriver
Jul 31 at 17:56