Bash use variable in 'for' keyword [duplicate] - bash

Question

This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
In bash, how do I expand a wildcard while it's inside double quotes?
(1 answer)
Closed 5 months ago.
How to get correctly work?
Below line is working but need use argument in 'for'.
for f in ~/files/*/*.txt do
Code:
list ()
{
for f in $1; do
echo $f
done
}
list "~/files/*/*.txt"
list "~/files/*.txt"
Output:
~/files/*/*.txt
~/files/*.txt

Answer 1

The problem lies in the expansion order in bash.
There is a certain logic in what you did.
It would work with list "./dir/*/*.txt" for example.
You enclosed list argument with double quotes, so that pattern expansion is not done when calling list.
So, $1 in "list" function is literally ./dir/*/*.txt. As you wanted it to be.
Then you don't enclose $1 in the for instruction, so that it is expanded into a list of file names.
So it does what you want.
Or almost so.
Problem is that the order of expansion is :
~ then parameters then patterns (plus other irrelevant in between).
So, when $1 is substituted by its value, it is too late to substitute ~.
If you had done that (not at all a suggestion. Just an explanation) :
list ()
{
for f in ~/$1; do
echo $f
done
}
list "files/*/*.txt"
list "files/*.txt"
It would have worked. Not a solution obviously. But it helps understand what happens: "files/*.txt" is literally passed to "list".
Then
for f in ~/$1
is transformed into
for f in /home/you/$1 [~ substitution]
then transformed into
for f in /home/you/files/*.txt [parameter substitution]
then transformed into
for f in /home/you/files/a.txt /home/you/files/b.txt [pattern expansion]
Now for the solution, quoting the arguments and then using $#, as suggested in comments, would do the trick indeed.
If you don't quote the argument and call
list ~/files/*.txt
Then expansion will occur before the call.
list ~/files/*.txt
is transformed into
list /home/you/files/*.txt
then info
list /home/you/files/a.txt /home/you/files/b.txt.
Then passed to list.
But then, inside list, what you have are 2 arguments.
So indeed, for the for the for, then, you need to use "$#"
list ()
{
for f in "$#"; do
echo $f
done
}
list ~/files/list.txt
Another way, if you have a reason to want the expansion to occurs inside "list" (for example, if you may want to pass two of those patterns), would be to force the expansion after the parameter substitution.
There is no ideal way to do that.
Either you need to recode (or use external commands, such as realpath) the path expansion.
Or you use "eval" to force double evaluation of your "$1". But that is a huge security breach if the arguments come from the user (one could use $(rm -fr /) as an argument, and eval would execute it), plus it can also be tricky, if you have, for example, filenames containing "$".
If you know that the patterns will always look like your examples (maybe a tilde and some * and likes) then you could just do the tilde substitution yourself and keep the rest of the code as is
list ()
{
param=${1/#\~/$HOME}
for f in $param; do
echo $f
done
}
list "~/files/list.txt"
Not the best solution. But the one closest to yours.
tl;dr:
The problem is the order of substitution is bash. You need to understand how bash works by rewriting commands in several stages before execution.
More specifically, because ~ is expanded before parameters and variables. So if x="~/*", then echo $x means echo $x after ~ expansion (no ~ in echo $x), then echo ~/* after variable expansion ($x is replaced by its value), and then echo ~/* after * expansion (since you have to directory literally named ~, * matches nothing).
The easiest solution is to have list take many arguments, not just one, let the expansion occurs before the call to list (so not enclosing argument to list in "), and then, rewrite list by taking into account that $1 is just the first of many arguments.
If you insist on having a single argument to list, you have to deal with potential ~ yourself. Like with ${1/#\~/$HOME} if ~ are always single ~ (not ~user) at the beginning of the pattern.