I am writing my first bash script and I can't find the solution to my problem.
Lets say I am calling ls -l and I want to save the names of certain files to a variable.
Output of ls -l:
-rw-r--r-- 1 user1 user1 125 Apr 19 2021 aaa
drwxrwxr-x 5 user2 user2 4096 Sep 7 15:54 bbbb
drwxr-xr-x 4 user3 user3 4096 Mär 16 2021 cccc
drwxr-xr-x 7 user1 user1 4096 Mai 18 15:32 asdf
To parse the output I use the following command:
`ls -l | while read a b c d e f g h j; do echo $c $j
Which results to:
user1 aaa
user2 bbbb
user3 cccc
user1 asdf
Now the step I cant figure it out is how to filter out based on on the values of j. Lets say, we have an array values=(aaa cccc). How could I extend my command, so that it prints out the users only if the value of j is a value in the array values ?
Final result should be:
user1 aaa
user3 cccc
How could I extend my command, so that it prints out the users only if the value of j is a value in the array values ?
For each line, check if the second column is in the array. Check if a Bash array contains a value
containsElement () {
local e match="$1"
shift
for e; do [[ "$e" == "$match" ]] && return 0; done
return 1
}
result="user1 aaa
user2 bbbb
user3 cccc
user1 asdf"
values=(aaa cccc)
printf "%s\n" "$result" |
while IFS=' ' read -r user file; do
if containsElement "$file" "${values[#]}"; then
printf "%s %s\n" "$user" "$file"
fi
done
A more crude solution would be to:
... | grep -f <(printf ' %s$\n' "${values[#]}")
It would probably be simpler if your array was an associative one (supported by recent versions of bash):
declare -A values=([aaa]=1 [cccc]=1)
ls -l | while read a b c d e f g h j; do [ -v values[$j] ] && echo "$c $j"; done
If your bash version supports associative arrays but not yet the [ -v var ] test, you can replace it by [ -n "${values[$j]+ok}" ].
But as explained in comments parsing the ls output is strongly discouraged. In your case any file name with spaces or tabs, or even worse, newline characters, would break all this.
If what you want is the owner of each file, use stat. (If a single call to stat per file is too big a bottleneck compared to one call to ls, then you shouldn't be using bash in the first place: use a language which provides direct access to the underlying system calls to retrieve the owner.)
for v in "${values[#]}"; do
user=$(stat ... "$v") # See man stat for the correct invocation
echo "$user $v"
done
Related
I am trying to use bash to merge/combine all text files in a directory with the same prefix into one text file. Thank you :).
directory
111.txt
aaa
aaa
222_1.txt
bbb
222_2.txt
ccc
ccc
333_1.txt
aaa
333_2.txt
ccc
ccc
333_3.txt
bbb
desired
111.txt
aaa
aaa
222.txt
bbb
ccc
ccc
333.txt
aaa
ccc
ccc
bbb
bash
for file in `ls`|cut -d"_" -f1 ; do
cat ${file}_* > ${file}
done
This is a good use of an associative array as a set. Iterate over the file names, trimming the trailing _* from each name before adding it to the associative array. Then you can iterate over the array's keys, treating each one as a filename prefix.
# IMPORTANT: Assumes there are no suffix-less file names that contain a _
declare -A prefixes
for f in *; do
prefixes[${f%_*}]=
done
for f in "${!prefixes[#]}"; do
[ -f "$f".txt ] && continue # 111.txt doesn't need anything done
cat "$f"_* > "$f".txt
done
build a test environment just as you did
mkdir -p tmp/test
cd !$
touch {111,222,333}.{txt,_2.txt,_3.txt}
cat > 111.txt
aaa
aaa
and so on
then you know how to increment filnames :
for i in $( seq 1 3 ) ; do echo $i* ; done
111._2.txt 111._3.txt 111.txt
222._2.txt 222._3.txt 222.txt
333._2.txt 333._3.txt 333.txt
so you make your resulting files and here is the answer of mechanism to your needs :
for i in $( seq 1 9 ) ; do cat $i* >> new.$i.txt ; done
and finaly
ls -l new.[1-3]*
-rw-r--r-- 1 francois francois 34 Aug 4 14:04 new.1.txt
-rw-r--r-- 1 francois francois 34 Aug 4 14:04 new.2.txt
-rw-r--r-- 1 francois francois 34 Aug 4 14:04 new.3.txt
all 3* contents are in new.".txt for example here.
you only have to set the desired file destination to add in the content & if needed but not set in initial question a sorting of datas by alphabetic order or numerical... etc
I'm trying to read into two file (name,number) at the same time and get value of each possible pair.
The two file are like this:
*name1
John
*name2
Paul
*number1
25
*number2
45
What i'm trying to obtain are label and result like:
*name1 *number1 John 25
*name2 *number2 John 45
*name2 *number1 Paul 25
*name2 *number2 Paul 45
Since i come from python i've tried to do it with two loop like this:
name=/home/davide/name.txt
number=/home/davide/number.txt
while read name; do
if [[ ${name:0:1} == "*" ]]; then
n=$(echo $name)
else
while read number; do
if [[ ${number:0:1} == "*" ]]; then
echo $number $n
else
echo $name $number
fi
done < $number
fi
done < $name
I have the first two pair so my guess it's that i need a command to start from the beginning of number again (like seek(0) on python) but i haven't found a similar one for bash.
I also get an "ambiguous redirect" error and i don't understand why.
After setting up your input files:
printf >name.txt '%s\n' '*name1' John '*name2' Paul
printf >number.txt '%s\n' '*number1' 25 '*number2' 45
...the following code:
#!/usr/bin/env bash
name_file=name.txt
number_file=number.txt
while IFS= read -r name1 && IFS= read -r value1; do
while IFS= read -r name2 && IFS= read -r value2; do
printf '%s\n' "$name1 $name2 $value1 $value2"
done <"$number_file"
done <"$name_file"
...properly outputs:
*name1 *number1 John 25
*name1 *number2 John 45
*name2 *number1 Paul 25
*name2 *number2 Paul 45
What changed?
We stopped using name and number both for the filenames and for the values read from them. Because of this, when you ran <$number, it no longer had the filename number.txt in it after the first iteration; likewise for $name.
We started quoting all expansions ("$foo", not $foo). See the http://shellcheck.net/ warning SC2086, and BashPitfalls #14, explaining why even echo $foo is buggy.
Running read with the -r argument and IFS set to an empty value prevents it from consuming literal backslashes or pruning leading and trailing newlines.
Using two reads inside the condition of each while loop lets us read two lines at a time from each file (as is appropriate, given the intent to process content in pairs).
Bash operates more easly on "streams", not like, on the data itself.
first substitute every second newline starting from the first for a tabulation or a space or other separator
then "paste" the files together
Then rearange columns, from *name1 John *number1 25 to *name1 *number1 John 25
cat >name.txt <<EOF
*name1
John
*name2
Paul
EOF
cat <<EOF >number.txt
*number1
25
*number2
45
EOF
paste <(<name.txt sed 'N;s/\n/\t/') <(<number.txt sed 'N;s/\n/\t/') |
awk '{print $1,$3,$2,$4}'
will output:
*name1 *number1 John 25
*name2 *number2 Paul 45
First, in your example you overwrite the variable $number. So you have issues on reading file $number beginning from the second loop-run.
Solution with paste
Command paste can combine multiple files, and with option -d line-by-line.
#!/usr/bin/env bash
name=/home/davide/name.txt
number=/home/davide/number.txt
# combine both files linb-by-line
paste $'-d\n' "$name" "$number" |
while read nam
do
#after reading name to var 'nam', read number to var 'num':
read num
# print both
echo "$nam $num"
done
if you want TABS or any other separator and no other processing, you don't need the while loop. Examples
paste "$name" "$number"
paste -d: "$name" "$number"
paste -d\| "$name" "$number"
$ cat tst.awk
NR==FNR {
if ( NR%2 ) {
tags[++numPairs] = $0
}
else {
vals[numPairs] = $0
}
next
}
!(NR%2) {
for (pairNr=1; pairNr<=numPairs; pairNr++) {
print prev, tags[pairNr], $0, vals[pairNr]
}
}
{ prev = $0 }
$ awk -f tst.awk number.txt name.txt
*name1 *number1 John 25
*name1 *number2 John 45
*name2 *number1 Paul 25
*name2 *number2 Paul 45
In your script, you use the variable name for both the file path and the while-loop variable. This causes the "ambiguous redirect" error. Two lines need fix e.g.:
name_file=/home/davide/name.txt
done < $name_file
No need to for seek(0) in shell scripts. Just process the file again, e.g:
while read line ; do
echo "== $line =="
done < /some/file
while read line ; do
echo "--> ${line:0:1}"
done < /some/file
This is less efficient and less flexible than a more real programming language where you can seek(). But that's about differences, advantages and disadvantages between shell scripting and programming.
By the way, this line:
n=$(echo $name)
... is merely a awkward way of just doing:
n=$name
This can cause your script to behave quite unpredictable when $name contains special character like *. And since $name is read from a text file, this not unlikely to happen. (thanks Charles Duffy for making this point)
I have over 100000 files.
for example, I mentioned 3 files below
bcbb79d8-1d4a-4fbb-b16c-4df86839773e.htseq.counts.gz
bcdc68db-c874-4097-9c46-b06e331caaf5.htseq.counts.gz
bd4b6975-90d9-43f8-aadc-344d04644822.htseq.counts.gz
I have a text file named key.txt with the following information.
File Name ID
bcbb79d8-1d4a-4fbb-b16c-4df86839773e.htseq.counts.gz TCCC-06-0210
bcdc68db-c874-4097-9c46-b06e331caaf5.htseq.counts.gz TCHA-27-2519
bd4b6975-90d9-43f8-aadc-344d04644822.htseq.counts.gz TCHU-76-4929
I want to take only those files that their name are in the key , move them to a new folder and change their name to the ID.
I guess a little more of a write up rather than a comment would be helpful. The approach to take is to read the filename (fname) and ID (id) from each line in key.txt and then validate that fname is a file and does exist, and then move the file in "$fname" to whatever "/path/to/move/to/$id" you need.
For example:
#!/bin/bash
## read each line into variables fname and id (handle non-POSIX eof)
while read -r fname id || [ -n "$fname" ]; do
## test that "$fname" is a file, and if so, move to destination
[ -f "$fname" ] && mv "$fname" "/path/to/move/to/$id"
done < key.txt
(note: a POSIX end-of-file (eof) is simply the final '\n' at the end of the last line. Some editors do not enforce it and it will cause your read to miss the final line of data unless you check that "$fname" was filled with data (is non-empty) -- the [ -n "$fname" ] added to the end of the white read -r ...)
You are feeding the loop with a redirection of key.txt. Each iteration of the while loop will read a new line from key.txt into the variables fname and id (word-splitting on the default Internal Field Separator (IFS). After the read and separation into fname and id, you simply verify $fname holds a valid filename (in the current working directory) and then mv the file where you want it.
You should execute the script in the directory containing the files, or append a relative or absolute filename to where they are located to "$fname".
Example
Here is a short example that may help clear things up:
The move_rename.sh script:
$ cat move_rename.sh
#!/bin/bash
## read each line into variables fname and id (handle non-POSIX eof)
while read -r fname id || [ -n "$fname" ]; do
## test that "$fname" is a file, and if so, move to destination
[ -f "$fname" ] && mv "$fname" "dest/$id.txt"
done < key.txt
The key.txt file:
$ cat key.txt
File Name ID
bcbb79d8-1d4a-4fbb-b16c-4df86839773e.htseq.counts.gz TCCC-06-0210
bcdc68db-c874-4097-9c46-b06e331caaf5.htseq.counts.gz TCHA-27-2519
bd4b6975-90d9-43f8-aadc-344d04644822.htseq.counts.gz TCHU-76-4929
File locations before script execution. (dest) is the directory to move to. (that is ls -one output not ls -L(lowercase), the ls -al is `L(lowercase))
$ ls -1
dest
bcbb79d8-1d4a-4fbb-b16c-4df86839773e.htseq.counts.gz
bcdc68db-c874-4097-9c46-b06e331caaf5.htseq.counts.gz
bd4b6975-90d9-43f8-aadc-344d04644822.htseq.counts.gz
key.txt
move_rename.sh
$ ls -al dest
total 16
drwxr-xr-x 2 david david 4096 Jan 17 20:05 .
drwxr-xr-x 16 david david 12288 Jan 17 20:05 ..
Execute the script
$ bash move_rename.sh
Working directory contents after execution
$ ls -1
dest
key.txt
move_rename.sh
Contents of dest after execution.
$ ls -al dest
total 8
drwxr-xr-x 2 david david 4096 Jan 17 20:00 .
drwxr-xr-x 3 david david 4096 Jan 17 20:00 ..
-rw-r--r-- 1 david david 0 Jan 17 19:59 TCCC-06-0210.txt
-rw-r--r-- 1 david david 0 Jan 17 19:59 TCHA-27-2519.txt
-rw-r--r-- 1 david david 0 Jan 17 19:59 TCHU-76-4929.txt
Running "ls -lrt" on my terminal I get a large list that looks something like this:
-rw-r--r-- 1 pratik staff 1849089 Jun 23 12:24 cam13-vid.webm
-rw-r--r-- 1 pratik staff 1850653 Jun 23 12:24 cam12-vid.webm
-rw-r--r-- 1 pratik staff 1839110 Jun 23 12:24 cam11-vid.webm
-rw-r--r-- 1 pratik staff 1848520 Jun 23 12:24 cam10-vid.webm
-rw-r--r-- 1 pratik staff 1839122 Jun 23 12:24 cam1-vid.webm
I have only shown part of it above as a sample.
I would like to rename all the files to have a number one less than current.
For example,
mv cam1-vid.webm cam0-vid.webm
mv cam2-vid.webm cam1-vid.webm
.....
....
mv cam 200-vid.webm cam199-vid.webm
How can this be done using a os x / linux bash script (perhaps using sed) ?
You can do this with plain bash:
for i in {1..200}
do
mv "cam${i}-vid.webm" "cam$((i-1))-vid.webm"
done
I would use find, split up the file names, to find the number, subtract one, and rename:
find . -name "cam*-vid.webm" -print0 | while read -d\$0 old_name
do
number=${old_name#cam} #Filter left to remove 'cam' prefix
number=${number%-vid.webm"} #Filter right to remove '-vid.webm' suffix
$((number -= 1))
new_name="cam${number}-vid.webm"
echo "mv \"$old_name\" \"$new_name\""
done | tee results
This will merely print out the commands (that is why I have echo). I'm piping it into a file named results. Once this command completes, look at results and make sure it does everything it should. Whenever there's an operation like this, there can be a nasty surprise. For example, if I rename cam02-vid.webm to cam01-vid.webm before I rename cam01-vid.webm, I am going to overwrite cam01-vid-webm.
Maybe a safer way is to explicitly give the file numbers I need:
for number in {1..200}
do
$((old_number = $number + 1))
echo mv "\"cam${old_number}-vid.webm\" \"cam${number}-vid.webm\""
done | tee results
Useful hint: If the result file looks good, you can actually just run it as a shell script:
$ bash results
Another possibility is to test to make sure the old file exist:
for number in {1..200}
do
$((old_number = $number + 1))
if [ -f "$cam${old_number}-vid.webm" ]
then
echo mv "\"cam${old_number}-vid.webm\" \"cam${number}-vid.webm\""
else
echo "ERROR: Can't find a file called 'cam${old_number}-vid.webm'"
fi
done | tee results
A perl solution.
First it traverses all input files (#ARGV) and filters those that are plain files and not links (grep), extracts the number (map) and sorts numerically in ascendant to avoid overwritting (sort). Later creates a new file decrementing the number and renames the original:
perl -e '
for (
sort { $a->[0] <=> $b->[0] }
map { m/(\d+)/; [$1, $_ ] }
grep { -f $_ && ! -l $_ }
#ARGV
) {
$n = --$_-> [0];
($newname = $_->[1]) =~ s/\A(?i)(cam)\d+(.*)\z/$1$n$2/;
print "Executing command ===> rename $_->[1], $newname\n";
rename $_->[1], $newname;
}' *
Assuming initial content of the directory as:
cam1-vid.webm
cam13-vid.webm
cam12-vid.webm
cam11-vid.webm
cam10-vid.webm
cam2-vid.webm
After running the command yields:
cam0-vid.webm
cam10-vid.webm
cam11-vid.webm
cam12-vid.webm
cam1-vid.webm
cam9-vid.webm
I'm trying to store the files listing into an array and then loop through the array again.
Below is what I get when I run ls -ls command from the console.
total 40
36 -rwxrwxr-x 1 amit amit 36720 2012-03-31 12:19 1.txt
4 -rwxrwxr-x 1 amit amit 1318 2012-03-31 14:49 2.txt
The following bash script I've written to store the above data into a bash array.
i=0
ls -ls | while read line
do
array[ $i ]="$line"
(( i++ ))
done
But when I echo $array, I get nothing!
FYI, I run the script this way: ./bashscript.sh
I'd use
files=(*)
And then if you need data about the file, such as size, use the stat command on each file.
Try with:
#! /bin/bash
i=0
while read line
do
array[ $i ]="$line"
(( i++ ))
done < <(ls -ls)
echo ${array[1]}
In your version, the while runs in a subshell, the environment variables you modify in the loop are not visible outside it.
(Do keep in mind that parsing the output of ls is generally not a good idea at all.)
Here's a variant that lets you use a regex pattern for initial filtering, change the regex to be get the filtering you desire.
files=($(find -E . -type f -regex "^.*$"))
for item in ${files[*]}
do
printf " %s\n" $item
done
This might work for you:
OIFS=$IFS; IFS=$'\n'; array=($(ls -ls)); IFS=$OIFS; echo "${array[1]}"
Running any shell command inside $(...) will help to store the output in a variable. So using that we can convert the files to array with IFS.
IFS=' ' read -r -a array <<< $(ls /path/to/dir)
You may be tempted to use (*) but what if a directory contains the * character? It's very difficult to handle special characters in filenames correctly.
You can use ls -ls. However, it fails to handle newline characters.
# Store la -ls as an array
readarray -t files <<< $(ls -ls)
for (( i=1; i<${#files[#]}; i++ ))
{
# Convert current line to an array
line=(${files[$i]})
# Get the filename, joining it together any spaces
fileName=${line[#]:9}
echo $fileName
}
If all you want is the file name, then just use ls:
for fileName in $(ls); do
echo $fileName
done
See this article or this this post for more information about some of the difficulties of dealing with special characters in file names.
My two cents
The asker wanted to parse output of ls -ls
Below is what I get when I run ls -ls command from the console.
total 40
36 -rwxrwxr-x 1 amit amit 36720 2012-03-31 12:19 1.txt
4 -rwxrwxr-x 1 amit amit 1318 2012-03-31 14:49 2.txt
But there are few answer addressing this parsing operation.
ls's output
Before trying to parse something, we have to ensure command output is consistant, stable and easy to parse as possible
In order to ensure output wont be altered by some alias you may prefer to specify full path of command: /bin/ls.
Avoid variations of output due to locales, prefix your command by LANG=C LC_ALL=C
Use --time-style command switch to use UNIX EPOCH more easier to parse time infos.
Use -b switch for holding special characters
So we will prefer
LANG=C LC_ALL=C /bin/ls -lsb --time-style='+%s.%N'
to just
ls -ls
Full bash sample
#!/bin/bash
declare -a bydate=() bysize=() byname=() details=()
declare -i cnt=0 vtotblk=0 totblk
{
read -r _ totblk # ignore 1st line
while read -r blk perm lnk usr grp sze date file;do
byname[cnt]="${file//\\ / }"
details[cnt]="$blk $perm $lnk $usr $grp $sze $date"
bysize[sze]+="$cnt "
bydate[${date/.}]+="$cnt "
cnt+=1 vtotblk+=blk
done
} < <(LANG=C LC_ALL=C /bin/ls -lsb --time-style='+%s.%N')
From there, you could easily sort by dates, sizes of names (sorted by ls command).
echo "Path '$PWD': Total: $vtotblk, sorted by dates"
for dte in ${!bydate[#]};do
printf -v msec %.3f .${dte: -9}
for idx in ${bydate[dte]};do
read -r blk perm lnk usr grp sze date <<<"${details[idx]}"
printf ' %11d %(%a %d %b %T)T%s %s\n' \
$sze "${date%.*}" ${msec#0} "${byname[idx]}"
done
done
echo "Path '$PWD': Total: $vtotblk, sorted by sizes"
for sze in ${!bysize[#]};do
for idx in ${bysize[sze]};do
read -r blk perm lnk usr grp sze date <<<"${details[idx]}"
printf -v msec %.3f .${date#*.}
printf ' %11d %(%a %d %b %T)T%s %s\n' \
$sze "${date%.*}" ${msec#0} "${byname[idx]}"
done
done
echo "Path '$PWD': Total: $vtotblk, sorted by names"
for((idx=0;idx<cnt;idx++));{
read -r blk perm lnk usr grp sze date <<<"${details[idx]}"
printf -v msec %.3f .${date#*.}
printf ' %11d %(%a %d %b %T)T%s %s\n' \
$sze "${date%.*}" ${msec#0} "${byname[idx]}"
}
( Accessory, you could check if total block printed by ls match total block by lines:
(( vtotblk == totblk )) ||
echo "WARN: Total blocks: $totblk != Block count: $vtotblk" >&2
Of course, this could be inserted before first echo "Path...;)
Here is an output sample. (Note: there is a filename with a newline)
Path '/tmp/so': Total: 16, sorted by dates
0 Sun 04 Sep 10:09:18.221 2.txt
247 Mon 05 Sep 09:11:50.322 Filename with\nsp\303\251cials characters
13 Mon 05 Sep 10:12:24.859 1.txt
1313 Mon 05 Sep 11:01:00.855 parseLs.00
1913 Thu 08 Sep 08:20:20.836 parseLs
Path '/tmp/so': Total: 16, sorted by sizes
0 Sun 04 Sep 10:09:18.221 2.txt
13 Mon 05 Sep 10:12:24.859 1.txt
247 Mon 05 Sep 09:11:50.322 Filename with\nsp\303\251cials characters
1313 Mon 05 Sep 11:01:00.855 parseLs.00
1913 Thu 08 Sep 08:20:20.836 parseLs
Path '/tmp/so': Total: 16, sorted by names
13 Mon 05 Sep 10:12:24.859 1.txt
0 Sun 04 Sep 10:09:18.221 2.txt
247 Mon 05 Sep 09:11:50.322 Filename with\nsp\303\251cials characters
1913 Thu 08 Sep 08:20:20.836 parseLs
1313 Mon 05 Sep 11:01:00.855 parseLs.00
And if you want to format characters (with care: there could be some issues, if you don't know who create content of path). But if folder is your, you could:
echo "Path '$PWD': Total: $vtotblk, sorted by dates, with special chars"
printf -v spaces '%*s' 37 ''
for dte in ${!bydate[#]};do
printf -v msec %.3f .${dte: -9}
for idx in ${bydate[dte]};do
read -r blk perm lnk usr grp sze date <<<"${details[idx]}"
printf ' %11d %(%a %d %b %T)T%s %b\n' $sze \
"${date%.*}" ${msec#0} "${byname[idx]//\\n/\\n$spaces}"
done
done
Could output:
Path '/tmp/so': Total: 16, sorted by dates, with special chars
0 Sun 04 Sep 10:09:18.221 2.txt
247 Mon 05 Sep 09:11:50.322 Filename with
spécials characters
13 Mon 05 Sep 10:12:24.859 1.txt
1313 Mon 05 Sep 11:01:00.855 parseLs.00
1913 Thu 08 Sep 08:20:20.836 parseLs
Isn't these 2 code lines, either using scandir or including the dir pull in the declaration line, supposed to work?
src_dir="/3T/data/MySQL";
# src_ray=scandir($src_dir);
declare -a src_ray ${src_dir/*.sql}
printf ( $src_ray );
In the conversation over at https://stackoverflow.com/a/9954738/11944425
the behavior can be wrapped into a convenience function which applies some action to entries of the directory as string values.
#!/bin/bash
iterfiles() {
i=0
while read filename
do
files[ $i ]="$filename"
(( i++ ))
done < <( ls -l )
for (( idx=0 ; idx<${#files[#]} ; idx++ ))
do
$# "${files[$idx]}" &
wait $!
done
}
where $# is the complete glob of arguments passed to the function! This lets the function have the utility to take an arbitrary command as a partial function of sorts to operate on the filename:
iterfiles head -n 1 | tee -a header_check.out
When a script needs to iterate over files, returning an array of them is not possible. The workaround is to define the array outside of the function scope (and possibly unset it later) — modifying it inside the function's scope. Then, after the function is called by a script, the array variable becomes available. For instance, the mutation on files demonstrates how this could be done.
declare -a files # or just `files= ` (nothing)
iterfiles() {
# ...
files=...
}
Extending the conversation above, #Jean-BaptistePoittevin pointed out a valuable detail.
#!/bin/bash
# Adding a section to unset certain variable names that
# may already be active in the shell.
unset i
unset files
unset omit
i=0
omit='^([\n]+)$'
while read file
do
files[ $i ]="$file"
(( i++ ))
done < <(ls -l | grep -Pov ${omit} )
Note: This can be tested using echo ${files[0]} or for entry in ${files[#]}; do ... ; done
Often times, the circumstance could require an absolute path in double quotes, where the file (or ancestor directories) have spaces or unusual characters in the name. find is one answer here. The simplest usage might look like the above one, except done < <(ls -l ... ) is replaced with:
done < <(find /path/to/directory ! -path /path/to/directory -type d)
Its convenient when you need absolute paths in double quotes as an iterable collection to use a recipe like the one below. When export is not used, the shell does not update the environment namespace to include it in the find subshell:
#!/bin/bash
export DIRECTORY="$PWD" # For example
declare -a files
i=0
while read filename; do
files[ $i ]="$filename"
done < <(find $DIRECTORY ! -path $DIRECTORY -type d)
for (( idx=0; idx<${#files[#]}; idx++ )); do
# Make a templated string for macro script generation
quoted_path="\"${files[$idx]}\""
if [[ "$(echo $quoted_path | grep some_substring | wc -c)" != "0" ]]; then
echo "mv $quoted_path /some/other/watched/folder/" >> run_nightly.sh
fi
done
Upon running this, ./run_nightly.sh will be populated with bulk commands to move a quoted path to /some/other/watched/folder/. This kind of scripting pattern will make it possible to supercharge your scripts.
simply you can use this below for loop (do not forget to quote to handle filenames with spaces)
declare -a arr
arr=()
for file in "*.txt"
do
arr=(${arr[*]} "$file")
done
Run
for file in ${arr[*]}
do
echo "<$file>"
done
to test.