I have N files in my directory, I want to be able to loop through them with batches, for example for 2/3/4 files I want to execute one command. I saw something like this, but can't find it, unfortunately. I will try to explain what I want using pseudo code, for example:
i=0
while i<N:
a=getFile(i)
b=getFile(i+1)
dosomething a
dosomething b
i+=2
Is there any way to do it in Bash? Right now I'm using regexp, but it gets one file at a time (I'm using *.ext because all files have one extension, so you can just loop through all files in the directory in your answer, if it's easier):
for j in *.ext; do
...
done
This would be the same with most programming languages: loop and store items somewhere until you have enough of them:
declare -i count=0
declare -a files=()
for f in *.ext; do
files[count]="$f"
(( count += 1 ))
if (( count == 2 )); then
dosomething "${files[0]}"
dosomething "${files[1]}"
count=0
fi
done
If you want to process your files in batches of more than 2 at a time we can also design something a bit more generic (adapt the value of batch):
declare -i batch=3
declare -i count=0
declare -a files=()
for f in *.ext; do
files[count]="$f"
(( count += 1 ))
if (( count == batch )); then
for (( i=0; i<batch; i++ )); do
dosomething "${files[i]}"
done
count=0
fi
done
You can collect your arguments into an array, and then slice that any way you like.
array=( *.ext )
for ((i=0; i<=${#array[#]}; i+=2)); do
dosomething "${array[i]}" "${array[i+1]}"
done
Related
I'm making a simple script that iterates through the files in the current directories,
the $1 is for a size parameter and the others $1,$2 ..... are for the manipulated files and set for the file names.
the problem is after using the for loop the variable are losing their value and start with integers like 1,2,3, and the script doesn't work unless I use files named as 1,2,3,....
How can I keep the original values?
ex:
./script 50 my_first_file .....
#!/bin/bash
size=$1
allfiles=$#
shift
#here the value of the $1 is "my_first_file"
for ((i = 1 ; i < allfiles ; i++))
do
#here the value of the $1 = 1
done
Instead of using a for loop with integers, you can loop on arguments directly like this:
#!/bin/bash
size="$1"
allfiles=$#
shift
counter=1
for i in "$#"
do
echo "$counter= $i"
(( counter = counter + 1 ))
done
echo "size= $size"
This will show you each argument in sequence.
If you need to show or use the position of each argument, you can use a counter.
If I call this: script.bash 25 a b c
The output is:
1= a
2= b
3= c
size= 25
Another option is just use a short hand of looping through the files.
#!/usr/bin/env bash
size=$1
shift
counter=1
for f; do
printf '%d. %s\n' "$((counter++))" "$f"
done
printf 'size=%s\n' "$size"
I have 100 subdirectories and I wanted to loop through the first ten of them in a bash for loop such like that:
for d in ./output/*[0..9];
do
echo $d
done
But the output seems not what I expected:
./output/405050
./output/405140
./output/405309
./output/405310
./output/405319
./output/500550
./output/500589
./output/500610
Why only 8 were printed and my question is how to select a fix number elements from this type of for loop.
*[0..9] loops over ones that end in a 0, 9, or .. If you had written *{0..9} that would loop over ones ending in a digit 0 through 9--closer, but still not right.
Try this loop, which reads the first 10 directory names in a loop. It's kinda obtuse. The primary idea is using while read ... < <(cmd) to read a command's output one line at a time. IFS= and -r are pedantic bits to handle directory names with whitespace and backslashes correctly.
while IFS= read -r dir; do
echo "$dir"
done < <(ls output/*/ | head -10)
Or use this more straightforward version with a counter:
i=0
for dir in output/*/; do
echo "$dir"
((++i < 10)) || break
done
Or this one storing the directories in an array:
dirs=(output/*/)
for dir in "${dirs[#]::10}"; do
echo "$dir"
done
You can make a counter:
#!/bin/bash
i=0;
for d in ./output/*/;
do
echo $d
echo ""
if [[ i == 10 ]]; then
break
fi
i+=1
done
With this you asure to get 10 folders.
I very important to do the last backslash to match only directories.
In need to process a bunch of files in bash and split up the process based on the number of enumerations. I'd like to process first 1000 files and then another 1000 files and stop if the limit of 2000 files is exceeded. In python I would do something like this:
for e,any_file in enumerate("some_files"):
if e<1000: # for the first 1000 files
print(any_file) # or do anything you want
else:
if e<2000: # for the second 1000 files
print(any_file)
else:
print("exceeded maximum limit!")
is there anything similar for the bash?
You can use a counter:
e=1 ## Counter
for f in /path/*; do
if (( e < 1000 )); then
echo "$f"
else
if (( e < 2000 )); then
echo "$f"
else
echo "exceeded maximum limit!"
fi
fi
(( ++e ))
done
Or an array:
files=(0 /path/*); unset 'files[0]'
for e in "${!files[#]}"; do
f=${files[e]}
...
done
You can also create an array from an enumerator's output like find with process substitution and readarray:
readarray -O 1 -t files < <(exec find ...)
Refer to the Bash manual (man bash) for everything. See help and help <command> as well.
I have several .jpg images in a folder that have names like:
20140331_134927.jpg
20140331_124933.jpg
20140331_124933.jpg
etc..
I want to rename them to something like:
Agra-1.jpg
Agra-2.jpg
Agra-3.jpg
etc..
I tried running the following script (stored as my.sh):
for files in *.jpg; do
i=1
echo mv "$files" "Agra-$i.jpg"
i=$((i+1))
done
However, if I were to run that without the echo, all files would be renamed to "Agra-1.jpg"
Why does this not work as I expect and how should this be written?
Put the assignment out of the loop:
i=1 # only once
for files in *.jpg; do
mv "$files" "Agra-$i.jpg"
let i++
done
Here is an example - you should declare the counter variable outside of the loop otherwise it will be reset to its initial value on each iteration:
Inside loop:
$ for file in *; do i=1; echo $i; (( i++ )); done
1
1
1
Outside loop:
$ i=1
$ for file in *; do echo $i; (( i++ )); done
1
2
3
I have 32 files (named by the same pattern, the only difference is the $sample number as written below) that I want to divide into 4 folders. I am trying to use the following script to do this job, but the script is not working, can someone help me with the following shell script please? - Thanks
#!/bin/bash
max=8 #8 files in each sub folder
numberFolder=4
sample=0
while ($numberFolder > 1) #skip the current folder, as 8 files will remain
do
for (i=1; i<9; i++)
do
$sample= $i * $numberFolder # this distinguish one sample file from another
echo "tophat_"$sample"_ACTTGA_L003_R1_001" //just an echo test, if works, will replace it with "cp".
done
$numberFolder--
end
You need to use math contexts -- (( )) -- correctly.
#!/bin/bash
max=8
numberFolder=4
sample=0
while (( numberFolder > 1 )); do # math operations need to be in a math context
for ((i=1; i<9; i++)); do # two (( )), not ( ).
(( sample = i * numberFolder ))
echo "tophat_${sample}_ACTTGA_L003_R1_001" # don't unquote before the expansion
done
(( numberFolder-- )) # math operations need to be inside a math context
done