I want to run a program multiple times with different parameters, and then put the results piped into files that use parameters in their names. Here is what I've come up with:
#!/bin/bash
for i in 'seq 1 5';
do
for j in 'seq 1 8';
do
for m in 'seq 1 8';
do
./program -s i -v j -k m ../input_files/input_file1.txt < results_ijm.txt
done
done
done
This doesn't work. It says "no file results_ijm.txt".... I know that - I want it to create this file implicitly.
Otherwise, I also doubt it will assign ijm in the filename correctly - how does it know whether I want the VARIABLES ijm.... or just the characters? It's ambiguous.
You must use variable $i, $j, $m etc.
Better to use ((...)) construct in BASH.
In BASH you can do:
#!/bin/bash
for ((i=1; i<=5; i++)); do
for ((j=1; h<=8; j++)); do
for ((m=1; m<=8; m++)); do
./program -s $i -v $j -k $m ../input_files/input_file1.txt > "results_${i}${j}${m}.txt"
done
done
done
Two problems. As I mentioned in the comments, your arrow is backwards. We want the results of the program to go from stdout to the file so flip that thing around. Second, variables when used gain a dollar sign in front of them... so it won't be ambiguous.
Edited to add: Third thing, use backticks instead of single quotes for seq 1 5 You want the results of that command, not the text "seq 1 5". Thanks #PSkocik
#!/bin/bash
for i in `seq 1 5`;
do
for j in `seq 1 8`;
do
for m in `seq 1 8`;
do
./program -s $i -v $j -k $m ../input_files/input_file1.txt > results_${i}${j}${m}.txt
done
done
done
./program -s i -v j -k m ../input_files/input_file1.txt < results_ijm.txt
I believe the less than symbol should be flipped to greater than, so as to input to file, instead of from file. I've not worked too much with bash, but it seems logical.
Related
I have a list of files stored in a variable obtained by entering
files="./*.fasta"
I would like to create a for loop that will loop through: the first 200 elements, elements 201-400, elements 401-578, for example.
How can I achieve this? I tried something like
for file in $files[1-200]; do
echo $file
done
but clearly this does not work.
Using a variable to populate a list of files is not recommended. The best way to do it would be using arrays!
You need to enable a shell option to avoid null glob expansion by doing shopt -s nullglob so that if no files are found the for-loop exits gracefully. The example below shows iterating over the 200 files at a time. You could change the indices as needed to print from 200-400 and 400-600 as needed in the for-loop.
shopt -s nullglob
files=(*.fasta)
if (( "${#files}" >= 200 )); then
for ((i=0; i<200; i++)); do
printf '%s\n' "${files[i]}"
done
fi
Put them in an array, then use substring expansion to get batches of files.
files=(./*.fasta)
for ((i=0; i< ${#fasta[*]}; i+=200)); do
process "${files[#]:i:200}" &
done
The problem may be approach differently. Instead of using a for loop you may use find and xargs:
find * -name '*.fasta' -maxdepth 0 -print0 | xargs -0 -n 200 -P 0 echo
find passes every file name to xargs which in turn spawns a process (-P 0) for every 200 input files (-n 200).
This one-liner uses -print0 and -0 flags just in case your filenames contain whitespace.
The for loop construct is less than ideal in this scenario.
Alternatively, you might use a while loop and a readarray builtin:
find * -name '*.fasta' -maxdepth 0 | while readarray -n 3 a && [[ ${#a} -ne 0 ]]
do
echo ${a[#]}
done
I have a set of scripts, e.g.
01_some_stuff1
02_some_stuff2
03_some_stuff3a
03_some_stuff3b
04_some_stuff4a
04_some_stuff4b
These scripts should run ordered by their number and scripts with the same number should run in parallel.
My first idea was to iterate the possible numbers
for n in $(seq -f %02.0f 0 99); do
for s in "${n}_*"; do
export CURRENT_JOB="${s}"
"${s}" &
done
wait
done
Is this a safe method? Is there a more elegant solution that also allows to set a different environment for the inner loop elements?
You could use GNU Parallel like this:
#!/bin/bash
# Don't barf if no matching files when globbing
shopt -s nullglob
for n in $(printf "%02d " {1..4}); do
# Get list (array) of matching scripts
scripts=( ${n}_* )
if [ ${#scripts[#]} -gt 0 ]; then
parallel --dry-run -k 'CURRENT_JOB={} ./{}' ::: ${scripts[#]}
fi
echo barrier
done
Sample Output
CURRENT_JOB=01_some_stuff1 ./01_some_stuff1
barrier
CURRENT_JOB=02_some_stuff2 ./02_some_stuff2
barrier
CURRENT_JOB=03_some_stuff3a ./03_some_stuff3a
CURRENT_JOB=03_some_stuff3b ./03_some_stuff3b
CURRENT_JOB=03_some_stuff3c ./03_some_stuff3c
barrier
CURRENT_JOB=04_some_stuff4a ./04_some_stuff4a
CURRENT_JOB=04_some_stuff4b ./04_some_stuff4b
barrier
Remove the echo barrier and --dry-run to actually run it properly.
The only real change you need is to avoid quoting the * in your pattern. If you are using bash 4.0 or later, you can use brace expansion to eliminate the dependency on seq.
# for n in $(seq -f %02.0f 0 99); do
for n in {00..99}; do
for s in "${n}"_*; do
export CURRENT_JOB="${s}"
"${s}" &
done
wait
done
I want to run certain actions on a group of lexicographically named files (01-09 before 10). I have to use a rather old version of FreeBSD (7.3), so I can't use yummies like echo {01..30} or seq -w 1 30.
The only working solution I found is printf "%02d " {1..30}. However, I can't figure out why can't I use $1 and $2 instead of 1 and 30. When I run my script (bash ~/myscript.sh 1 30) printf says {1..30}: invalid number
AFAIK, variables in bash are typeless, so how can't printf accept an integer argument as an integer?
Bash supports C-style for loops:
s=1
e=30
for i in ((i=s; i<e; i++)); do printf "%02d " "$i"; done
The syntax you attempted doesn't work because brace expansion happens before parameter expansion, so when the shell tries to expand {$1..$2}, it's still literally {$1..$2}, not {1..30}.
The answer given by #Kent works because eval goes back to the beginning of the parsing process. I tend to suggest avoiding making habitual use of it, as eval can introduce hard-to-recognize bugs -- if your command were whitelisted to be run by sudo and $1 were, say, '$(rm -rf /; echo 1)', the C-style-for-loop example would safely fail, and the eval example... not so much.
Granted, 95% of the scripts you write may not be accessible to folks executing privilege escalation attacks, but the remaining 5% can really ruin one's day; following good practices 100% of the time avoids being in sloppy habits.
Thus, if one really wants to pass a range of numbers to a single command, the safe thing is to collect them in an array:
a=( )
for i in ((i=s; i<e; i++)); do a+=( "$i" ); done
printf "%02d " "${a[#]}"
I guess you are looking for this trick:
#!/bin/bash
s=1
e=30
printf "%02d " $(eval echo {$s..$e})
Ok, I finally got it!
#!/bin/bash
#BSD-only iteration method
#for day in `jot $1 $2`
for ((day=$1; day<$2; day++))
do
echo $(printf %02d $day)
done
I initially wanted to use the cycle iterator as a "day" in file names, but now I see that in my exact case it's easier to iterate through normal numbers (1,2,3 etc.) and process them into lexicographical ones inside the loop. While using jot, remember that $1 is the numbers amount, and the $2 is the starting point.
Looking for en nicer solution for the next code:
for i in a b c
do
for j in A B C
do
for k in 1 2 3
do
echo "$i$j$k"
done
done
done
Sure is here some simpler solution.
This is a bit simpler
echo {a,b,c}{A,B,C}{1,2,3}
or if want one per line, so
echo {a,b,c}{A,B,C}{1,2,3} | xargs -n1
BTW, you can use the above bracer expansion for example saving keyboard typing for example when need make backup files, like:
cp /some/long/path/And_very-ugly-fileName{,.copy}
will make the /some/long/path/And_very-ugly-fileName.copy without second filename typing.
Use a brace expansion:
echo {a,b,c}{A,B,C}{1,2,3}
This will print all possible combinations between the sets {a,b,c}, {A,B,C}, and {1,2,3}.
Or even simpler, using ranges:
echo {a..c}{A..C}{1..3}
And if you want to print each combination per line, use a for-loop:
for i in {a..c}{A..C}{1..3}; do echo $i; done
I wrote a .sh file to compile and run a few programs for a homework assignment. I have a "for" loop in the script, but it won't work unless I use only integers:
#!/bin/bash
for (( i=10; i<=100000; i+=100))
do
./hw3_2_2 $i
done
The variable $i is an input for the program hw3_2_2, and I have non-integer values I'd like to use. How could I loop through running the code with a list of decimal numbers?
I find it surprising that in five years no one ever mentioned the utility created just for generating ranges, but, then again, it comes from BSD around 2005, and perhaps it wasn't even generally available on Linux at the time the question was made.
But here it is:
for i in $(seq 0 0.1 1)
Or, to print all numbers with the same width (by prepending or appending zeroes), use -w. That helps prevent numbers being sent as "integers", if that would cause issues.
The syntax is seq [first [incr]] last, with first defaulting to 1, and incr defaulting to either 1 or -1, depending on whether last is greater than or less than first. For other parameters, see seq(1).
you can use awk to generate your decimals eg steps of0.1
num=$(awk 'BEGIN{for(i=1;i<=10;i+=0.1)print i}')
for n in $num
do
./hw3_2_2 $n
done
or you can do it entirely in awk
awk 'BEGIN{cmd="hw3_2_2";for(i=1;i<=10;i+=0.1){c=cmd" "i;system(cmd) } }'
The easiest way is to just list them:
for a in 1.2 3.4 3.11 402.12 4.2 2342.40
do
./hw3_2_2 $a
done
If the list is huge, so you can't have it as a literal list, consider dumping it in a file and then using something like
for a in $(< my-numbers.txt)
do
./hw3_2_2 $a
done
The $(< my-numbers.txt) part is an efficient way (in Bash) to substitute the contents of the names file in that location of the script. Thanks to Dennis Williamson for pointing out that there is no need to use the external cat command for this.
Here's another way. You can use a here doc to include your data in the script:
read -r -d '' data <<EOF
1.1
2.12
3.14159
4
5.05
EOF
for i in "$data"
do
./hw3_2_2 "$i"
done
Similarly:
array=(
1.1
2.12
3.14159
4
5.05
)
for i in "${array[#]}"
do
./hw3_2_2 "$i"
done
I usually also use "seq" as per the second answer, but just to give an answer in terms of a precision-robust integer loop and then bc conversion to a float:
#!/bin/bash
for i in {2..10..2} ; do
x=`echo "scale=2 ; ${i}/10" | bc`
echo $x
done
gives:
.2
.4
.6
.8
1.0
bash doesn't do decimal numbers. Either use something like bc that can, or move to a more complete programming language. Beware of accuracy problems though.