Bash read multiple files in for loop (as a execute arguement) - bash

I am trying to read a file in bash forloop. But I do not know how to put write the script for this.
for i in $( seq 0 $step 10 )
do
echo "Rendering: "$(( i + j ))
python auto_fine.py density000000.vtu velocity000000.vtu $(( i + j ))
done
each and every loop I need to call
i -> 0 python auto_fine.py density000000.vtu velocity000000.vtu
i -> 1 python auto_fine.py density000010.vtu velocity000010.vtu
i -> 2 python auto_fine.py density000020.vtu velocity000020.vtu

It seems to me that you need to zero pad the numbers sed provides to you:
As seen in How to zero pad a sequence of integers in bash so that all have the same width?, you need to do something like
$ seq -f "%06g" 0 10 100
Which returns:
000000
000010
000020
...
000100
All together,
for i in $(seq -f "%06g" 0 10 100)
do
# echo "Rendering: "$(( i + j )) -- not sure what this does
python auto_fine.py density$i.vtu velocity$i.vtu
done

Bash can do this without requiring external tools like seq.
for i in {0..100}; do
[[ $i = *0 ]] || continue
python auto_fine.py density$(printf '%06d' $i).vtu velocity$(printf '%06d' $i).vtu
done
This uses pattern matching (*0) to limit your list to every 10 numbers, which is a bit of a hack, but will work against your sample data.
You could alternately loop against your zero-padded numeric strings directly:
for i in $(printf '%05d0 ' {0..10}); do
python auto_fine.py density$i.vtu velocity$i.vtu
done
This option shows you every 10 items by placing a zero in the printf format after the incrementing number, which becomes the tens digit. If you want more arbitrary sequencing, you might use multipliers, still without spawning external processes:
low=0
high=100
mult=10
for i in $(eval echo {$low..$((high/mult))}); do
n=$(printf '%06d' $((i*mult)))
python auto_fine.py density$n.vtu velocity$n.vtu
done
Note the eval, which lets you expand variables for use in your sequence expression. (If you are getting these numbers from an external source, have your script validate them before using them!)
If you're using bash version 4 (i.e. not the native version on OSX), you also have increments available in sequence expressions. From the man page:
A sequence expression takes the form {x..y[..incr]}, where x and y are
either integers or single characters, and incr, an optional increment,
is an integer.
So perhaps:
low=0
high=100
mult=10
for i in $(eval "printf '%06d ' {$low..$high..$mult}"); do
python auto_fine.py density$i.vtu velocity$i.vtu
done
Note that in sequence expressions, the first member of the sequence is the first number provided, rather than merely a product of a multiplier. We have quotes around the printf to ensure that the sequence expression is expanded by eval, and not interpreted by the command substitution ($(..)).

looping for all the files in the current dir is trivial:
for i in $( ls -1 )
do
# your code here, variable is referenced with $i
done
what's the j variable you are using?

Related

Addin 0 in front of one digit number/text in bash script

I would like to compare a number, which is the output of a command, with a constant and do some manipulation. That is, if $id < 10, I want to see 590$id and if it is above 10, I want to see 59$id.
I found that expr doesn't working here:
ID=3
NUM=59$ID
if [ `expr $ID` -lt 10]; then
NUM=590$ID
fi
echo $NUM
The output of the code is 593 and not 5903. Even, $(($ID + 5900)) -lt 5910 writes 593.
How can I fix that?
Could you please try following.
cat script.sh
#!/bin/bash
ID=$(printf "%02d" 3 )
##NUM=59$ID ##Commented this to check if, condition is getting satisfied or not. Doesn't seem to be fit here.
(( $ID < 10 )) && NUM="59$ID"
echo "$NUM"
Output will be 5903 after running above code.
Don't use expr. It's old and tricky.
Don't use backticks `. They are discouraged and $( ... ) is preferred.
For arithmetic comparisons use arithmetic expansions. Just
if (( ID < 10 )); then
Note that bash is space aware and your script has a syntax error, it is missing a space - the 10]; should be 10 ];.
Note that by convention uppercase variables should be used for exported variables.
Looking at your code I think you just want:
NUM=$((5900 + ID))

Taking a count from file, I want to print no of variables using shell/bash

Taking count from file, say if count = 5, I want to print 5 variables. i.e. A B C D E.
If count = 2, Print 2 variables A B, etc.
I have tried using the ASCII values but couldn't go through it.
for i in {1..5}; do
count=5; a=0;
printf "\x$(printf %x '65+$a')";
count=count+1;
done
if count = 5, I want to print 5 variables. i.e. A B C D E. If count = 2, Print 2 variables A B, etc.
Here's a program that matches your style that does what you are looking for:
a=0
for i in {1..5}; do
printf "\x$(printf %x $(( 65 + a )) )";
a=$((a+1));
done
The first thing to note is that in order to do math in bash, you'll need to use the $(( )) operation. Above, you can see I replaced you '65+$a' with $(( 65 + a )) . That's the big news that you need to get math done.
There were a couple of other little issues, but you were stuck on the $(()) stuff so they weren't clear yet. Incidentally, the 'a' variable can be completely removed from the program to just use the 'i' variable like this:
for i in {1..5}; do
printf "\x$(printf %x $(( 64 + i )) )";
done
I had to change the constant to 64 since we are now counting starting at 1.
The {1..5} expression is a good short cut for 1 2 3 4 5, but you won't be able to put a variable into it. So, if you need to add a count variable back in, consider using the seq program instead like this:
count=$1
for i in $(seq 1 $count); do
printf "\x$(printf %x $(( 64 + i )) )";
done
Note that $() is different than the math operator $(()). $() runs a subcommand returning the results.
method 1: simple brace expansion
#!/bin/bash
# generate a lookup table
vars=( - $(echo {A..Z}) )
# use the elements
for i in {1..5}; do
echo ${vars[$i]}
done
{A..Z} generates 26 strings: A, B, ..., Z
which get stored in an array variable by vars=(...)
we prepend a - that we'll ignore
we can then do 1-based indexing into the array
limited to 26 variables (or whatever range we choose)
method 2: multiple brace expansion to generate arbitrary long variables
#!/bin/bash
if [[ ! $1 =~ ^[0-9]+$ ]]; then
echo "Usage: $0 count"
exit
fi
cmd='{A..Z}'
for (( i=$1; i>26; i=i/26 )); do
cmd="${A..Z}$cmd"
done
vars=( $(eval echo $cmd) )
for (( i=0; i<$1; i++ )); do
echo ${vars[$i]}
done
i/26 does integer division (throws away the remainder)
I'm lazy and generate "more than enough" variables rather than attempting to calculate how many is "exactly enough"
{a..b}{a..b}{a..b} becomes aaa aab aba abb baa bab bba bbb
using eval lets us do the brace expansion without knowing in advance how many sets are needed
Sample output:
$ mkvar.sh 10000 |fmt -64 | tail -5
ORY ORZ OSA OSB OSC OSD OSE OSF OSG OSH OSI OSJ OSK OSL OSM
OSN OSO OSP OSQ OSR OSS OST OSU OSV OSW OSX OSY OSZ OTA OTB
OTC OTD OTE OTF OTG OTH OTI OTJ OTK OTL OTM OTN OTO OTP OTQ
OTR OTS OTT OTU OTV OTW OTX OTY OTZ OUA OUB OUC OUD OUE OUF
OUG OUH OUI OUJ OUK OUL OUM OUN OUO OUP

How To Split Up Digits Into Character Array

I'm a bit stuck with something. I have a for loop like this:
#!/bin/bash
for i in {10..15}
do
I want to obtain the last digit of the number, so if i is 12, I want to get 2. I'm having difficulties with the syntax though. I've read that I should convert it into a character array, but when I do something like:
j=${i[#]}
echo $j
I don't get 1 0 1 1 1 2 and so on...I get 10, 11, 12...How do I get the numbers to be split up so I can get the last one of i, when I don't always know how many digits will make up i (ex. it may be 1, or 10, or a 100, etc.)?
Trick is to treat $i like a string.
for i in {10..15}; do j="${i: -1}"; echo $j; done
Of course, you do not need to assign to a variable if you don't want to:
for i in {10..15}; do echo "${i: -1}"; done
This answer which uses GNU shell parameter expansion is the most sensible method, I guess.
However, you can also use the double parenthesis construct which allows C-style manipulation of variables in Bash.
for i in {10..15}
do
(( j = i % 10 )) # modulo 10 always gives the ones' digit
echo $j
done
This awk command could solve your problem:
awk '{print substr($0,length,1)}' test_file
I'm assuming that the numbers are saved in a file test_file
If you want to use for loop:
for i in `cat test_1`
do
echo $i |tail -c 2
done

Is there a way to implement a counter in bash but for letters instead of numbers?

I'm working with an existing script which was written a bit messily. Setting up a loop with all of the spaghetti code could make a bigger headache than I want to deal with in the near term. Maybe when I have more time I can clean it up but for now, I'm just looking for a simple fix.
The script deals with virtual disks on a xen server. It reads multipath output and asks if particular LUNs should be formatted in any way based on specific criteria. However, rather than taking that disk path and inserting it, already formatted, into a configuration file, it simply presents every line in the format
'phy:/dev/mapper/UUID,xvd?,w',
UUID, of course, is an actual UUID.
The script actually presents each of the found LUNs in this format expecting the user to copy and paste them into the config file replacing each ? with a letter in sequence. This is tedious at best.
There are several ways to increment a number in bash. Among others:
var=$((var+1))
((var+=1))
((var++))
Is there a way to do the same with characters which doesn't involve looping over the entire alphabet such that I could easily "increment" the disk assignment from xvda to xvdb, etc?
To do an "increment" on a letter, define the function:
incr() { LC_CTYPE=C printf "\\$(printf '%03o' "$(($(printf '%d' "'$1")+1))")"; }
Now, observe:
$ echo $(incr a)
b
$ echo $(incr b)
c
$ echo $(incr c)
d
Because, this increments up through ASCII, incr z becomes {.
How it works
The first step is to convert a letter to its ASCII numeric value. For example, a is 97:
$ printf '%d' "'a"
97
The next step is to increment that:
$ echo "$((97+1))"
98
Or:
$ echo "$(($(printf '%d' "'a")+1))"
98
The last step is convert the new incremented number back to a letter:
$ LC_CTYPE=C printf "\\$(printf '%03o' "98")"
b
Or:
$ LC_CTYPE=C printf "\\$(printf '%03o' "$(($(printf '%d' "'a")+1))")"
b
Alternative
With bash, we can define an associative array to hold the next character:
$ declare -A Incr; last=a; for next in {b..z}; do Incr[$last]=$next; last=$next; done; Incr[z]=a
Or, if you prefer code spread out over multiple lines:
declare -A Incr
last=a
for next in {b..z}
do
Incr[$last]=$next
last=$next
done
Incr[z]=a
With this array, characters can be incremented via:
$ echo "${Incr[a]}"
b
$ echo "${Incr[b]}"
c
$ echo "${Incr[c]}"
d
In this version, the increment of z loops back to a:
$ echo "${Incr[z]}"
a
How about an array with entries A-Z assigned to indexes 1-26?
IFS=':' read -r -a alpharray <<< ":A:B:C:D:E:F:G:H:I:J:K:L:M:N:O:P:Q:R:S:T:U:V:W:X:Y:Z"
This has 1=A, 2=B, etc. If you want 0=A, 1=B, and so on, remove the first colon.
IFS=':' read -r -a alpharray <<< "A:B:C:D:E:F:G:H:I:J:K:L:M:N:O:P:Q:R:S:T:U:V:W:X:Y:Z"
Then later, where you actually need the letter;
var=$((var+1))
'phy:/dev/mapper/UUID,xvd${alpharray[$var]},w',
The only problem is that if you end up running past 26 letters, you'll start getting blanks returned from the array.
Use a Bash 4 Range
You can use a Bash 4 feature that lets you specify a range within a sequence expression. For example:
for letter in {a..z}; do
echo "phy:/dev/mapper/UUID,xvd${letter},w"
done
See also Ranges in the Bash Wiki.
Here's a function that will return the next letter in the range a-z. An input of 'z' returns 'a'.
nextl(){
((num=(36#$(printf '%c' $1)-9) % 26+97));
printf '%b\n' '\x'$(printf "%x" $num);
}
It treats the first letter of the input as a base 36 integer, subtracts 9, and returns the character whose ordinal number is 'a' plus that value mod 26.
Use Jot
While the Bash range option uses built-ins, you can also use a utility like the BSD jot utility. This is available on macOS by default, but your mileage may vary on Linux systems. For example, you'll need to install athena-jot on Debian.
More Loops
One trick here is to pre-populate a Bash array and then use an index variable to grab your desired output from the array. For example:
letters=( "" $(jot -w %c 26 a) )
for idx in 1 26; do
echo ${letters[$idx]}
done
A Loop-Free Alternative
Note that you don't have to increment the counter in a loop. You can do it other ways, too. Consider the following, which will increment any letter passed to the function without having to prepopulate an array:
increment_var () {
local new_var=$(jot -nw %c 2 "$1" | tail -1)
if [[ "$new_var" == "{" ]]; then
echo "Error: You can't increment past 'z'" >&2
exit 1
fi
echo -n "$new_var"
}
var="c"
var=$(increment_var "$var")
echo "$var"
This is probably closer to what the OP wants, but it certainly seems more complex and less elegant than the original loop recommended elsewhere. However, your mileage may vary, and it's good to have options!

bash create array of ids for launching processes

I have to come up with an array of values. They're going to represent the server and/or virtual server ids they'll represent. So I have to run 100 application instances on 25 servers.
server 1 runs process 1,26,51,76
server 2 runs process 2,27,52,77
...
server 25 runs process 25,50,75,100
I need a way to execute the scripts like this
# hostname is in a format like production-01.localdomain
host_id=`hostname | tr 'A-Za-z-.' ' ' | tr -d '[[:space:]]'`
# need to create array of server ids in server_ids
for server_id in server_ids
do
/usr/local/bin/virtual_process $server_id
done
but I don't know how to create the server_ids array. Help!
here ya go. This solves your problem by creating a space separated string instead of an array which would be overkill
# hostname is in a format like production-01.localdomain
server_id=`hostname | tr 'A-Za-z-.' ' ' | tr -d '[[:space:]]' | awk '{print NR}'`
# need to create array of server ids in server_ids
server_ids="$server_id $(( $server_id+25 )) $(( $server_id+50 )) $(( $server_id+75 ))"
for server_id in $server_ids
do
/usr/local/bin/virtual_process $server_id
done
Here's an idiomatic Bash solution that should work as-is:
#!/usr/bin/env bash
# Extract the base server ID by extracting the number from the hostname,
# eliminating any leading zeros.
# E.g., 'production-01.localdomain' -> '1'
baseServerId=$(( 10#$(tr -dC '[0-9]' <<<"$HOSTNAME") ))
# Loop 4 times, starting with the base server ID and
# adding 25 in each subsequent iteration.
for (( i = 0; i < 4; ++i )); do
/usr/local/bin/virtual_process "$(( baseServerId + i * 25 ))"
done
As you can see,
A single tr command utilizing -C for the complement is enough to extract digits only: -dC '[0-9]' means: delete (-d) everything but (-C) digits ([0-9]).
Since the resulting number can have leading zeros, these must be removed to avoid misinterpretation as an octal number. Evaluating the tr command inside an arithmetic expansion $(( ... )) with 10# - the desired number base - prepended to the output from tr, does just that.
No need for an array of server IDs when an arithmetic, C-style loop will do.
That said, since the list of offsets is so small and has fixed increments, a more succinct solution is:
for offset in 0 25 50 75; do
/usr/local/bin/virtual_process "$(( baseServerId + offset ))"
done
A more algorithmic approach using brace expansion (which, it should be noted, only supports literal arguments; Bash v4+ is required due to using a 3rd argument to specify a step value):
for offset in {0..75..25}; do
/usr/local/bin/virtual_process "$(( baseServerId + offset ))"
done
If you just need a sequence of numbers from 1 to 25 you can use bash's brace expansion.
for server_id in {1..25}
do
/usr/loca/bin/virtual_process $server_id
done

Resources