Say I want to iterate over two lists of letters and numbers.
A B C D and seq 1 100.
How can I iterate over letters along with numbers but not as in nested for-loop? So it would be A1B2C3D4 A5B6C7D8 ...
What I've tried so far: nested for-loop and & done don't seem to be of any help, since they produce either A1 B1 C1 D1 A2 B2... or inconsistent results of parallel execution.
Also it feels like a very basic parallel loop, so no need for a detailed explanation or actual code: ANY ANSWER mentioning link to docs or the conventional name of such sequence would be immediately accepted.
The following script generates your expected output with a leading space:
Script
for i in {1..100}; do
IFS= read c
printf %s "$c$i"
done < <(yes $' A\nB\nC\n\D')
Output
A1B2C3D4 A5B6C7D8 A9B10C11D12 A13B14C15D16 A17B18C19D20 A21B22C23D24 A25B26C27D28 A29B30C31D32 A33B34C35D36 A37B38C39D40 A41B42C43D44 A45B46C47D48 A49B50C51D52 A53B54C55D56 A57B58C59D60 A61B62C63D64 A65B66C67D68 A69B70C71D72 A73B74C75D76 A77B78C79D80 A81B82C83D84 A85B86C87D88 A89B90C91D92 A93B94C95D96 A97B98C99D100
Explanation
To read the sequence 1 2 3 ... 100 in its full length, we need to repeat the sequence A B C D over and over again. yes is a command that repeats its argument ad infinitum. yes x prints
x
x
x
...
To let yes print something different in every line, we use a trick. $' A\nB\nC\nD' is a string that contains linebreaks ($'' is a so called bash ansi-c quote). yes $' A\nB\nC\nD' will print
A
B
C
D
A
B
...
Instead of printing to the console, we want to consume the text later. To this end, we could write yes ... | someCommand or someCommand < <(yes ...) which has some advantages over a pipe. The latter is called process substitution. Note that for ...; done is also just one command. The redirected stdin can be read from anywhere inside the for loop.
#!/bin/bash
# ASCII code for A
A=65
# Loop from 1 to 100
for ii in $( seq 1 100 )
do
# Compute ASCII code with using modulo
code=$(( (ii-1) % 4 + A ))
# Print letter
printf "\x$(printf %x $code)"
# Print number
echo $ii
done
Related
Taking count from file, say if count = 5, I want to print 5 variables. i.e. A B C D E.
If count = 2, Print 2 variables A B, etc.
I have tried using the ASCII values but couldn't go through it.
for i in {1..5}; do
count=5; a=0;
printf "\x$(printf %x '65+$a')";
count=count+1;
done
if count = 5, I want to print 5 variables. i.e. A B C D E. If count = 2, Print 2 variables A B, etc.
Here's a program that matches your style that does what you are looking for:
a=0
for i in {1..5}; do
printf "\x$(printf %x $(( 65 + a )) )";
a=$((a+1));
done
The first thing to note is that in order to do math in bash, you'll need to use the $(( )) operation. Above, you can see I replaced you '65+$a' with $(( 65 + a )) . That's the big news that you need to get math done.
There were a couple of other little issues, but you were stuck on the $(()) stuff so they weren't clear yet. Incidentally, the 'a' variable can be completely removed from the program to just use the 'i' variable like this:
for i in {1..5}; do
printf "\x$(printf %x $(( 64 + i )) )";
done
I had to change the constant to 64 since we are now counting starting at 1.
The {1..5} expression is a good short cut for 1 2 3 4 5, but you won't be able to put a variable into it. So, if you need to add a count variable back in, consider using the seq program instead like this:
count=$1
for i in $(seq 1 $count); do
printf "\x$(printf %x $(( 64 + i )) )";
done
Note that $() is different than the math operator $(()). $() runs a subcommand returning the results.
method 1: simple brace expansion
#!/bin/bash
# generate a lookup table
vars=( - $(echo {A..Z}) )
# use the elements
for i in {1..5}; do
echo ${vars[$i]}
done
{A..Z} generates 26 strings: A, B, ..., Z
which get stored in an array variable by vars=(...)
we prepend a - that we'll ignore
we can then do 1-based indexing into the array
limited to 26 variables (or whatever range we choose)
method 2: multiple brace expansion to generate arbitrary long variables
#!/bin/bash
if [[ ! $1 =~ ^[0-9]+$ ]]; then
echo "Usage: $0 count"
exit
fi
cmd='{A..Z}'
for (( i=$1; i>26; i=i/26 )); do
cmd="${A..Z}$cmd"
done
vars=( $(eval echo $cmd) )
for (( i=0; i<$1; i++ )); do
echo ${vars[$i]}
done
i/26 does integer division (throws away the remainder)
I'm lazy and generate "more than enough" variables rather than attempting to calculate how many is "exactly enough"
{a..b}{a..b}{a..b} becomes aaa aab aba abb baa bab bba bbb
using eval lets us do the brace expansion without knowing in advance how many sets are needed
Sample output:
$ mkvar.sh 10000 |fmt -64 | tail -5
ORY ORZ OSA OSB OSC OSD OSE OSF OSG OSH OSI OSJ OSK OSL OSM
OSN OSO OSP OSQ OSR OSS OST OSU OSV OSW OSX OSY OSZ OTA OTB
OTC OTD OTE OTF OTG OTH OTI OTJ OTK OTL OTM OTN OTO OTP OTQ
OTR OTS OTT OTU OTV OTW OTX OTY OTZ OUA OUB OUC OUD OUE OUF
OUG OUH OUI OUJ OUK OUL OUM OUN OUO OUP
I'm working with an existing script which was written a bit messily. Setting up a loop with all of the spaghetti code could make a bigger headache than I want to deal with in the near term. Maybe when I have more time I can clean it up but for now, I'm just looking for a simple fix.
The script deals with virtual disks on a xen server. It reads multipath output and asks if particular LUNs should be formatted in any way based on specific criteria. However, rather than taking that disk path and inserting it, already formatted, into a configuration file, it simply presents every line in the format
'phy:/dev/mapper/UUID,xvd?,w',
UUID, of course, is an actual UUID.
The script actually presents each of the found LUNs in this format expecting the user to copy and paste them into the config file replacing each ? with a letter in sequence. This is tedious at best.
There are several ways to increment a number in bash. Among others:
var=$((var+1))
((var+=1))
((var++))
Is there a way to do the same with characters which doesn't involve looping over the entire alphabet such that I could easily "increment" the disk assignment from xvda to xvdb, etc?
To do an "increment" on a letter, define the function:
incr() { LC_CTYPE=C printf "\\$(printf '%03o' "$(($(printf '%d' "'$1")+1))")"; }
Now, observe:
$ echo $(incr a)
b
$ echo $(incr b)
c
$ echo $(incr c)
d
Because, this increments up through ASCII, incr z becomes {.
How it works
The first step is to convert a letter to its ASCII numeric value. For example, a is 97:
$ printf '%d' "'a"
97
The next step is to increment that:
$ echo "$((97+1))"
98
Or:
$ echo "$(($(printf '%d' "'a")+1))"
98
The last step is convert the new incremented number back to a letter:
$ LC_CTYPE=C printf "\\$(printf '%03o' "98")"
b
Or:
$ LC_CTYPE=C printf "\\$(printf '%03o' "$(($(printf '%d' "'a")+1))")"
b
Alternative
With bash, we can define an associative array to hold the next character:
$ declare -A Incr; last=a; for next in {b..z}; do Incr[$last]=$next; last=$next; done; Incr[z]=a
Or, if you prefer code spread out over multiple lines:
declare -A Incr
last=a
for next in {b..z}
do
Incr[$last]=$next
last=$next
done
Incr[z]=a
With this array, characters can be incremented via:
$ echo "${Incr[a]}"
b
$ echo "${Incr[b]}"
c
$ echo "${Incr[c]}"
d
In this version, the increment of z loops back to a:
$ echo "${Incr[z]}"
a
How about an array with entries A-Z assigned to indexes 1-26?
IFS=':' read -r -a alpharray <<< ":A:B:C:D:E:F:G:H:I:J:K:L:M:N:O:P:Q:R:S:T:U:V:W:X:Y:Z"
This has 1=A, 2=B, etc. If you want 0=A, 1=B, and so on, remove the first colon.
IFS=':' read -r -a alpharray <<< "A:B:C:D:E:F:G:H:I:J:K:L:M:N:O:P:Q:R:S:T:U:V:W:X:Y:Z"
Then later, where you actually need the letter;
var=$((var+1))
'phy:/dev/mapper/UUID,xvd${alpharray[$var]},w',
The only problem is that if you end up running past 26 letters, you'll start getting blanks returned from the array.
Use a Bash 4 Range
You can use a Bash 4 feature that lets you specify a range within a sequence expression. For example:
for letter in {a..z}; do
echo "phy:/dev/mapper/UUID,xvd${letter},w"
done
See also Ranges in the Bash Wiki.
Here's a function that will return the next letter in the range a-z. An input of 'z' returns 'a'.
nextl(){
((num=(36#$(printf '%c' $1)-9) % 26+97));
printf '%b\n' '\x'$(printf "%x" $num);
}
It treats the first letter of the input as a base 36 integer, subtracts 9, and returns the character whose ordinal number is 'a' plus that value mod 26.
Use Jot
While the Bash range option uses built-ins, you can also use a utility like the BSD jot utility. This is available on macOS by default, but your mileage may vary on Linux systems. For example, you'll need to install athena-jot on Debian.
More Loops
One trick here is to pre-populate a Bash array and then use an index variable to grab your desired output from the array. For example:
letters=( "" $(jot -w %c 26 a) )
for idx in 1 26; do
echo ${letters[$idx]}
done
A Loop-Free Alternative
Note that you don't have to increment the counter in a loop. You can do it other ways, too. Consider the following, which will increment any letter passed to the function without having to prepopulate an array:
increment_var () {
local new_var=$(jot -nw %c 2 "$1" | tail -1)
if [[ "$new_var" == "{" ]]; then
echo "Error: You can't increment past 'z'" >&2
exit 1
fi
echo -n "$new_var"
}
var="c"
var=$(increment_var "$var")
echo "$var"
This is probably closer to what the OP wants, but it certainly seems more complex and less elegant than the original loop recommended elsewhere. However, your mileage may vary, and it's good to have options!
I am trying to read a file in bash forloop. But I do not know how to put write the script for this.
for i in $( seq 0 $step 10 )
do
echo "Rendering: "$(( i + j ))
python auto_fine.py density000000.vtu velocity000000.vtu $(( i + j ))
done
each and every loop I need to call
i -> 0 python auto_fine.py density000000.vtu velocity000000.vtu
i -> 1 python auto_fine.py density000010.vtu velocity000010.vtu
i -> 2 python auto_fine.py density000020.vtu velocity000020.vtu
It seems to me that you need to zero pad the numbers sed provides to you:
As seen in How to zero pad a sequence of integers in bash so that all have the same width?, you need to do something like
$ seq -f "%06g" 0 10 100
Which returns:
000000
000010
000020
...
000100
All together,
for i in $(seq -f "%06g" 0 10 100)
do
# echo "Rendering: "$(( i + j )) -- not sure what this does
python auto_fine.py density$i.vtu velocity$i.vtu
done
Bash can do this without requiring external tools like seq.
for i in {0..100}; do
[[ $i = *0 ]] || continue
python auto_fine.py density$(printf '%06d' $i).vtu velocity$(printf '%06d' $i).vtu
done
This uses pattern matching (*0) to limit your list to every 10 numbers, which is a bit of a hack, but will work against your sample data.
You could alternately loop against your zero-padded numeric strings directly:
for i in $(printf '%05d0 ' {0..10}); do
python auto_fine.py density$i.vtu velocity$i.vtu
done
This option shows you every 10 items by placing a zero in the printf format after the incrementing number, which becomes the tens digit. If you want more arbitrary sequencing, you might use multipliers, still without spawning external processes:
low=0
high=100
mult=10
for i in $(eval echo {$low..$((high/mult))}); do
n=$(printf '%06d' $((i*mult)))
python auto_fine.py density$n.vtu velocity$n.vtu
done
Note the eval, which lets you expand variables for use in your sequence expression. (If you are getting these numbers from an external source, have your script validate them before using them!)
If you're using bash version 4 (i.e. not the native version on OSX), you also have increments available in sequence expressions. From the man page:
A sequence expression takes the form {x..y[..incr]}, where x and y are
either integers or single characters, and incr, an optional increment,
is an integer.
So perhaps:
low=0
high=100
mult=10
for i in $(eval "printf '%06d ' {$low..$high..$mult}"); do
python auto_fine.py density$i.vtu velocity$i.vtu
done
Note that in sequence expressions, the first member of the sequence is the first number provided, rather than merely a product of a multiplier. We have quotes around the printf to ensure that the sequence expression is expanded by eval, and not interpreted by the command substitution ($(..)).
looping for all the files in the current dir is trivial:
for i in $( ls -1 )
do
# your code here, variable is referenced with $i
done
what's the j variable you are using?
I'm trying to write a Bash script that reads files with several columns of data and multiplies each value in the second column by each value in the third column, adding the results of all those multiplications together.
For example if the file looked like this:
Column 1 Column 2 Column 3 Column 4
genome 1 30 500
genome 2 27 500
genome 3 83 500
...
The script should multiply 1*30 to give 30, then 2*27 to give 54 (and add that to 30), then 3*83 to give 249 (and add that to 84) etc..
I've been trying to use awk to parse the input file but am unsure of how to get the operation to proceed line by line. Right now it stops after the first line is read and the operations on the variables are performed.
Here's what I've written so far:
for file in fileone filetwo
do
set -- $(awk '/genome/ {print $2,$3}' $file.hist)
var1=$1
var2=$2
var3=$((var1*var2))
total=$((total+var3))
echo var1 \= $var1
echo var2 \= $var2
echo var3 \= $var3
echo total \= $total
done
I tried placing a "while read" loop around everything but could not get the variables to update with each line. I think I'm going about this the wrong way!
I'm very new to Linux and Bash scripting so any help would be greatly appreciated!
That's because awk reads the entire file and runs its program on each line. So the output you get from awk '/genome/ {print $2,$3}' $file.hist will look like
1 30
2 27
3 83
and so on, which means in the bash script, the set command makes the following variable assignments:
$1 = 1
$2 = 30
$3 = 2
$4 = 27
$5 = 3
$6 = 83
etc. But you only use $1 and $2 in your script, meaning that the rest of the file's contents - everything after the first line - is discarded.
Honestly, unless you're doing this just to learn how to use bash, I'd say just do it in awk. Since awk automatically runs over every line in the file, it'll be easy to multiply columns 2 and 3 and keep a running total.
awk '{ total += $2 * $3 } ENDFILE { print total; total = 0 }' fileone filetwo
Here ENDFILE is a special address that means "run this next block at the end of each file, not at each line."
If you are doing this for educational purposes, let me say this: the only thing you need to know about doing arithmetic in bash is that you should never do arithmetic in bash :-P Seriously though, when you want to manipulate numbers, bash is one of the least well-adapted tools for that job. But if you really want to know, I can edit this to include some information on how you could do this task primarily in bash.
I agree that awk is in general better suited for this kind of work, but if you are curious what a pure bash implementation would look like:
for f in file1 file2; do
total=0
while read -r _ x y _; do
((total += x * y))
done < "$f"
echo "$total"
done
I have a file with the following format:
a 1 2 3 4
b 7 8
c 120
I want it to be parsed into:
a 10
b 15
c 120
I know this can be easily done with awk, but I'm not familiar with the syntax and can't get it to work for me.
Thanks for any help
ok simple awk primer:
awk '{ for (i=2;i<=NF;i++) { total+=$i }; print $1,total; total=0 }' file
NF is an internal variable that is reset on each line and is equal to the number of fields on that line so
for (i=2;i<=NF;i++) starts a for loop starting at 2
total+=$i means the var total has the value of the i'th field added to it. and is performed for each iteration of the loop above.
print $1,total prints the 1st field followed by the contents of OFS variable (space by default) then the total for that line.
total=0 resets the totals var ready for the next iteration.
all of the above is done on each line of input.
For more info see grymoires intro here
Start from column two and add them:
awk '{tot=0; for(i=2;i<$NF;i++) tot+=$i; print $1, tot;}' file
A pure bash solution:
$ while read f1 f2
> do
> echo $f1 $((${f2// /+}))
> done < file
On running it, got:
a 10
b 15
c 120
The first field is read into variable f1 and the rest of the fields are i f2. In variable f2 , spaces are replaced in place with + and evaluated.
Here's a tricky way to use a subshell, positional parameters and IFS. Works with various amounts of whitespace between the fields.
while read label numbers; do
echo $label $(set -- $numbers; IFS=+; bc <<< "$*")
done < filename
This works because the shell expands "$*" into a single string of the positional parameters joined by the first char of $IFS (documentation)