I have a question which I think is fairly simply but I am new to Bash and can't find much info on this.
5 references 3
10 references 4
20 references 10
30 references 20
inputBeforeLookup = 5 #this the number which needs to look up 3 above^^^^
# 10 would lookup and return 4
#20 returns 10
start = 1
end = $start + $lookupNumberfromFile # 3 in this case, since input was 5
seq $start $end
1
2
3
4
I guess my question here is what is the proper way to create like a configuration file which references numbers to other numbers?
If there is a better way than the snippet of code I posted I am always open to suggestions, like i said I am learning.
I am new to this so I am not sure if the syntax is 100% correct. I am more so looking for a solution on the best way to solve the problem.
Hope this sample helps you regarding variable expansion in bash:
Notice that: the \ prevents the expansion of $$ (current process id). For triple substitution you need double eval and so on....
#!/bin/bash
one=1
two=one
three=two
four=three
five=four
echo $one
eval echo \$$two
eval eval echo \\$\$$three
eval eval eval echo \\\\$\\$\$$four
eval eval eval eval echo \\\\\\\\$\\\\$\\$\$$five
Output:
1
1
1
1
1
Bonus:
In zsh you can use nested substitution much more easily:
#!/bin/zsh
one=1
two=one
three=two
four=three
five=four
echo $one
echo ${(P)two}
echo ${(P)${(P)three}}
...
http://zsh.sourceforge.net/Doc/Release/Expansion.html
Set up an associative array, then test it with numbers 1 to 30. Those numbers that don't reference other numbers are printed as is:
MYMAP=( [5]=3 [10]=4 [20]=10 [30]=20 )
seq 30 | while read x ; do echo ${MYMAP[$x]:-$x} ; done | paste - - - - -
That last | paste - - - - - isn't necessary, but 5 column output is easier to follow given that the input has several multiples of 5. Output:
1 2 3 4 3
6 7 8 9 4
11 12 13 14 15
16 17 18 19 10
21 22 23 24 25
26 27 28 29 20
Related
So I have a C program that outputs many numbers. I have to check them all. The problem is, each time I run my program, I need to change seeds. In order to do that, I've been doing it manually and was trying to make a shell script to work around this.
I've tried using sed but couldn't manage to do it.
I'm trying to get the output like this:
a=$(./algorithm < input.txt)
b=$(./algorithm2 < input.txt)
c=$(./algorithm3 < input.txt)
The output of each algorithm program is something like this:
12 13 315
1 2 3 4 5 6 7 8 10 2 8 9 1 0 0 2 3 4 5
So the variable a has all this output, and what I need is
variable a to contain this whole string
and variable a1 to contain only the third number, in this case, 315.
Another example:
2 3 712
1 23 15 12 31 23 3 2 5 6 6 1 2 3 5 51 2 3 21
echo $b should give this output:
2 3 712
1 23 15 12 31 23 3 2 5 6 6 1 2 3 5 51 2 3 21
and echo $b1 should give this output:
712
Thanks!
Not exactly what you are asking, but one way to do this would be to store the results of your algorithm in arrays, and then dereference the item of interest. You'd write something like:
a=( $(./algorithm < input.txt) )
b=( $(./algorithm2 < input.txt) )
c=( $(./algorithm3 < input.txt) )
Notice the extra () that encloses the statements. Now, a, b and c are arrays, and you can access the item of interest like ${a[0]} or $a[1].
For your particular case, since you want the 3rd element, that would have index = 2, hence:
a1=${a[2]}
b1=${b[2]}
c1=${c[2]}
Since you are using the Bash shell (see your tags), you can use Bash arrays to easily access the individual fields in your output strings. For example like so:
#!/bin/bash
# Your lines to gather the output:
# a=$(./algorithm < input.txt)
# b=$(./algorithm2 < input.txt)
# c=$(./algorithm3 < input.txt)
# Just to use your example output strings:
a="$(printf "12 13 315 \n 1 2 3 4 5 6 7 8 10 2 8 9 1 0 0 2 3 4 5")"
b="$(printf "2 3 712 \n 1 23 15 12 31 23 3 2 5 6 6 1 2 3 5 51 2 3 21")"
# Put the output in arrays.
a_array=($a)
b_array=($b)
# You can access the array elements individually.
# The array index starts from 0.
# (The names a1 and b1 for the third elements were your choice.)
a1="${a_array[2]}"
b1="${b_array[2]}"
# Print output strings.
# (The newlines in $a and $b are gobbled by echo, since they are not quoted.)
echo "Output a:" $a
echo "Output b:" $b
# Print third elements.
echo "3rd from a: $a1"
echo "3rd from b: $b1"
This script outputs
Output a: 12 13 315 1 2 3 4 5 6 7 8 10 2 8 9 1 0 0 2 3 4 5
Output b: 2 3 712 1 23 15 12 31 23 3 2 5 6 6 1 2 3 5 51 2 3 21
3rd from a: 315
3rd from b: 712
Explanation:
The trick here is that array constants (literals) in Bash have the form
(<space_separated_list_of_elements>)
for example
(1 2 3 4 a b c nearly_any_string 99)
Any variable that gets such an array assigned, automatically becomes an array variable. In the script above, this is what happens in a_array=($a): Bash expands the $a to the <space_separated_list_of_elements> and reads the whole expression again interpreting it as an array constant.
Individual elements in such arrays can be referenced like variables by using expressions of the form
<array_name>[<idx>]
like a variable name. Therein, <array_name>is the name of the array and <idx> is an integer that references the individual element. For arrays that are represented by array constants, the index counts elements continuously starting from zero. Therefore, in the script, ${a_array[2]} expands to the third element in the array a_array. If the array would have less elements, a_array[2] would be considered unset.
You can output all elements in the array a_array, the corresponding index array, and the number of elements in the array respectively by
echo "${a_array[#]}"
echo "${!a_array[#]}"
echo "${#a_array[#]}"
These commands can be used to track down the fate of the newline: Given the script above, it is still in $a, as can be seen by (watch the quotes)
echo "$a"
which yields
12 13 315
1 2 3 4 5 6 7 8 10 2 8 9 1 0 0 2 3 4 5
But the newline did not make it into the array a_array. This is because Bash considers it as part of the whitespace that separates the third and the fourth element in the array assignment. The same applies if there are no extra spaces around the newline, like here:
12 13 315\n1 2 3 4 5 6 7 8 10 2 8 9 1 0 0 2 3 4 5
I actually assume that the output of your C program comes in this form.
This will store the full string in a[0] and the individual fields in a[1-N]:
$ tmp=$(printf '12 13 315\n1 2 3 4 5 6 7 8 10 2 8 9 1 0 0 2 3 4 5\n')
$ a=( $(printf '_ %s\n' "$tmp") )
$ a[0]="$tmp"
$ echo "${a[0]}"
12 13 315
1 2 3 4 5 6 7 8 10 2 8 9 1 0 0 2 3 4 5
$ echo "${a[3]}"
315
Obviously replace $(printf '12 13 315\n1 2 3 4 5 6 7 8 10 2 8 9 1 0 0 2 3 4 5\n') with $(./algorithm < input.txt) in your real code.
Here is what I have:
#!/bin/bash
#create a multiplication table 5 columns 10 rows
echo " Multiplication Table "
echo "-----+-------------------------"
for x in {0..5}
do
for y in {0..10}
do
echo -n "$(( $x * $y )) "
done
echo
echo "-----+--------------------------"
done
This is my Output:
Multiplication Table
-----+-------------------------
0 0 0 0 0 0 0 0 0 0 0
-----+--------------------------
0 1 2 3 4 5 6 7 8 9 10
-----+--------------------------
0 2 4 6 8 10 12 14 16 18 20
-----+--------------------------
0 3 6 9 12 15 18 21 24 27 30
-----+--------------------------
0 4 8 12 16 20 24 28 32 36 40
-----+--------------------------
0 5 10 15 20 25 30 35 40 45 50
-----+--------------------------
This is the Needed Output:
Multiplication Table
----+-------------------------------------
| 0 1 2 3 4
----+-------------------------------------
0 | 0 0 0 0 0
1 | 0 1 2 3 4
2 | 0 2 4 6 8
3 | 0 3 6 9 12
4 | 0 4 8 12 16
5 | 0 5 10 15 20
6 | 0 6 12 18 24
7 | 0 7 14 21 28
8 | 0 8 16 24 32
9 | 0 9 18 27 36
----+-------------------------------------
I've tried to write this many different ways, but I'm struggling with finding a way to format it correctly. The first is pretty close, but I need it to have the sequential numbers being multiplied on the top and left side. I'm not sure how to use, or if I can use, the seq command to achieve this or if there is a better way. I also need to have straight columns and rows with the defining lines setting the table layout, but my looking up the column command hasn't produced the right output.
Here was my final output and code:
#!/bin/bash
#create a multiplication table 5 columns 10 rows
#Create top of the table
echo " Multiplication Table"
echo "----+------------------------------"
#Print the nums at top of table and format dashes
echo -n " |"; printf '\t%d' {0..5}; echo
echo "----+------------------------------"
#for loops to create table nums
for y in {0..9}
do
#Print the side nums and |
echo -n "$y |"
#for loop to create x
for x in {0..5}
do
#Multiply vars, tab for spacing
echo -en "\t$((x*y))"
done
#Print
echo
done
#Print bottom dashes for format
echo "----+------------------------------"
I changed a bit of Armali's code just to make it more appealing to the eye, and the echo was moved to the bottom (out of the loop) so it didn't print as many lines. But again, thank you Armali, as I would've spent a lot more time figuring out exactly how to write that printf code to get the format correct.
I'm not sure how to use, or if I can use, the seq command to achieve this …
seq offers no advantage here over bash's sequence expression combined with printf.
This variant of your script produces (with the usual 8-column tabs) the needed output:
#!/bin/bash
#create a multiplication table 5 columns 10 rows
echo " Multiplication Table"
echo "----+-------------------------------------"
echo -n " |"; printf '\t%d' {0..4}; echo
echo "----+-------------------------------------"
for y in {0..9}
do echo -n "$y |"
for x in {0..4}
do echo -en "\t$((x*y))"
done
echo
echo "----+-------------------------------------"
done
I'm currently using:
printf "%14s %14s %14s %14s %14s %14s\n" $(cat NFE.txt)>prueba.txt
This reads a list in NFE.txt and generates 6 columns. I need to generate N columns where N is a variable.
Is there a simple way of saying something like:
printf "N*(%14s)\n" $(cat NFE.txt)>prueba.txt
Which generates the desire output?
# T1 is a white string with N blanks
T1=$(printf "%${N}s")
# Replace every blank in T with string %14s and assign to T2
T2="${T// /%14s }"
# Pay attention to that T2 contains a trailing blank.
# ${T2% } stands for T2 without a trailing blank
printf "${T2% }\n" $(cat NFE.txt)>prueba.txt
You can do this although i don't know how robust it will be
$(printf 'printf '; printf '%%14s%0.s' {1..6}; printf '\\n') $(<file)
^
This is your variable number of strings
It prints out the command with the correct number of string and executes it in a subshell.
Input
10 20 30 40 50 1 0
1 3 45 6 78 9 4 3
123 4
5 4 8 4 2 4
Output
10 20 30 40 50 1
0 1 3 45 6 78
9 4 3 123 4 5
4 8 4 2 4
You could write this in pure bash, but then you could just use an existing language. For example:
printf "$(python -c 'print("%14s "*6)')\n" $(<NFE.txt)
In pure bash, you could write, for example:
repeat() { (($1)) && printf "%s%s" "$2" "$(times $(($1-1)) "$2")"; }
and then use that in the printf:
printf "$(repeat 6 "%14s ")\n" $(<NFE.txt)
I have been googling and trying different methods but nothing seems to work.
I have the following code
string=0 4 5 27 8 7 0 6
total=0
for n in "$string"; do
total=$(($total + $n))
done
This way I want to count the total sum of all the numbers within that string.
I have also tried expr "$total" + "$n" but that gives me an error saying the operand is not an integer.
Any suggestion how I might make this work?
Don't quote the string in the in clause, quoted string is not split into words:
#! /bin/bash
total=0
string='0 4 5 27 8 7 0 6'
for n in $string ; do
(( total += n ))
done
echo $total
string=0 4 5 27 8 7 0 6
This attempts to set the variable string to 0, then invoke the command 4 with arguments 5 27 8 7 0 6.
You need to quote the value:
string="0 4 5 27 8 7 0 6"
And you need to remove the quotes when you refer to it; change
for n in "$string"; do
to
for n in $string; do
You should use :
total=$(( total + n ))
no need for the $ before variables inside a $(( )) statement
I have a time series of files 0000.vx.dat, 0000.vy.dat, 0000.vz.dat; ...; 0077.vx.dat, 0077.vy.dat, 0077.vz.dat... Each file is a space-separated 2D matrix. I would like to take each triplet of files and combine them all into a coordinate-based data format, i.e.:
[timestep + 1] [i] [j] [vx(i,j)] [vy(i,j)] [vz(i,j)]
Each file number corresponds to a particular time step. Given the amount of data I have in this time series (~ 4 GB), bash wasn't cutting it so it seemed to be time to head over to awk... specifically mawk. It was pretty stupid to try this in bash but here is
my ill-fated attempt:
for x in $(seq 1 78)
do
tfx=${tf[$x]} # an array of padded zeros
for y in $(seq 1 1568)
do
for z in $(seq 1 1344)
do
echo $x $y $z $(awk -v i=$z -v j=$y "FNR == i {print j}" $tfx.vx.dat) $(awk -v i=$z -v j=$y "FNR == i {print j}" $tfx.vy.dat) $(awk -v i=$z -v j=$y "FNR == i {print j}" $tfx.vz.dat) >> $file
done
done
done
edit: Thank you, ruakh, for pointing out that I had kept j in shell variable format with a $ in front! This is just a snippet of the original script, but I guess would be considered the guts of it!
Suffice it to say this would have taken about six months because of all the memory overhead in bash associated with O(MxN) algorithms, subshells and pipes and whatnot. I was looking for more along the lines of a day at most. Each file is around 18 MB, so it should not be that much of a problem. I would be happy with doing this one timestep at a time in awk provided that I get one output file per timestep. I could just cat them all together without much issue afterwords, I think. It is important, though, that the time step number be the first item on the coordinate list. I could achieve this with an awk -v argument (see above) in with a bash routine. I do not know how to look up specific elements of matrices in three separate files and put them all together into one output. This is the main hurdle I would like to overcome. I was hoping mawk could provide a nice balance between effort and computational speed. If this seems to be too much for an awk script, I could go to something lower level, and would appreciate any of those answering letting me know I should just go to C instead.
Thank you in advance! I really like awk, but am afraid I am a novice.
The three files, 0000.vx.dat, 0000.vy.dat, and 0000.vz.dat would read as follows (except huge and of the correct dimensions):
0000.vx.dat:
1 2 3
4 5 6
7 8 9
0000.vy.dat:
10 11 12
13 14 15
16 17 18
0000.vz.dat:
19 20 21
22 23 24
25 26 27
I would like to be able to input:
awk -v t=1 -f stackoverflow.awk 0000.vx.dat 0000.vy.dat 0000.vz.dat
and get the following output:
1 1 1 1 10 19
1 1 2 2 11 20
1 1 3 3 12 21
1 2 1 4 13 22
1 2 2 5 14 23
1 2 3 6 15 24
1 3 1 7 16 25
1 3 2 8 17 26
1 3 3 9 18 27
edit: Thank you, shellter, for suggesting I put the desired input and output more clearly!
Personally, I use gawk to process most of my text files. However, since you have requested a mawk compatible solution, here's one way to solve your problem. Run, in your present working directory:
for i in *.vx.dat; do nawk -f script.awk "$i" "${i%%.*}.vy.dat" "${i%%.*}.vz.dat"; done
Contents of script.awk:
FNR==1 {
FILENAME++
c=0
}
{
for (i=1;i<=NF;i++) {
c++
a[c] = (a[c] ? a[c] : FILENAME FS NR FS i) FS $i
}
}
END {
for (j=1;j<=c;j++) {
print a[j] > sprintf("%04d.dat", FILENAME)
}
}
When you run the above, the results should be a single file for each set of three files containing your coordinates. These output files will have the filenames in the form: timestamp + 1 ".dat". I decided to pad these filenames with four 0's for your convenience. But you can change this to whatever format you like. Here's the results I get from the sample data you've posted. Contents of 0001.dat:
1 1 1 1 10 19
1 1 2 2 11 20
1 1 3 3 12 21
1 2 1 4 13 22
1 2 2 5 14 23
1 2 3 6 15 24
1 3 1 7 16 25
1 3 2 8 17 26
1 3 3 9 18 27