I have two variables like:
a=200
b=205
and want to find out all numbers between these two numbers (including. these specified numbers).
Check the seq instruction:
seq $a $b
The good tool is seq (as ChronoTrigger already stated), but this is not a bash internal function. Unfortunately the {1..4} notation is not working with variables. But there is a sideway:
a=200; b=205; eval "t=({$a..$b})"; echo ${t[*]}
Output:
200 201 202 203 204 205
The resulting array can be used in a for cycle later. for i in ${t[*]};{ ...;}. But better to use for((...)) loop for that as 1_CR stated.
ADDED
If it should be added some string as prefix or postfix to all elements then it is pretty easy to do:
echo ${t[*]/#/ab}
echo ${t[*]/%/cd}
Output:
ab200 ab201 ab202 ab203 ab204 ab205
200cd 201cd 202cd 203cd 204cd 205cd
ADDED #2
If fixed number of digits needed to be placed to the array this can be used
a=0; b=5; eval "t=({$a..$b})"; printf -v tt "%03d " ${t[*]}; t=($tt)
echo Array length: ${#t[*]}
echo ${t[*]}
Output:
Array length: 6
000 001 002 003 004 005
You could use the bash C-style for loop. Note that a $ is not needed before the a and b; this is characteristic of bash arithmetic expressions.
for ((i=a; i<=b; ++i))
do
echo $i
done
Alternately, to capture the numbers in an array
arr=()
for ((i=a; i<=b; ++i))
do
arr+=($i)
done
echo "${arr[*]}"
Related
Im trying to make a script that creates a file say file01.txt that writes a number on each line.
001
002
...
998
999
then I want to read the file line by line and sum each line and say whether the number is even or odd.
sum each line like 0+0+1 = 1 which is odd
9+9+8 = 26 so even
001 odd
002 even
..
998 even
999 odd
I tried
while IFS=read -r line; do sum+=line >> file02.txt; done <file01.txt
but that sums the whole file not each line.
You can do this fairly easily in bash itself making use of built-in parameter expansions to trim leading zeros from the beginning of each line in order to sum the digits for odd / even.
When reading from a file (either a named file or stdin by default), you can use the initialization with default to use the first argument (positional parameter) as the filename (if given) and if not, just read from stdin, e.g.
#!/bin/bash
infile="${1:-/dev/stdin}" ## read from file provide as $1 or stdin
Which you will use infile with your while loop, e.g.
while read -r line; do ## loop reading each line
...
done < "$infile"
To trim the leading zeros, first obtain the substring of leading zeros trimming all digits from the right until only zeros remain, e.g.
leading="${line%%[1-9]*}" ## get leading 0's
Now using the same type parameter expansion with # instead of %% trim the leading zeros substring from the front of line saving the resulting number in value, e.g.
value="${line#$leading}" ## trim from front
Now zero your sum and loop over the digits in value to obtain the sum of digits:
for ((i=0;i<${#value};i++)); do ## loop summing digits
sum=$((sum + ${value:$i:1}))
done
All that remains is your even / odd test. Putting it altogether in a short example script that intentionally outputs the sum of digits in addition to your wanted "odd" / "even" output, you could do:
#!/bin/bash
infile="${1:-/dev/stdin}" ## read from file provide as $1 or stdin
while read -r line; do ## read each line
[ "$line" -eq "$line" 2>/dev/null ] || continue ## validate integer
leading="${line%%[1-9]*}" ## get leading 0's
value="${line#$leading}" ## trim from front
sum=0 ## zero sum
for ((i=0;i<${#value};i++)); do ## loop summing digits
sum=$((sum + ${value:$i:1}))
done
printf "%s (sum=%d) - " "$line" "$sum" ## output line w/sum
## (temporary output)
if ((sum % 2 == 0)); then ## check odd / even
echo "even"
else
echo "odd"
fi
done < "$infile"
(note: you can actually loop over the digits in line and skip removing the leading zeros substring. The removal ensure that if the whole value is used it isn't interpreted as an octal value -- up to you)
Example Use/Output
Using a quick process substitution to provide input of 001 - 020 on stdin you could do:
$ ./sumdigitsoddeven.sh < <(printf "%03d\n" {1..20})
001 (sum=1) - odd
002 (sum=2) - even
003 (sum=3) - odd
004 (sum=4) - even
005 (sum=5) - odd
006 (sum=6) - even
007 (sum=7) - odd
008 (sum=8) - even
009 (sum=9) - odd
010 (sum=1) - odd
011 (sum=2) - even
012 (sum=3) - odd
013 (sum=4) - even
014 (sum=5) - odd
015 (sum=6) - even
016 (sum=7) - odd
017 (sum=8) - even
018 (sum=9) - odd
019 (sum=10) - even
020 (sum=2) - even
You can simply remove the output of "(sum=X)" when you have confirmed it operates as you expect and redirect the output to your new file. Let me know if I understood your question properly and if you have further questions.
Would you please try the bash version:
parity=("even" "odd")
while IFS= read -r line; do
mapfile -t ary < <(fold -w1 <<< "$line")
sum=0
for i in "${ary[#]}"; do
(( sum += i ))
done
echo "$line" "${parity[sum % 2]}"
done < file01.txt > file92.txt
fold -w1 <<< "$line" breaks the string $line into lines of character
(one digit per line).
mapfile assigns array to the elements fed by the fold command.
Please note the bash script is not efficient in time and not suitable
for the large inputs.
With GNU awk:
awk -vFS='' '{sum=0; for(i=1;i<=NF;i++) sum+=$i;
print $0, sum%2 ? "odd" : "even"}' file01.txt
The FS awk variable defines the field separator. If it is set to the empty string (this is what the -vFS='' option does) then each character is a separate field.
The rest is trivial: the block between curly braces is executed for each line of the input. It compute the sum of the fields with a for loop (NF is another awk variable, its value is the number of fields of the current record). And it then prints the original line ($0) followed by the string even if the sum is even, else odd.
pure awk:
BEGIN {
for (i=1; i<=999; i++) {
printf ("%03d\n", i) > ARGV[1]
}
close(ARGV[1])
ARGC = 2
FS = ""
result[0] = "even"
result[1] = "odd"
}
{
printf("%s: %s\n", $0, result[($1+$2+$3) % 2])
}
Processing a file line by line, and doing math, is a perfect task for awk.
pure bash:
set -e
printf '%03d\n' {1..999} > "${1:?no path provided}"
result=(even odd)
mapfile -t num_list < "$1"
for i in "${num_list[#]}"; do
echo $i: ${result[(${i:0:1} + ${i:1:1} + ${i:2:1}) % 2]}
done
A similar method can be applied in bash, but it's slower.
comparison:
bash is about 10x slower.
$ cd ./tmp.Kb5ug7tQTi
$ bash -c 'time awk -f ../solution.awk numlist-awk > result-awk'
real 0m0.108s
user 0m0.102s
sys 0m0.000s
$ bash -c 'time bash ../solution.bash numlist-bash > result-bash'
real 0m0.931s
user 0m0.929s
sys 0m0.000s
$ diff --report-identical result*
Files result-awk and result-bash are identical
$ diff --report-identical numlist*
Files numlist-awk and numlist-bash are identical
$ head -n 5 *
==> numlist-awk <==
001
002
003
004
005
==> numlist-bash <==
001
002
003
004
005
==> result-awk <==
001: odd
002: even
003: odd
004: even
005: odd
==> result-bash <==
001: odd
002: even
003: odd
004: even
005: odd
read is a bottleneck in a while IFS= read -r line loop. More info in this answer.
mapfile (combined with for loop) can be slightly faster, but still slow (it also copies all the data to an array first).
Both solutions create a number list in a new file (which was in the question), and print the odd/even results to stdout. The path for the file is given as a single argument.
In awk, you can set the field separator to empty (FS="") to process individual characters.
In bash it can be done with substring expansion (${var:index:length}).
Modulo 2 (number % 2) to get odd or even.
I'm researching the rhythmic elements of prime number sequences in binary. I have multiple sets of files containing vertical lists, and I want to apply bitwise logic operators between any two of them line-by-line.
i.e.
$cat binary-superprimes.txt
11
101
1011
10001
11111
101001
111011
1000011
1010011
1101101
$cat binary-perniciousprimes.txt
11
101
111
1011
1101
10001
10011
11111
100101
101001
I'm looking for commands, a script, or an application (I'd prefer commands/a script but its not deal-breaking) that will let me and/or/xor/etc. these outputs in order line-by-line, in much the same style/similar to the output of diff or comm.
Using CentOS 7/Ubuntu 18.04/MacOS 10.15.
edit
Expected output (binary expansion of XORing each entry above in decimal):
0
0
1100
11010
10010
111000
101000
1011100
1110110
1000100
As for what I've tried, as I said I've played around with for loops, but I don't know how (or if its possible) two iterate two lists for comparison in this context (i.e. two "for i in"'s with a single "done" - using $i and $x as inputs for a basic "echo (($x^$i))"
I've also tried a program called "bitwise" but its output is too verbose and it cannot seem to read files, only values.
Assuming your bash version is >= 4.0 and supports mapfile,
would you try the following:
mapfile -t x < "binary-superprimes.txt"
mapfile -t y < "binary-perniciousprimes.txt"
for (( i=0; i<${#x[#]}; i++ )); do
echo "obase=2;" $(( 2#${x[i]} ^ 2#${y[i]} )) | bc
done
Output:
0
0
1100
11010
10010
111000
101000
1011100
1110110
1000100
In case your bash does not support mapfile command, please try the alternative:
while read -r line; do
x+=($line)
done < "binary-superprimes.txt"
while read -r line; do
y+=($line)
done < "binary-perniciousprimes.txt"
for (( i=0; i<${#x[#]}; i++ )); do
echo "obase=2;" $(( 2#${x[i]} ^ 2#${y[i]} )) | bc
done
Hope this helps.
You can use bc for this purpose. First you create file
xor.bc
define xor(x,y) {
auto n,z,t,a,b,c,os,qx,qy;
os=scale;scale=0
n=0;x/=1;y/=1
if(x<0){x=-1-x;n=!n}
if(y<0){y=-1-y;n=!n}
z=0;t=1;while(x||y){
qx=x/4;qy=y/4;
c=(a=x-4*qx)+(b=y-4*qy)
if(!c%2)c=a+4-b
z+=t*(c%4)
t*=4;x=qx;y=qy
}
if(n)z=-1-z
scale=os;return (z)
}
Then you create loop to get the numbers one by one. And you can exec XOR by this:
paste binary-superprimes.txt binary-perniciousprimes.txt |while read var1 var2;
do
echo "ibase=2;obase=2;xor($var1;$var)|bc -l xor.bc
done
I am attempting to loop through a list of integers starting out like so:
start=000
for i in $(seq -w $start 48 006);
However, when I try this code above, the loop seems to loop once and then quit.
What do I need to modify? (The leading zeroes need to stay)
Could you please try following.
start=0
diff=6
for i in $(seq $start $diff 48);
do
printf '%03d\n' $i
done
Output will be as follows.
000
006
012
018
024
030
036
042
048
Problem in OP's tried code:
I believe you have given wrong syntax in seq it should be startpoint then increment_number then endpoint eg-->(seq(start_point increment end_point)). Since you have given them wrongly thus it is printing them only once in loop.
In your attempt it is taking starting point as 0 and should run till 6 with difference of 48 which is NOT possible so it is printing only very first integer value which is fair enough.
EDIT: As per #Cyrus sir's comment adding BASH builtin solution here without using seq.
for ((i=0; i<=48; i=i+6)); do printf '%03d\n' $i; done
seq's input takes a start, increment-by, and finish.
You've reversed the increment-by with finish: seq -w $start 48 006 means start at zero, increment by 48 to finish at 6. The simple fix is seq -w $start 6 48. Note: 006 is not needed, just 6 since seq will equalize the widths of the numbers to two places.
I want to loop through 40 netCDF files. There are 20 files with the variable PRECC and 20 files with the variable PRECL("modelmember001.PRECC.192001-200512.nc", "modelmember002.PRECC.192001-200512.nc", ... ,"modelmember020.PRECC.192001-200512.nc" and for PRECL respectively).
I need to perform multiple cdo (climate data operator) commands with the loop (add PRECC and PRECL files, and change time series form 1920-2005 to 1955-2005).
This is the code that I use:
datadir="path_to_mydatat"
workdir="path_to_folder_for_newfiles"
members="{001 .. 020}"
for model in $members
do
echo 'working with model' ${model}
echo cdo -s add ${datadir}/modelmember${members}.PRECC.192001-200512.nc${datadir}/modelmember${members}.PRECL.192001-200512.nc ${workdir}/modelmember${members}PRECT.192001-200512.nc
# echo cdo -s selyear,1955/2005 ${workdir}/modelmember${members}.PRECT.192001-200512.nc ${workdir}/modelmember${members}.PRECT.195501-200512.nc
Eventually I need 20 files with the name
"modelmember001.PRECT.195501-200512.nc", "modelmember002.PRECT.195501-200512.nc", ... , "modelmember020.PRECT.195501-200512.nc"
This is what I get when I run my code (deliberately with an "echo" in front of the cdo line):
$./cdo_add.sh
{001 .. 020}
working with model {001
cdo -s add /path_to_mydatat/modelmember{001 .. 020}.PRECC.192001-200512.nc /path_to_mydatat/modelmember{001 .. 020}.PRECL.192001-200512.nc /path_to_folder_for_newfiles/modelmember{001 .. 020}.PRECT.192001-200512.nc
working with model ..
cdo -s add /path_to_mydatat/modelmember{001 .. 020}.PRECC.192001-200512.nc /path_to_mydatat/modelmember{001 .. 020}.PRECL.192001-200512.nc /path_to_folder_for_newfiles/modelmember{001 .. 020}.PRECT.192001-200512.nc
working with model 020}
cdo -s add /path_to_mydatat/modelmember{001 .. 020}.PRECC.192001-200512.nc /path_to_mydatat/modelmember{001 .. 020}.PRECL.192001-200512.nc /path_to_folder_for_newfiles/modelmember{001 .. 020}.PRECT.192001-200512.nc
My code doesn't seem to loop through the members. There is something wrong with the way I use the placeholder "members" but I can't figure out how to fix it.
Does anyone have a suggestion?
Cheers!
Your code does not seem to loop because you cannot assign a brace expansion to a variable and expect it to expand when substituted in a for loop. The following saves the literal string "{001 .. 020}" to the variable members, e.g.
members="{001 .. 020}"
When you use members in for model in $members, normal word-splitting occurs because it is just a string and you loop once with 001, then with .. and finally with 020 -- not the expected sequence from 001, 002, 003, ... 020. (there should be no spaces between the number and .. to begin with -- but that still doesn't allow you to use the expansion in a variable)
To properly use the expansion, get rid of the members variable altogether and use {001..020} in the loop, e.g.
for model in {001..020} ## (notice NO space between 001 and ..)
example:
$ for m in {001..020}; do echo $m; done
001
002
003
004
005
006
007
008
009
010
011
012
013
014
015
016
017
018
019
020
That will allow you to loop with your sequence in model.
From the conversation in the comments, I know understand that you have 40 files prefixed by modelmemberXXX (where XXX is 001-020) followed by .PRECC* or .PRECL* (20 files each) that you want to coordinate feeding matching pairs to a cdo command. While the preferred way would be to loop over one matching glob, e.g. for i in modelmember*.PRECC*; do, you can also use your brace expansion approach, e.g.
for i in {001..020}
do
a=$(echo modelmember${i}.PRECC*)
b=$(echo modelmember${i}.PRECL*)
if [ -e "$a" ] && [ -f "$b" ]
then
printf "%s\n%s\n\n" "$a" "$b"
fi
done
(note the [ -e "$a" ] && [ -f "$b" ] test just makes sure both files in the pair exist before proceeding with the command (printf here))
Example Output
modelmember001.PRECC.192001-200512.nc
modelmember001.PRECL.192001-200512.nc
modelmember002.PRECC.192001-200512.nc
modelmember002.PRECL.192001-200512.nc
modelmember003.PRECC.192001-200512.nc
modelmember003.PRECL.192001-200512.nc
...
modelmember020.PRECC.192001-200512.nc
modelmember020.PRECL.192001-200512.nc
You simply need to make use of $a and $b with whatever cdo_cmd you need within the loop. (as noted in the comments, you need to change to the directory containing the files, or precede the filenames with path/to/the/files)
Preferred Way
Rather than using your brace expansion, it is probably preferred to loop over one set (either PRECC or PRECL), validate the other exists, then execute the command, e.g.
for i in modelmember*.PRECC*
do
b="${i/PRECC/PRECL}"
if [ -e "$i" ] && [ -f "$b" ]
then
printf "%s\n%s\n\n" "$i" "$b"
fi
done
(same output)
If I wanted to use ffmpeg to download a bunch of .ts files from a website, and the url format is
http://example.com/video-1080Pxxxxx.ts
Where the xxxxx is a number from 00000 to 99999 (required zero padding), how would I iterate through that in bash so that it tries every integer starting at 00000, 00001, 00002, etc.?
Loop over the integer values from 0 to 99999, and use printf to pad to 5 digits.
for x in {0..99999}; do
zx=$(printf '%05d' $x) # zero-pad to 5 digits
url="http://example.com/video-1080P${zx}.ts"
... # Do something with url
done
In pure bash:
$ n=99999 ; for ((i=0; i<=n; i++)) { s=$(printf "%05d" $i); echo $s ; }
or with a utility:
$ seq -w 0 99999
$ seq --help
Usage: seq [OPTION]... LAST
or: seq [OPTION]... FIRST LAST
or: seq [OPTION]... FIRST INCREMENT LAST
Print numbers from FIRST to LAST, in steps of INCREMENT.
Mandatory arguments to long options are mandatory for short options too.
-f, --format=FORMAT use printf style floating-point FORMAT
-s, --separator=STRING use STRING to separate numbers (default: \n)
-w, --equal-width equalize width by padding with leading zeroes
Why not get do something with a for loop:
for i in 0000{0..9} 000{10..99} 00{100..999} 0{1000..9999} {10000..99999}
do
# Curl was used since some minimal installs of linux do not have wget
curl -O http://example.com/video-1080P"$i".ts
sleep 1
done
(I am sure that there is a much better way to do this but it is not presenting itself to me at the moment)
My Bash (4.3) can do this:
$ echo {001..010}
001 002 003 004 005 006 007 008 009 010
So you could just do
for i in {00000..99999}; do
url="http://example.com/video-1080P${i}.ts"
# Use url
done