When using seq to generate an ip address, I use seq 0 255 and it generate the last octet. How can I transition this so it will generate all the other octets and their possible combinations (over 4 million combinations). Any help to start would be appreciated
If you were looking for a bash solution:
for h in {1..255}; do for i in {1..255}; do for j in {1..255}; do for k in {1..255}; do echo "$h.$i.$j.$k"; done; done; done; done
Or the multi-line version
for h in {1..255}
do for i in {1..255}
do for j in {1..255}
do for k in {1..255}
do echo "$h.$i.$j.$k"
done
done
done
done
Or if you are really intent on using seq
for h in `seq 255`; do for i in `seq 255`; do for j in `seq 255`; do for k in `seq 255`; do echo "$h.$i.$j.$k"; done; done; done; done
With awk and four loops:
awk 'BEGIN{OFS="."; for(h=0;h<256;h++){for(i=0;i<256;i++){for(j=0;j<256;j++){for(k=0;k<256;k++){print h,i,j,k}}}}}'
With C and four loops:
Put this a file with name ipgen.c:
#include <stdio.h>
int main() {
int h, i, j, k;
for (h = 0; h < 256; h++) {
for (i = 0; i < 256; i++) {
for (j = 0; j < 256; j++) {
for (k = 0; k < 256; k++) {
printf("%d.%d.%d.%d\n", h, i, j, k);
}
}
}
}
return 0;
}
Compile it: gcc ipgen.c -o ipgen
Start it: ./ipgen
For speed I'd vote for a 4-way awk/for loop (per Cyrus comment), but thought I'd look at a recursive function:
tuple () {
local level=$1 # number of tuples to generate
local max=$2 # assume tuples are numbered 1 to max
local in=$3 # current tuple from parent
local pfx="." # for all but the topmost call we append a period on the front of our loop counter
[[ -z "${in}" ]] && pfx="" # topmost call appends no prefix
local i
local out
for (( i=1 ; i<=${max} ; i++ ))
do
out="${in}${pfx}${i}"
if [[ "${level}" -eq 1 ]] # if we've reached the bottom of our function calls
then
echo "${out}" # print our latest string
else
tuple $((level-1)) "${max}" "${out}" # otherwise recurse with the latest string
fi
done
}
To build a 3-tuple with values ranging from 1 to 2:
$ tuple 3 2
1.1.1
1.1.2
1.2.1
1.2.2
2.1.1
2.1.2
2.2.1
2.2.2
To build a 3-tuple with values ranging from 1 to 4:
$ tuple 3 4
1.1.1
1.1.2
1.1.3
1.1.4
1.2.1
...
4.3.4
4.4.1
4.4.2
4.4.3
4.4.4
To build a 4-tuple with values ranging from 1 to 255 (keep in mind this is NOT going to be fast since we're making a LOT of bash-level calls):
$ tuple 4 255
1.1.1.1
1.1.1.2
1.1.1.3
1.1.1.4
1.1.1.5
1.1.1.6
1.1.1.7
1.1.1.8
1.1.1.9
1.1.1.10
... if you let it run long enough ...
255.255.255.250
255.255.255.251
255.255.255.252
255.255.255.253
255.255.255.254
255.255.255.255
Related
I need get checksum Adler32 and store to variable in bash.
It will be used in automatic script and it will be useful if no additional app/liberty will used which user need to install.
Is it possible to use common / basic command Bash command to get this value?
This is monumentally slow (about 60,000 times slower than C), but it shows that yes, it is possible.
#!/bin/bash
sum1=1
sum2=0
while LANG=C IFS= read -r -d '' -n 1 ch; do
printf -v val '%d\n' "'$ch"
(( val = val < 0 ? val + 256 : val, sum1 = (sum1 + val) % 65521, sum2 = (sum2 + sum1) % 65521 ))
done
(( adler = sum1 + 65536 * sum2 ))
echo $adler
Hopefully someone who actually knows bash could vastly improve on this.
Maybe this solution?:
python -c "import zlib; print(zlib.adler32(\"${file}\"))"
Tried two adler bash functions
one with an ordination dictionary and one with printf
also tried some bit shifting like
instead of sum1=(sum1+val)%65521 -> temp= (sum1+val),sum1=temp >> 16 *15 + (temp & 65355)%65521
wasn't able to improve it a lot, perhaps somebody knows a faster one.
last function is a awk function, it is the fastest, works also on files.
#!/bin/bash
a=$'Hello World'; b=""
for ((i=0;i<1000;i++)); do b+=$a; done
#-- building associative array ord byte character array
declare -Ai ordCHAR=()
for ((i=1;i<256;i++)); do printf -v hex "%x" $i; printf -v char "\x"$hex; ordCHAR[$char]=$i; done
unset hex char i
#-- building associative array ord byte character array -- END
#-- with dictionary
function adler32_A ()
{
local char; local -i sum1=1 sum2=0 val
LC_ALL=C; while read -rN 1 char; do
val=${ordCHAR[$char]};
((sum1=(sum1+val) % 65521, sum2 = (sum2 + sum1) % 65521 ))
done <<< $1
#-- removing 0A=\n addition, because of here string
(( sum2-=sum1, sum2<0 ? sum2+=65521 :0, sum1-=val, sum1<0 ? sum1+=65521 :0 ));
printf "%08x" $(( (sum2 << 16) + sum1 ))
LC_ALL=""
}
#-- with printf
function adler32_B ()
{
local char; local -i sum1=1 sum2=0 val
LC_ALL=C; while read -rN 1 char;
do
printf -v val '%d' "'$char"
(( sum1 = (sum1 + val) % 65521, sum2 = (sum2 + sum1) % 65521 ))
done <<< $1
#-- removing 0A=\n addition, because of here string
(( sum2-=sum1, sum2<0 ? sum2+=65521 :0, sum1-=val, sum1<0 ? sum1+=65521 :0 ));
printf "%x" $((sum1 + 65536 * sum2 ))
LC_ALL=""
}
#-- call adler32_awk [text STR] [evaluate text as path bINT]
function adler32_awk ()
{
local -i bPath=$2;
awk -b \
' BEGIN {RS="^$"; bPath='"$bPath"'; for (i=0;i<256;i++) charOrdARR[sprintf("%c",i)]=i; A=1; B=0;}
{
recordSTR=substr($0,1,length($0)-1); if (bPath) {getline byte_data < recordSTR; close(recordSTR);} else byte_data=recordSTR;
l=length(byte_data); for (i=1;i<=l;i++) {
A+=charOrdARR[substr(byte_data,i,1)]; if (A>65520) A-=65521;
B+=A; if (B>65520) B-=65521;}
printf "%x", lshift(B,16)+A; }
' <<<$1
}
time adler32_A "$b"
time adler32_B "$b"
#-- adler 32 of file -> adler32_awk "/home/.../your file" 1
time adler32_awk "$b"
I am trying to imlplement String.hashCode() function in Bash. I Couldn't figure out the bug.
this is my sample implementation
function hashCode(){ #similar function to java String.hashCode()
foo=$1
echo $foo
h=0
for (( i=0; i<${#foo}; i++ )); do
val=$(ord ${foo:$i:1})
echo $val
if ((31 * h + val > 2147483647))
then
h=$((-2147483648 + (31 * h + val) % 2147483648 ))
elif ((31 * h + val < -2147483648))
then
h=$(( 2147483648 - ( 31 * h + val) % 2147483648 ))
else
h=$(( 31 * h + val))
fi
done
printf %d $h
}
function ord() { #asci to int conversion
LC_CTYPE=C printf %d "'$1"
}
Java function looks like this
public int hashCode() {
int h = hash;
if (h == 0 && value.length > 0) {
char val[] = value;
for (int i = 0; i < value.length; i++) {
h = 31 * h + val[i];
}
hash = h;
}
return h;
}
Expected output for string "__INDEX_STAGING_DATA__0_1230ee6d-c37a-46cf-821c-55412f543fa6" is "1668783629" but the output is -148458597
Note - Have to handle java int overflow and underflow.
Vinujan, your code is working for the purpose of hashing a given string using the algorithm you have included. You do not need the ord function as you can cause the literal conversion to ASCII value with printf -v val "%d" "'${foo:$i:1}" (unless you need the LC_CTYPE=C for character set differences).
For example, with just minor tweaks to your code, it will hash the string "hello" properly:
#!/bin/bash
function hashCode() {
local foo="$1"
local -i h=0
for ((i = 0; i < ${#foo}; i++)); do
printf -v val "%d" "'${foo:$i:1}" # val is ASCII val
if ((31 * h + val > 2147483647)) # hash scheme
then
h=$((-2147483648 + (31 * h + val) % 2147483648 ))
elif ((31 * h + val < -2147483648))
then
h=$(( 2147483648 - ( 31 * h + val) % 2147483648 ))
else
h=$(( 31 * h + val))
fi
done
printf "%d" $h # final hashCode in decimal
}
hash=$(hashCode "$1")
printf "\nhashCode: 0x%02x (%d decimal)\n" $hash $hash
Example Use/Output
$ bash hashcode.sh hello
hashCode: 0x5e918d2 (99162322 decimal)
Where you look like you have problems is in the algorithm for hashing itself. For example a longer string like password will result in your scheme returning a negative 64-bit value that looks suspect, e.g.:
$ bash hashcode.sh password
hashCode: 0xffffffffb776462d (-1216985555 decimal)
This may be your intended hash, I have nothing to compare the algorithm against. Look things over, and if you still have problems, edit your question and describe exactly what problems/error/etc. you are getting when you run the script and add that output to your question.
Edit of Hash Function for Better Behavior
Without an algorithm to implement, the only thing I can do is to reformulate the algorithm you provided to be better behaved when the calculations exceed INT_MAX/INT_MIN. Looking at your existing algorithm, it appeared to make the problems worse as large numbers were encountered rather than smoothing the values to insure they remained within the bounds.
Frankly, it looked like you had omitted subtracting INT_MIN or adding INT_MAX to h before reducing the value modulo 2147483648 when it exceeded/fell below those limits. (e.g. you forgot the parenthesis around the subtraction and addition) Simply adding that to the hash algorithm seemed to produce better behavior and your desired output.
I also save the result of your hash calculation in hval, so that it is not computed multiple times each loop, e.g.
function hashCode() {
local foo="$1"
local -i h=0
for ((i = 0; i < ${#foo}; i++)); do
printf -v val "%d" "'${foo:$i:1}" # val is ASCII val
hval=$((31 * h + val))
if ((hval > 2147483647)) # hash scheme
then
h=$(( (hval - 2147483648) % 2147483648 ))
elif ((hval < -2147483648))
then
h=$(( (hval + 2147483648) % 2147483648 ))
else
h=$(( hval ))
fi
done
printf "%d" $h # final hashCode in decimal
}
New Values
Note the hash for "hello" remains the same (as you would expect), but the value for "password" is now better behaved and returns what looks like would be expected, instead of some sign-extended 64-bit value. E.g.,
$ bash hashcode2.sh hello
hashCode: 0x5e918d2 (99162322 decimal)
$ bash hashcode2.sh password
hashCode: 0x4889ba9b (1216985755 decimal)
And note, it does produce your expected output:
$ bash hashcode2.sh "__INDEX_STAGING_DATA__0_1230ee6d-c37a-46cf-821c-55412f543fa6"
hashCode: 0x63779e0d (1668783629 decimal)
Let me know if that is more what you were attempting to do.
I got an lean solution:
hashCode() {
o=$1
h=0
for j in $(seq 1 ${#o})
do
a=$((j-1))
c=${o:$a:1}
v=$(echo -n "$c" | od -d)
i=${v:10:3}
h=$((31 * $h + $i ))
# echo -n a $a c $c i $i h $h
h=$(( (2**31-1) & $h ))
# echo -e "\t"$h
done
echo $h
}
which was wrong. :) The error was in my clever bitwise-ORing of (2**31-1) ^ $h a bitwise ANDing seems a bit wiser: (2**31-1) & $h
This might be condensed to:
hashCode() {
o=$1
h=0
for j in $(seq 1 ${#o})
do
v=$(echo -n "${$o:$((j-1)):1}" | od -d)
h=$(( (31 * $h + ${v:10:3}) & (2**31-1) ))
done
echo $h
}
My files look like
file0 file1 file2
a 1 ##
a 1 ##
b 2 ##
b 2 ##
and I want to merge these files lines by lines, so it should look like
merged file
a
a
1
1
##
##
b
b
2
2
##
##
I mean, choose some lines for each file and merge them into one file.
I tried below bash script.
touch ini.dat
n=2
linenum=$(wc -l < file0)
iter=$((linenum/n))
for i in $(seq 0 1 $iter)
do
for j in $(seq 0 1 2)
do
awk 'NR > '$(($i*$n))' && NR <= '$((($i+1)*$n))'' file"$j" > tmp
cat ini.dat tmp > tmpp
cp tmpp ini.dat
rm tmpp
done
done
It works fine, but takes too much time. Is there any efficient way?
Limiting Factors
Your script had two flaws which made it slow:
A lot of files were created and copied. Especially the ... > tmp; cat ini.dat tmp > tmpp; cp tmpp ini.dat could have been written as ... >> ini.dat.
To read the i-th line of a file, the script has to scan that file from the beginning until the i-th line is reached. If done repeatedly for i = 1, 2, 3, ..., n it will take O(n2). Reading the whole file once (O(n)) into an array and accesing the lines by indices (O(1)) only takes O(n).
Pure Bash Solution
The following bash script does the job a bit faster. linesPerBlock corresponds to the parameter n from your script. The script will print as much blocks as possible. That is:
Once the shortest input file was printed, the script terminates. Following lines from longer files will not be printed.
If the shortest input file's number of lines is not divisible by n, the last lines (fewer than n) will be omitted.
#! /bin/bash
files=(file{0..2})
linesPerBlock=2
starts=(0)
maxLines=9223372036854775807 # bash's max. number
for i in "${!files[#]}"; do
lineCount="$(wc -l < "${files[i]}")"
(( lineCount < maxLines )) && (( maxLines = lineCount ))
(( starts[i+1] = starts[i] + maxLines ))
mapfile -t -O "${starts[i]}" -n "$maxLines" lines < "${files[i]}"
done
for (( b = 0; b < maxLines / linesPerBlock; ++b )); do
for f in "${!files[#]}"; do
start="${starts[f]}"
for (( i = 0; i < linesPerBlock; ++i )); do
echo "${lines[start + b*linesPerBlock + i]}"
done
done
done > outputFile
This awk should do the job and will be much quicker that your shell script:
awk 'fn != FILENAME {
fn = FILENAME
n = 1
}
NF {
a[FILENAME,n++] = $0
}
END {
for(i=0; i<(n-1)/2; i++) {
for(j=1; j<ARGC; j++)
printf "%s\n%s\n", a[ARGV[j],i*2+1], a[ARGV[j],i*2+2];
print ""
}
}' file{0..2}
a
a
1
1
##
##
b
b
2
2
##
##
In a single line:
awk 'fn != FILENAME{fn=FILENAME; n=1} NF{a[FILENAME,n++]=$0} END{for(i=0; i<(n-1)/2; i++) { for(j=1; j<ARGC; j++) printf "%s\n%s\n", a[ARGV[j],i*2+1], a[ARGV[j],i*2+2]; print "" } }' file{0..2}
here is another awk, not caching all contents
paste file{0..2} | awk -v n=2 '
function pr() {for(j=1;j<=NF;j++)
for(i=0;i<n;i++) print a[i,j]}
{for(j=1;j<=NF;j++) a[c+0,j]=$j; c++}
!(NR%n) {pr(); delete a; c=0}
END {pr()}'
if the number of lines is not divisible by n, it will fill up with empty lines.
Hi :) Need help on a project, working on shell scripting and need to figure out how to print car names after certain numbers when they're divisible by certain numbers in a list.
Here's the generator, it takes two integers from the user, (Section where they're prompted not included), and prints the evens between those. I need to print car names after numbers divisible by: 5, 7, 10.
5 = Ford 7 = Bmw 10 = Rover
Generator:
for ((n = n1; n<= n2; ++n)); do
out=$(( $n % 2 ))
if [ $out -eg 0 ] ; then
echo "$n"
fi
done
Any help would be appreciated :( I'm completely clueless :(
Thanks
awk to the rescue!
$ awk -v start=0 -v end=70 -v pat='5=Ford,7=Bmw,10=Rover'
'BEGIN{n=split(pat,p,",");
for(i=1;i<=n;i++)
{split(p[i],d,"=");
a[d[1]]=d[2]}
for(i=start;i<=end;i+=2)
{printf "%s ",i;
for(k in a)
if(i%k==0)
printf "%s ", a[k];
print ""}}'
instead of hard coding the fizzbuzz values, let the program handle it for you. script parses the pattern and assigns divisor/value to a map a. While iterating over from start to end two by two check for each divisor and if divides append the tag to the line. Assumes start is entered as an even value if not need to guard for that too.
Do you mean something like this?
n1=0
n2=10
for ((n = n1; n <= n2; ++n)); do
if (( n % 2 == 0)); then
echo -n $n
if (( n % 5 == 0)); then
echo -n ' Ford'
fi
if (( n % 7 == 0)); then
echo -n ' BMW'
fi
if (( n % 10 == 0)); then
echo -n ' Rover'
fi
echo
fi
done
Output
0 Ford BMW Rover
2
4
6
8
10 Ford Rover
Not sure you want the 0 line containing names though, but that's how % works. You can add an explicit check for 0 if you wish.
I want to manage subvariables in Bash. I can assign the subvariables, but I dont know how to use it:
#/bin/bash
n=1
for lvl in 1 2;
do
export key$n="${RANDOM:0:2}"
let n=$n+1
done
for num in 1 2; do
echo $key$num
done
If I use echo $key$num, it print number sequence of variable $num, and not the random numbers
Use arrays.
for n in 1 2; do
key[n]="${RANDOM:0:2}"
done
for num in 1 2; do
echo "${key[num]}"
done
See http://mywiki.wooledge.org/BashGuide/Arrays.
Also, in bash you'll generally do better counting from 0 instead of 1, and you don't need to export variables unless you want to run some other program that is going to look for them in its inherited environment.
You may use arrays (see #MarkReed), or use declare:
for n in 1 2; do
declare -- key$n="${RANDOM:0:2}"
done
for n in 1 2; do
v=$(declare -p key$n) ; v="${v#*=}" ; echo "${v//\"/}"
done
The same using functions:
key_set () # n val
{
declare -g -- key$1=$2
}
key_get () # n
{
local v=$(declare -p key$1) ; v="${v#*=}" ; echo "${v//\"/}"
}
for n in 1 2; do
key_set $n "${RANDOM:0:2}"
done
for n in 1 2; do
key_get $n
done