Yet, i know its a lot faster than doing things by hand. But is there anyway to maybe speed up this script? Multi-thread or something? I'm new to unix and this is my first script =). Open for suggestions or any changes made. Script seems to pause a lot on a certain generated domain randomly.
#!/bin/bash
for domain in $(pwgen -1A0B 2 10000);
do
whois $domain.com | egrep -q '^No match|^NOT FOUND|^Not fo|AVAILABLE|^No Data Fou|has not been regi|No entri'
if [ $? -eq 0 ]; then
echo "$domain.com : available"
else
echo "$domain.com"
fi
done
Before splitting and distribution,
WARNING This seem not to be useful: Asking pwgen to build 10'000 lines formed by two characters between a and z... Also there is only echo $((26*26)) -> 676 possibilities (in fact, as pwgen try to build speakable words, there is only 625 possibilities).
pwgen -1A0B 2 10000 | sort | uniq -c | sort -n | tail
27 ju
27 mu
27 vs
27 xt
27 zx
28 df
28 sy
28 zc
29 dp
29 zd
So with this command, you will do upto 29 times same thing.
Trying 10x to run pwgen -1A0B 2 10000 for printing how much different combinaison is proposed and which combinaison was proposed more time and less time:
for ((i=10;i--;)); do
echo $(
(
(
pwgen -1A0B 2 10000 |
sort |
uniq -c |
sort -n |
tee /dev/fd/6 |
wc -l >/dev/fd/7
) 6>&1 | (
head -n1
tail -n1
)
) 7>&1
)
done
6 bd 625 31 bn
3 bj 625 29 sq
6 je 625 30 ey
4 ac 625 30 sz
5 ds 625 29 wf
4 xw 625 28 qb
4 jj 625 30 pa
6 io 625 29 sg
4 vw 625 30 kb
5 fz 625 31 os
this print:
| | | | |
| | | | \- max proposed pattern
| | | \---- number of times max proposed pattern was issued
| | \-------- number of different differents purposes
| \----------- min proposed pattern
\-------------- number of times min proposed pattern was issued
Create a file with desired domain names first. Call this domains.lst:
pwgen -1A0B 2 10000 > domains.lst
Then create smaller files out of this:
split --lines=100 domains.lst domains.lst.
Then create a script which gets a file-name and processes that file using whois. Also creates an output file input.out.
Create another script that uses & to start the above script in the background for all small chunks. Merge the outputs after all background tasks finish.
Related
I'm trying to retrieve items from Node01.pc and put it within a table.
Example:
echo ${NodeCPU[0]} is able to print the item from the line.
But when I use printf or echo it either breaks or does not display the output from the array item.
The formating of the table seems work and it displays only if it's not the arrays. Could it be that there's more than to the file that I can see?
Node01.pc contains
192.168.0.99
2
70
16
80
4
4
100
4
VS122:NMAD:20:20:1:1
VS122:NAMD:20:20:1:1
RS123:FEM:10:20:1:1
QV999:BEM:20:20:1:1
But I only need lines 3,5,7,9
I'm not sure if what is the best way to do this, or if I even need to store items into arrays.
I thought about retrieving all text from the texts files and making a new file which will contain all the data, but I'm not sure how to do that.
This is the code that I have right now.
#!/bin/bash
Node01=($(cat Node01.pc))
Node02=($(cat Node02.pc))
Node03=($(cat Node03.pc))
Node04=($(cat Node04.pc))
Node05=($(cat Node05.pc))
NodeCPU=("${Node01[2]}" "${Node02[2]}" "${Node03[2]}" "${Node04[2]}" "${Node05[2]}")
NodeMEM=("${Node01[4]}" "${Node02[4]}" "${Node03[4]}" "${Node04[4]}" "${Node05[4]}")
NodeHDD=("${Node01[6]}" "${Node02[6]}" "${Node03[6]}" "${Node04[6]}" "${Node05[6]}")
NodeNET=("${Node01[8]}" "${Node02[8]}" "${Node03[8]}" "${Node04[8]}" "${Node05[8]}")
seperator=----------------------
seperator=$seperator$seperator
rows="%-10s| %-7s| %-7s| %-7s| %-7s\n"
TableWidth=140
printf "%-10s| %-7s| %-7s| %-7s| %-7s\n" NodeNumber CPU MEM HDD NET
printf "%.${TableWidth}s\n" "$seperator"
for((i=0;i<=4;i++))
do
printf "$rows" "$(( $i+1 ))" "${NodeCPU[i]}" "${NodeMEM[i]}" "${NodeHDD[i]}" "${NodeNET[i]}"
done
read
This is an example of what I want to display
NodeNumber | CPU | MEM | HDD | NET
----------------------------------
1 | 10 | 20 | 20 | 40
2 | 10 | 20 | 20 | 40
3 | 10 | 20 | 20 | 40
4 | 10 | 20 | 20 | 40
5 | 10 | 20 | 20 | 40
EDIT This is what I'm currently getting:
NodeNumber| CPU | MEM | HDD | NET
--------------------------------------------
| 4 | 70
| 5 | 90
| 6 | 100
| 6 | 70
| 40 | 40
Issue I'm having is with
printf "$rows" "$(( $i+1 ))" "${NodeCPU[i]}" "${NodeMEM[i]}" "${NodeHDD[i]}" "${NodeNET[i]}"
Why worry about all the separate array? Simply loop over all "Node*.pc" files in the current directory and read the contents of each file into an array with readarray and then output the file count and elements nos. 2, 4, 6, 8 of the array in the proper format (adjust elements output as needed), e.g.
#!/bin/bash
cnt=1 ## file counter
## print heading
printf "NodeNumber | CPU | MEM | HDD | NET\n----------------------------------\n"
for i in Node*.pc; do ## loop over all Node*.pc files in directory
readarray -t node < "$i" ## read contents into array
## output count and elements 2, 4, 6, 8 in proper format
printf "%-11s| %-4s| %-4s| %-4s| %s\n" $((cnt++)) \
"${node[2]}" "${node[4]}" "${node[6]}" "${node[8]}"
done
Example Use/Output
With the example data shown copied to the file Node01.pc in the current directory, you would get:
$ bash node.sh
NodeNumber | CPU | MEM | HDD | NET
----------------------------------
1 | 70 | 80 | 4 | 4
(I called the script node.sh)
It would output the information from each file as separate lines numbered 1, 2, ... Look things over an let me know if this is what you intended. (you can also do the same thing with awk faster by setting FS=\n and treating the lines as columns in a single record)
You can do the same thing in awk with:
awk '
BEGIN {
RS=""; FS="\n"
printf "NodeNumber | CPU | MEM | HDD | NET\n----------------------------------\n"
}
NF >= 9 {
printf "%-11s| %-4s| %-4s| %-4s| %s\n",++cnt,$3,$5,$7,$9
}
' Node*.pc
(note: in awk the field numbers are 1-based, while in bash the array indexes are 0-based)
Output is the same.
Trying to solve this Hackerrank problem " Equalize the Array " using BASH. Here's my Solution:
read size
mostfreq=$(tr "[:space:]" '\n' | sort -n | uniq -c | sort -r -k1 | head -c7 | tr -d "[:space:]")
expr $size - $mostfreq
Passes all test cases except
22
51 51 51 51 51 51 51 51 51 51 51 51 51 51 51 51 51 51 51 51 51 51
When I run on PC, it produces the expected output 0. (0 = minimum deletions to produce an array of only duplicates). However, when I run using Hackerrank platform it gives me a runtime error. I was wondering if anyone has had a similar problem when using BASH on the Hackerrank platform.
Basically, on Hackerrank, you will be getting an error if your expression evaluates to 0. So, any test containing an array which elements are all equal will fail, even the simplest one consisting of a single element:
1
The way I overcame this issue was to evaluate the expression using '' and then printing the result using echo as below:
result=`expr $size - $mostfreq`
echo $result
Hence, the full code may look as follows:
read size
mostfreq=$(tr "[:space:]" '\n' | sort -n | uniq -c | sort -r -k1 | head -c7 | tr -d "[:space:]")
result=`expr $size - $mostfreq`
echo $result
In a bash shell script, I want to go through a list of numbers and then print out the number that occurs most often. If there are several different numbers appearing an equal amount of times, I want to print the highest number. For example, in a file like this:
10
10
10
15
15
20
20
20
20
I want to print the value 20.
How can I achieve this?
If the numbers are in a file, one per line:
sort < myfile | uniq -c | sort -r | head -1
without the count:
A=$(sort < myfile | uniq -c | sort -r | head -1)
set $A
echo $2
You can use this command -
echo 10 10 10 15 15 20 20 20 20 | sed 's/ /\n/g' | sort | uniq -c | sort -V | tail -n 1 | awk '{print $2}'
It will print the number you want.
cat Error00
4 0 375
4 2001 21
4 2002 20
cat Error01
4 0 465
4 2001 12
4 2002 40
4 2016 1
I want output as below
4 0 375 465
4 2001 21 12
4 2002 20 20
4 2016 - 1
i am using the below query. here problem is i m not able to handle grep for two field because space is coming.
please suggest how can to get rid of this.
keylist=$(awk '{print $1,$2'} Error0[0-1] | sort | uniq)
for key in ${keylist} ; do
echo ${key}
val_a=$(grep "^${key}" Error00 | awk '{print $3}') ;val_a=${val_a:---}
val_b=$(grep "^${key}" Error01 | awk '{print $1,$2}') ; val_b=${val_b:--- --}
echo $key ${val_a} >>testreport
done
i m geting the oputput as below
4 375 465
0
4 21 12
2001
4 20 20
2002
4 - 1
2016
A single awk one liner can handle this easily:
awk 'FNR==NR{a[$1,$2]=$3;next}{print $1,$2,(a[$1,$2]?a[$1,$2]:"-"),$3}' err0 err1
4 0 375 465
4 2001 21 12
4 2002 20 40
4 2016 - 1
For formatted output you can use printf instead of print. Like Jonathan Leffler suggest:
printf "%s %-6s %-6s %s\n",$1,$2,(a[$1,$2]?a[$1,$2]:"-"),$3
4 0 375 465
4 2001 21 12
4 2002 20 40
4 2016 - 1
However a general solution is to use column -t for a nice table output:
awk '{....}' err0 err1 | column -t
4 0 375 465
4 2001 21 12
4 2002 20 40
4 2016 - 1
grep is not really the right tool for this job. You can either play with awk or Perl (or Python, or …), or you can use join. However, join only joins on a single column at a time, and you appear to need to join on two columns. So, we're going to have to massage the data so that it will work with join. I'm about to assume you're using bash and so have process substitution available. You can do the job without, but it is fiddlier and involves temporary files (and traps to clean them up, etc).
The key to the join will be to replace the blank between the first two columns with a colon (or any other convenient character — control-A would work fine too), then join the files on column 1 with a replacement character. The inputs must be sorted; the output must have the colon replaced with a blank.
$ join -o 0,1.2,2.2 -a 1 -a 2 -e '-' \
> <(sed 's/ */:/' Error00 | sort) \
> <(sed 's/ */:/' Error01 | sort) |
> sed 's/:/ /'
4 0 375 465
4 2001 21 12
4 2002 20 40
4 2016 - 1
$
The 's/ */:/' operation replaces the first sequence of one or more blanks with a colon; the input data has two blanks between the 4 and the 0 in the first line of Error00. The input to join must be in sorted order of the joining field, here the first field. The output is the join field, the second column of Error00 and the second column of Error01 (remembering that means the second column after the first two have been fused by the colon). If there's an unmatched line in the first file, generate an output line (-a 1); ditto for the second file; and for the missing fields, insert a dash (-e '-'). The final sed removes the colon that was added.
If you want the data formatted, pipe it through awk.
$ join -o 0,1.2,2.2 -a 1 -a 2 -e '-' \
> <(sed 's/ */:/' Error00 | sort) \
> <(sed 's/ */:/' Error01 | sort) |
> sed 's/:/ /' |
> awk '{printf("%s %-6s %-6s %s\n", $1, $2, $3, $4)}'
4 0 375 465
4 2001 21 12
4 2002 20 40
4 2016 - 1
$
I want to use a command to make a crontab that plays an alarm (for my wife). The program is called ipraytime and it gives an output like this.
$ ipraytime -u +2
Prayer schedule for,
City : Custom
Latitude : 021�� 25' 12" N
Longitude : 039�� 49' 47" E
Angle Method : Umm Al-Qurra University
TimeZone : UTC+2.0
Qibla : 061�� 45' 42" W of true North
Date Fajr Shorooq Zuhr Asr Maghrib Isha
--------------------------------------------------------------------
[09-05-2012] 4:19 5:43 12:16 15:35 18:48 20:18
Today's Imsaak : 4:11
Tomorrow's Imsaak : 4:10
Tomorrow's Fajr : 4:18
What i want is that the times format good for a crontab which means i need to switch places of the minute and hour. To be 19 4 instead.
I have made this command but don't know how to make that switch.
ipraytime -u +2| awk 'NR==12 {print $2"\n"$3"\n"$4"\n"$5"\n"$6"\n"$7}' | sed 's/:/ /g'
This gives me an output like this
4 19
5 43
12 16
15 35
18 48
20 18
But i want it to be like this
19 4
43 5
16 12
35 15
48 18
18 20
As that is what a crontab is using. I have played with sort a bit but couldn't find a solution there either.
(Sorry for the bad topic.. didn't know how to write a good one for this)
It's not necessary to use sed at all.
$ ipraytime -u +2 | awk -F ' +|:' 'NR == 12 {for (i = 2; i <= 12; i += 2) print $(i+1), $i}'
19 4
43 5
16 12
35 15
48 18
18 20
Use sed 's/\(.*\):\(.*\)/\2 \1/'
Command:
ipraytime -u +2 | awk 'NR==12 {print $2"\n"$3"\n"$4"\n"$5"\n"$6"\n"$7}'
| sed 's/\(.*\):\(.*\)/\2 \1/'