I have a script that loops over a curl command, which pulls in data from an API.
LIST_OF_ID=$(curl -s -X POST -d "username=$USER&password=$PASS&action=action" http://link.to/api.php)
for PHONE_NUMBER in $(echo $LIST_OF_ID | tr '_' ' ' | awk '{print $2}');
do
$VOIP_ID = $(echo $LIST_OF_ID | tr '_' ' ' | awk '{print $1}')
done
I also have a variable of 16 numbers in the range of "447856321455"
NUMBERS=$(cat << EOF
441111111111
441111111112
441111111113
... etc
)
The output on the API call is:
652364_441111111112
As you may notice I have taken the output and cut it into 2 parts and put it in a variable.
What I need is to match the 6 digit code from the output where the number in the output, matches with the number in the variable.
I've attempted it using if statements but I can't work my head around the correct way of doing it.
Any help would be appreciated.
Thank you.
I would do it using join rather than a loop in bash. Like this:
curl -s -X POST -d "$PARAMS" "$URL" | sort \
| join -t _ -2 2 -o 2.1 <(sort numbers.txt) -
What this does is take the sorted output from curl and join it with the sorted contents of numbers.txt (you could use $NUMBERS too), using _ as the separator, using column 2 of file 2 which is - meaning stdin (from curl). Then output field 2.1 which is the six-digit ID.
Read why-is-using-a-shell-loop-to-process-text-considered-bad-practice and then do something like this:
curl ... |
awk -v numbers="$NUMBERS" -F'_' '
BEGIN { split(numbers,tmp,/[[:space:]]+/); for (i in tmp) nums[tmp[i]] }
$2 in nums
'
but to be honest I cant really tell what it is you are trying to do as the numbers in your sample input don't seem to match each other (what does in the range of "447856321455" mean and how does it relate to $NUMBERS containing 441111111111 through 441111111113 and how does any of that relate to match the 6 digit code) and the expected output is missing.
Related
This is a text file called a.txt
ok.google.com
abc.google.com
I want to select every subdomain separately
cat a.txt | cut -d "." -f1 (it select ok From left side)
cat a.txt | cut -d "." -f2 (it select google from left side)
Is there any way, so I can get result from right side
cat a.txt | cut (so it can select com From right side)
There could be few ways to do this, one way which I could think of right now could be using rev + cut + rev solution. Which will reverse the input by rev command and then set field separator as . and print fields as per they are from left to right(but actually they are reversed because of the use of rev), then pass this output to rev again to get it in its actual order.
rev Input_file | cut -d'.' -f 1 | rev
You can use awk to print the last field:
awk -F. '{print $NF}' a.txt
-F. sets the record separator to "."
$NF is the last field
And you can give your file directly as an argument, so you can avoid the famous "Useless use of cat"
For other fields, but counting from the last, you can use expressions as suggested in the comment by #sundeep or described in the users's guide under
4.3 Nonconstant Field Numbers. For example, to get the domain, before the TLD, you can substract 1 from the Number of Fields NF :
awk -F. '{ print $(NF-1) }' a.txt
You might use sed with a quantifier for the grouped value repeated till the end of the string.
( Start group
\.[^[:space:].]+ Match 1 dot and 1+ occurrences of any char except a space or dot
){1} Close the group followed by a quantifier
$ End of string
Example
sed -E 's/(\.[^[:space:].]+){1}$//' file
Output
ok.google
abc.google
If the quantifier is {2} the output will be
ok
abc
Depending on what you want to do after getting the values then you could use bash for splitting your domain into an array of its components:
#!/bin/bash
IFS=. read -ra comps <<< "ok.google.com"
echo "${comps[-2]}"
# or for bash < 4.2
echo "${comps[${#comps[#]}-2]}"
google
this is a file that i have named people.txt
10001:Larry Simpson:65:NewYork:555666777
10002:Jonh Fin:91:Rome:333444555
10003:George Jas:86:Amsterdam:777888999
10004:Larry Simpson:65:NewYork:555666777
10005:Jonh Fin:91:Rome:333444555
I was trying to count how many people there was on a specific city that is given as argument of the script.
First thing i thought was:
grep "$1:" people.txt | wc -l
The ":" was because we can have a city named Amster and another named Amsterdam.
But then I realized that we can have people named Amsterdam, so I tried this to search in cities column:
k=$(awk -F ":" -v loc=$1 -v max=0 ' {if ($4==loc) max++; print max}' people.txt)
echo $k
But now the output is like 0 0 1 1 1 and how can I have just the last digit of this output?
I also tried with cut but when doing -f we don´t know how long that output is.
Desired output is just
1
Regards
Assuming $1 is equal to "NewYork":
awk -F: -v loc="$1" '$4==loc { cnt++ } END { print cnt}' people.txt
You need to use the END block to print the final count.
You can just do it with a single grep command:
grep -Ec "^([^:]*:){3}$1:" people.txt
I would like to count the number of students in a .csv file depending on the category
Category 1 is the name, Category 2 is the country, Category 3 is the city
The .csv file is displayed as such :
michael_s;jpa;NYC
john_d;chn;TXS
jim_h;usa;POP
I have tried in my .sh script but it didn't work
sort -k3 -t; students.csv
edit:
I am trying to make a bash script that counts students by city and something that can also count one city just by executing the script such as
cat students.csv | ./script.sh NYC
The terminal will only display the students from NYC
If I've understood you correctly, something like this?
cut -d";" -f3 mike.txt | sort | uniq -c
(Sorry, incorrect solution first time - updated now)
To count only one city:
cut -d";" -f3 mike.txt | grep "NYC" | wc -l
Depending on the size of the file, how often you'll be doing this etc. it may be sensible to look at other solutions, eg. awk. But this solution will work just fine.
The reason for the error message "sort: multi-character tab 'students.csv'" is you haven't given the -t option the separator character. If you add a semicolon after -t, the sort will work as expected:
sort -k3 -t';' students.csv
There is always awk:
$ awk -F\; 'a[$1]++==0{c++}END{print c}' file
3
Once you describe your requirements more throughly, (count the names but sort -k3. Update the OP, please) we can help you better.
Edited to match your update:
$ awk -F\; -v col=3 -v val=NYC '
(length(val) && $col==val) || length(val)==0 && a[$col]++==0 {
c++
}
END { print c }
' file
1
If you set -v val= with the value you are looking for and -v col= with the column number, it counts the occurrences of val in col. You you set col but not val ot counts different values in col.
I have this command :
id=$(xl list|egrep $Name| tr -s ' ' | cut -d ' ' -f 2)
which xl list output something like this:
Name ID Mem VCPUs State Time(s)
Domain-0 0 5923 8 r----- 4266.0
new_redhat9-clone 3 1027 1 r----- 1019.6
new_redhat9 4 1027 1 -b---- 40.1
Actually I want to get the ID of a given Name. This works when Name=new_redhat9-clone (it returns 3) but doesnt work when Name=new_redhat9 (it returns: 3 4!!!!).
what is wrong?!!!
grep searches the string pattern match. egrep new_redhat9 match with "new_redhat9" and "new_redhat9-clone". Try add whiteespace (or \t) after pattern, rewrite like this
id=$(xl list|egrep 'new_redhat9 '| tr -s ' ' | cut -d ' ' -f 2)
You could use awk instead of egrep,tr and cut commands,
id=$(xl list | awk '$1=="new_redhat9" {print $2}')
Awk command searches for the exact string new_redhat9 in the first column of xl list output . If it finds any then then value of column2 on the corresponding record is stored to the variable id.
You could check the output through echo $id command.
If the name is stored in a variable, then give a try to the below command
id=$(xl list | awk -v var=$Name '$1==var {print $2}')
I am really new a bash and I was trying to search a csv file column for a value and then add a counter. I found this online but it prints it and I have been trying to count how many times an R shows up and not print the whole thing.
awk -F "\"*,\"*" '{print $2}' $file
The csv file is like:
12345,R,N,N,Y,N,N,N,Bob Builder
I am looking for R in column 2. Can anybody point me in the right direction?
The following should do what you want (where file.csv is your csv file):
Case sensitive version:
cut -f 2 -d , file.csv | grep -c R
Case insensitive version:
cut -f 2 -d , file.csv | grep -ic R
Explanation
cut -f 2 -d , file.csv
This takes each line of file.csv and extracts the specified fields. The -f 2 option means extract field 2 and the -d , means use a ',' as the field delimiter. The output of this is then piped to grep.
grep -c R This looks for lines containing 'R'. Since it is passed the contents of the previous cut command, it is looking for an 'R' in field two. The -c option means count the number of matching lines.
Using awk only:
awk -F "\",\"" '{if ($2 == "R") cnt++} END{print cnt}' file
For a fun - perl only - this count everything.
perl -F, -anle 'map{$cnt{$_}{$F[$_]}++}0..$#F;END{print $cnt{1}{R}}'