I have a text file with a whole bunch of lines (1000 exactly) and they all have 4 bits of text, seperated by a ;.
Here is the for loop I'm using, to go through each line:
while IFS= read -r line; do
let liner++
if [[ liner -eq "1" ]]; then
continue
fi
name=$(echo "${line}" | cut -d';' -f1)
fullname=$(echo "${line}" | cut -d';' -f2)
id=$(echo "${line}" | cut -d';' -f3)
test=$(echo "${line}" | cut -d';' -f4)
echo "${GREEN}$(($liner-1))) ${name} ${ORANGE}v${test} ${RED}(${id})${NC}"
stuff+=("${fullname}")
done < list.txt
It takes about 5 seconds before it finishes running and I believe it's from all those cut (name, fullname, id, test) variables. What would be the best solution to speed this up?
Awk undoubtedly provides a better solution, but if you don't want to learn Awk right now, you could speed your function up a lot by just using read to split the lines into fields:
liner=0
stuff=()
while IFS=\; read -r name fullname id test; do
echo "$GREEN$((++liner))) $name ${ORANGE}v$test $RED($id)$NC"
stuff+=("$fullname")
done < <(tail -n+2 1000num.txt)
Related
i have this CSV file. how can i copy the first row and the next 2 values. server.2=cl1z2 server.3=cl1z3. i want to repeat this task with the other Values. (e.g. third row and server.2=cl2z2 server.3=cl2z3 )
z_name;z_hosts;z_clientPort;z_leaderPort;z_electionport;tickTime;initLimit;syncLimit;snapRetainCount;purgeInterval
cl1z1;server.1=cl1z1;2180;2890;3890;200;5;2;3;24
cl1z2;server.2=cl1z2;2181;2891;3891;200;5;2;3;24
cl1z3;server.3=cl1z3;2182;2892;3892;200;5;2;3;24
cl2z1;server.1=cl2z1;2183;2893;3893;200;5;2;3;24
cl2z2;server.2=cl2z2;2184;2894;3894;200;5;2;3;24
cl2z3;server.3=cl2z3;2185;2895;3895;200;5;2;3;24
Thanks for Help.
You can try this
result=""
while read line ; do
if echo "$line" | grep -q 'z_name'; then
result+="$line\n"
else
result+=$(echo "$line" | cut -d';' -f2 | cut -d';' -f1)"\n"
fi
done < your_csv_file
echo -e "$result"
#!/bin/bash
echo -n "Enter the domain name > "
read name
dig -t ns "$name" | cut -d ";" -f3 | cut -d ":" -f2| grep ns | cut -f6 > registerfile.txt;
cat registerfile.txt | while read line; do dig axfr "#$line" "$name"; done | cut -d"." -f-4 > nmap.txt
It is done till this section. Below, it could be not matched the line and name parameters. How should be changed?
cat nmap.txt | while read line; do if [ "$line" == "$name" ]; then host "$line"; fi; done > ping.txt
cat ping.txt | cut -d " " -f4 | while read line; do if [[ "$line" =~ ^[0-9]+$ ]]; then nmap -sS "$line";fi ;done
It's not clear where exactly things are going wrong, but here is a refactoring which might hopefully at least nudge you in the right direction.
#!/bin/bash
read -p "Enter the domain name > " name
dig +short -t ns "$name" |
tee registerfile.txt |
while read line; do
dig axfr "#$line" "$name"
done |
cut -d"." -f-4 |
tee nmap.txt |
while read line; do
if [ "$line" = "$name" ]; then
host "$line"
fi
done > ping.txt
cut -d " " -f4 ping.txt |
grep -E '^[0-9]+$' |
xargs -r -n 1 nmap -sS
Your remark in comments that if [ "$line" = "$name" ]; then host "$line"; fi isn't working suggests that the logic there is somehow wrong. It currently checks whether each line is identical to the original domain name, and then looks it up over and over again in those cases, which seems like a curious thing to do; but given only the code and the "does not work", it's hard to say what it's really supposed to accomplish. If you actually want something else, you need to be more specific about what you require. Perhaps you are actually looking for something like
... tee nmap.txt |
# Extract the lines which contain $name at the end
grep "\.$name\$" |
xargs -n 1 dig +short |
tee ping.txt |
grep -E '^[0-9]+$' ...
The use of multiple statically-named files is an antipattern; obviously, if these files serve no external purpose, just take out the tee commands and run the entire pipeline with no in-between output files. If you do need these files, having them overwritten on each run seems problematic -- maybe add a unique date stamp suffix to the file names?
I have a script which iterates through a file and finds matches in another file. How to I get the process to stop once I've found a match.
For example:
I take the first line in name.txt, and then try to find a match for it in file.txt.
name.txt:
7,7,FRESH,98,135,
65,10,OLD,56,45,
file.txt:
7,7,Dave,S
8,10,Frank,S
31,7,Gregg
45,5,Jake,S
Script:
while read line
do
name_id=`echo $line | cut -f1,2 -d ','`
identiferOne=`echo $name_id | cut -f1 -d ','`
identiferTwo=`echo $name_id | cut -f2 -d ','`
while IFS= read line
do
CHECK=`echo $line | cut -f4 -d','`
if [ $CHECK = "S" ]
then
symbolName=`echo $line | cut -f3 -d ','`
numberOne=`echo $line | awk -F',' '{print $1}'`
numberTwo=`echo $line | cut -f2 -d ','`
if [ "$numberOne" == $identiferOne ] && [ "$numberTwo" == $identifierTwo ]
then
echo "WE HAVE A MATCH with $symbolName"
break
fi
fi
done < /tmp/file.txt
done < /tmp/name.txt
My question is - how do I stop the script from iterating through file.txt once it has found an initial match, and then set that matched record into a variable, stop the if statement, then do some other stuff within the loop using that variable. I tried using break; but that exits the loop, which is not what I want.
You can tell grep different things:
Stop searching after the first match (option -m 1).
Read the searchkeys from a file (option -f file).
Pretend that the output of a command is a file (not really grep, bash helps here) with <(cmmnd).
Combining these will give you
grep -m1 -f <(cut -d"," -f1-2 name.txt) file.txt
Close, but not what you want. The substrings given by cut -d"," -f1-2 name.txt will match everywhere in the line, and you want to match the first two fields. Matching at the start of the line is done with ^, so we use sed to make strings like ^field1,field2 :
grep -m1 -f <(sed 's/\([^,]*,[^,]*,\).*/^\1/' name.txt) file.txt
Given a previously defined $LINE in a shell script, I do the following
var1=$(echo $LINE | cut -d, -f4)
var2=$(echo $LINE | cut -d, -f5)
var3=$(echo $LINE | cut -d, -f6)
Is there any way for me to combine it into one command, where the cut is run only once?
Something like
var1,var2,var3=$(echo $LINE | cut -d, -f4,5,6)
The builtin read command can assign to multiple variables:
IFS=, read _ _ _ var1 var2 var3 _ <<< "$LINE"
yes, if you're ok with arrays:
var= ( $(echo $LINE | cut -d, --output-delimiter=' ' -f4-6) )
Note that that make var 0-indexed.
Though it might just be quicker and easier to turn the CSV $LINE into something that bash parenthesis understand, and then just do var = ( $LINE ).
EDIT: The above will cause issues if you have spaces in your $LINE... if so, you need to be a bit more careful, and AWK might be a better choice to add quotes:
var= ( $( echo $LINE | awk IFS=, '{print "\"$4\" \"$5\" \"$6\""}' ) )
I have the following:
FILENAME=$1
cat $FILENAME | while read LINE
do
response="$LINE" | cut -c1-14
request="$LINE" | cut -c15-31
difference=($response - $request)/1000
echo "$difference"
done
When I run this script it returns blank lines. What am I doing wrong?
Might be simpler in awk:
awk '{print ($1 - $2)/1000}' "$1"
I'm assuming that the first 14 chars and the next 17 chars are the first two blank-separated fields.
You need to change it to:
response=`echo $LINE | cut -c1-14`
request=`echo $LINE | cut -c15-31`
difference=`expr $response - $request`
val=`expr $difference/1000`
You are basically doing everything wrong ;)
This should be better:
FILENAME="$1"
cat "$FILENAME" | while read LINE
do
response=$(echo "$LINE" | cut -c1-14) # or cut -c1-14 <<< "$line"
request=$(echo "$LINE" | cut -c15-31)
difference=$((($response - $request)/1000)
echo "$difference"
done