for loop: commands start from begin every time - bash

Question

I have write the following bash script to check list of domains from domain.list and multiple directories from dir.list.
# is the first domain, it first tries to find file at
http://example.com
if success script finish and exit no problem.
if failed it go to check it at
https://example.com
if ok , script finish and exit,
if not
check for it at
http://example.com/$list of different directories.
If file found script finished and exit , if failed to find
then go to check it at
https://example.com/$list of different directories
But the problem , when the first check failed and second check failed , it goes to third check , but it keep looping , at third command and 4th command, tell it find file or list of directories finished.
I want the script when reach the 3rd command to run it and check for it at list of directories tell the list finish and not to go for the 4th command tell it finished
As at my script it keep checking for single domain at multiple directories and every time to check a new directory it start the whole script from the bagain and run the 1st command and 2nd command again from the begin and I do not need that, big loss of time
Thanks
#!/bin/bash
dirs=(`cat dir.list`)
doms=( `cat domain.list`)
for dom in "${doms[#]}"
do
for dir in "${dirs[#]}"
do
target1="http://${dom}"
target2="https://${dom}"
target3="http://${dom}/${dir}"
target4="https://${dom}/${dir}"
if curl -s --insecure -m2 ${target1}/test.txt | grep "success" > /dev/null ;then
echo ${target1} >> dir.result
break
elif curl -s --insecure -m2 ${target2}/test.txt | grep "success" > /dev/null;then
echo ${target2} >> dir.result
break
elif curl -s --insecure -m2 ${target3}/test.txt | grep "success" > /dev/null; then
echo ${target3} >> dir.result
break
elif curl -s --insecure -m2 ${target4}/test.txt | grep "success" > /dev/null ; then
echo ${target4} >> dir.result
break
fi
done
done

Answer 1

Your code is sub-optimal; if you have a list of 5 'dir' values, you check 5 times whether http://${domain}/test.txt exists — but the chances are that if it didn't exist the first time, it doesn't exist on the other times either.
You use dir to indicate a sub-directory name, but your code uses http://${dom}:${dir} rather than the more normal http://${dom}/${dir}. Technically, what follows the colon up to the first slash is a port number, not a directory. I'm going to assume this is a typo and the colon should be replaced by a slash.
Generally, do not use the back-tick notation; use $(…) instead. Avoid swathes of repeated code, too.
I think you can compress your script down to something like this:
#!/bin/bash
dirs=( $(cat dir.list) )
file=test.txt
fetch_file()
{
if curl -s --insecure -m2 "${1:?}/${file}" | grep "success" > /dev/null
then
echo "${1}"
return 0
else
return 1
fi
}
for dom in $(cat domain.list)
do
for proto in http https
do
fetch_file "${proto}://{$dom}" && break
for dir in "${dirs[#]}"
do
fetch_file "${proto}://${dom}/${dir}" && break 2
done
done
done > dir.result
If the domain list is massive, you could consider using while read dom; do …; done < domain.list instead of using the $(cat domain.list). It would be feasible, and possibly even sensible, to define variable site="${proto}://${dom}" and then use that in the invocations of fetch_file.

Answer 2

You can use this script:
while read dom; do
while read dir; do
target1="http://${dom}"
target2="https://${dom}"
target3="http://${dom}:${dir}"
target4="https://${dom}:${dir}"
if curl -s --insecure -m2 ${target1}/test.txt | grep -q "success"; then
echo ${target1} >> dir.result
break 2
elif curl -s --insecure -m2 ${target2}/test.txt | grep -q "success"; then
echo ${target2} >> dir.result
break 2
elif curl -s --insecure -m2 ${target3}/test.txt | grep -q "success"; then
echo ${target3} >> dir.result
break 2
elif curl -s --insecure -m2 ${target4}/test.txt | grep -q "success"; then
echo ${target4} >> dir.result
break 2
fi
done < dir.list
done < domain.list