The following is a little bit of my code:
for((a=1;a<=8000000;a++))
do
if (($a%4==0))
then
b=`cat 101127_2_aa_1.fastq|head -$a|tail -1|sed 's/\(.\)B*$/\1/g'|wc -c`
echo `cat 101127_2_aa_1.fastq|head -$(($a-3))|tail -1`
echo `cat 101127_2_aa_1.fastq|head -$(($a-2))|tail -1|cut -c 1-$(($b-1))`
echo `cat 101127_2_aa_1.fastq|head -$(($a-1))|tail -1`
echo `cat 101127_2_aa_1.fastq|head -$a|tail -1|sed 's/\(.\)B*$/\1/g'`
fi
done
This if loop is "echo" the output; however I hope to save the echo output into some file. And I wanna manage this WITHIN script.
I mean probably sth. like:
`for((a=1;a<=8000000;a++))
do
if (($a%4==0))
then
b=`cat 101127_2_aa_1.fastq|head -$a|tail -1|sed 's/\(.\)B*$/\1/g'|wc -c`
echo `cat 101127_2_aa_1.fastq|head -$(($a-3))|tail -1`
echo `cat 101127_2_aa_1.fastq|head -$(($a-2))|tail -1|cut -c 1-$(($b-1))`
echo `cat 101127_2_aa_1.fastq|head -$(($a-1))|tail -1`
echo `cat 101127_2_aa_1.fastq|head -$a|tail -1|sed 's/\(.\)B*$/\1/g'`
fi
done` > output
But obviously this doesn't work; and I'm asking for the right way to save echo output within script.
thx
Always a good practice to use $(…) instead of ``.
for((a=1;a<=8000000;a++))
do
if (($a%4==0))
then
b=$(cat 101127_2_aa_1.fastq|head -$a|tail -1|sed 's/\(.\)B*$/\1/g'|wc -c)
echo $(cat 101127_2_aa_1.fastq|head -$(($a-3))|tail -1)
echo $(cat 101127_2_aa_1.fastq|head -$(($a-2))|tail -1|cut -c 1-$(($b-1)))
echo $(cat 101127_2_aa_1.fastq|head -$(($a-1))|tail -1)
echo $(cat 101127_2_aa_1.fastq|head -$a|tail -1|sed 's/\(.\)B*$/\1/g')
fi
done >> output
Update:
As #Sorpigal suggested, this still uses cat command when we can do the same with head
You can append >> output to each of the echo lines.
Also the echo is redundant in your script. Instead of
echo `cat ...`
you can use
cat ...
Just remove backticks and keep redirect.
Alternatively you can
exec >output
before the for.
Related
I am trying to make my bash script ssh into each server and then grep Selinux=enforcing/replace with Selinux=permissive. The issue I am facing is it checks the first server and but not the second server. I believe it arises from my if statement.
#!/bin/bash
selinux_path=/opt/configtest
hosts=(server1 server2)
for my_hosts in "${hosts[#]}"
do
ssh -q -o "StrictHostKeyChecking no" root#${my_hosts} "
if [ $(grep -c SELINUX=enforcing $selinux_path) -ne 0 ]
then
echo "------------------------------------------------"
echo "${my_hosts}"
echo "------------------------------------------------"
sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' ${selinux_path}
echo "Selinux has been changed to permissive"
cat ${selinux_path}
else
echo "------------------------------------------------"
echo "${my_hosts}"
echo "------------------------------------------------"
echo "Selinux has already been changed to permissive"
cat ${selinux_path}
fi
"
done
You can't nest " inside ". If you want to give multiline input to ssh, the easiest way is with a here-doc.
#!/bin/bash
selinux_path=/opt/configtest
hosts=(server1 server2)
for my_hosts in "${hosts[#]}"
do
ssh -q -o "StrictHostKeyChecking no" root#${my_hosts} <<EOF
if grep -q SELINUX=enforcing "$selinux_path"
then
echo "------------------------------------------------"
echo "${my_hosts}"
echo "------------------------------------------------"
sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' ${selinux_path}
echo "Selinux has been changed to permissive"
cat "${selinux_path}"
else
echo "------------------------------------------------"
echo "${my_hosts}"
echo "------------------------------------------------"
echo "Selinux has already been changed to permissive"
cat "${selinux_path}"
fi
EOF
done
Did you try to specify the full path for grep, echo, sed and cat?
i have a code:
L12(){
echo -e "/tftpboot/log/archive/L12/*/*$sn*L12*.log /tftpboot/log/diag/*$sn*L12*.log"
command="| grep -v hdd"
}
getlog(){
echo $(ls -ltr $(${1}) 2>/dev/null `${command}` | tail -1)
}
however $command does not seem to be inserting | grep -v hdd correctly
i need $command to be either empty or | grep
is there a simple solution to my issue or should i go for different approach
edit:
there may be another problem in there
i am loading a few "modules"
EVAL.sh
ev(){
case "${1}" in
*FAIL*) paint $red "FAIL";;
*PASS*) paint $green "PASS";;
*)echo;;
esac
result=${1}
}
rackinfo.sh (the "main script")
#! /bin/bash
#set -x
n=0
for src in $(ls modules/)
do
source modules/$src && ((n++))
## debugging
# source src/$src || ((n++)) || echo "there may be an issue in $src"
done
## debugging
# x=($n - $(ls | grep src | wc -l))
# echo -e "$x plugin(s) failed to laod correctly"
# echo -e "loaded $n modules"
########################################################################
command=cat
tests=("L12" "AL" "BI" "L12-3")
while read sn
do
paint $blue "$sn\t"
for test in ${tests[#]}
do
log="$(ev "$(getlog ${test})")"
if [[ -z ${log} ]]
then
paint $cyan "${test} "; paint $red "!LOG "
else
paint $cyan "${test} ";echo -ne "$log "
fi
done
echo
done <$1
the results i get are still containing "hdd" for L12()
Set command to cat as a default.
Also, it's best to use an array for commands with arguments, in case any of the arguments is multiple words.
There's rarely a reason to write echo $(command). That's essentially the same as just writing command.
#default command does nothing
command=(cat)
L12(){
echo -e "/tftpboot/log/archive/L12/*/*$sn*L12*.log /tftpboot/log/diag/*$sn*L12*.log"
command=(grep -v hdd)
}
getlog(){
ls -ltr $(${1}) 2>/dev/null | "${command[#]}" | tail -1)
}
Complete novice in Bash. Trying to iterate thru 1000 gzip files, may be GNU parallel is the solution??
#!/bin/bash
ctr=0
echo "file_name,symbol,record_count" > $1
dir="/data/myfolder"
for f in "$dir"/*.gz; do
gunzip -c $f | while read line;
do
str=`echo $line | cut -d"|" -f1`
if [ "$str" == "H" ]; then
if [ $ctr -gt 0 ]; then
echo "$f,$sym,$ctr" >> $1
fi
ctr=0
sym=`echo $line | cut -d"|" -f3`
echo $sym
else
ctr=$((ctr+1))
fi
done
done
Any help to speed the process will be greatly appreciated !!!
#!/bin/bash
ctr=0
export ctr
echo "file_name,symbol,record_count" > $1
dir="/data/myfolder"
export dir
doit() {
f="$1"
gunzip -c $f | while read line;
do
str=`echo $line | cut -d"|" -f1`
if [ "$str" == "H" ]; then
if [ $ctr -gt 0 ]; then
echo "$f,$sym,$ctr"
fi
ctr=0
sym=`echo $line | cut -d"|" -f3`
echo $sym >&2
else
ctr=$((ctr+1))
fi
done
}
export -f doit
parallel doit ::: *gz 2>&1 > $1
The Bash while read loop is probably your main bottleneck here. Calling multiple external processes for simple field splitting will exacerbate the problem. Briefly,
while IFS="|" read -r first second third rest; do ...
leverages the shell's built-in field splitting functionality, but you probably want to convert the whole thing to a simple Awk script anyway.
echo "file_name,symbol,record_count" > "$1"
for f in "/data/myfolder"/*.gz; do
gunzip -c "$f" |
awk -F "\|" -v f="$f" -v OFS="," '
/H/ { if(ctr) print f, sym, ctr
ctr=0; sym=$3;
print sym >"/dev/stderr"
next }
{ ++ctr }'
done >>"$1"
This vaguely assumes that printing the lone sym is just for diagnostics. It should hopefully not be hard to see how this can be refactored if this is an incorrect assumption.
I have this:
echo "all done creating tables" >> ${SUMAN_DEBUG_LOG_PATH}
but that should only append to the file, not write to stdout.
How can I write to stdout and append to a file in the same bash line?
Something like this?
echo "all done creating tables" | tee -a "${SUMAN_DEBUG_LOG_PATH}"
Use the tee command
$ echo hi | tee -a foo.txt
hi
$ cat foo.txt
hi
Normally tee is used, however a version using just bash:
#!/bin/bash
function mytee (){
fn=$1
shift
IFS= read -r LINE
printf '%s\n' "$LINE"
printf '%s\n' "$LINE" >> "$fn"
}
SUMAN_DEBUG_LOG_PATH=/tmp/abc
echo "all done creating tables" | mytee "${SUMAN_DEBUG_LOG_PATH}"
I have a list that looks like this:
sharename:shareX
comment:commentX
sharename:shareY
comment:commentY
sharename:shareZ
comment:commentZ
and so on...
And this is how I would like the list to look like:
shareX;commentX
shareY;commentY
shareZ;commentZ
How can I accomplish that in bash?
Pure Bash:
IFS=':'
while read a b; read c d; do # read 2 lines
echo -e "$b:$d"
done < "$infile"
one liner:
odd=0; for i in `cat list | cut -d":" -f2`; do if [ $odd -eq 0 ]; then echo -ne $i; odd=1; else echo $i; odd=0; fi; done
formatted:
odd=0;
for i in `cat list | cut -d":" -f2`;
do
if [ $odd -eq 0 ];
then
echo -ne $i";";
odd=1;
else
echo $i;
odd=0;
fi;
done
untested, the sed part may be wrong
paste -d ';' - - < filename | sed -r 's/(^|;)[^:]:/\1/g'
This might work for you:
sed '$!N;s/[^:]*:\([^\n]*\)\n[^:]*:/\1;/' file
shareX;commentX
shareY;commentY
shareZ;commentZ