I've been trying to run this code in a server but always send an 'Eqw' status.
I want to process a file that contains three columns, one with a path, and the others are information about that file.
Can anyone please help me to figure out what's wrong in my code?
#!/bin/bash
source $HOME/.bashrc
source /share/apps/Profiles/share-profile.sh
module load compilers/python-2.7.15
MANIFEST=path with the file with three columns
TRIM=trimming program that does work properly
#Run pyadapter_trim.py to trim adapter sequences
COUNTER=0;
#for each file in the input manifest, run tirimming program
while read LINE
do
READ=`echo "$LINE" | awk -F"\t" '{print $1}'`
SAMPLE=`echo "$LINE" | awk -F"\t" '{print $2}'`
GENOME=`echo "$LINE" | awk -F"\t" '{print $3}'`
WDIR=`(dirname $READ1)`
cd $WDIR
QCDIR=$WDIR"/ats"
mkdir -p $QCDIR
echo -e "Processing sample ${SAMPLE}"
echo -e "${READ1}..."
python $TRIM -a $READ
echo ""
COUNTER=`expr $COUNTER + 1`
done < "$MANIFEST"
echo "$COUNTER Files Processed"
Related
apacherelease=$(curl -s "https://httpd.apache.org" | grep Released | awk '{print $4}' | perl -p -e 's/2.4.54/2.4.54-1/g') &&
apacheinstallversion=$(dnf list installed | grep httpd.x86_64|awk '{print $2}') &&
echo $apacherelease
echo $apacheinstallversion
if test "$apacheinstallversion" = "$apacherelease"; then
: variables are the same
else
: variables are different
fi
`
If I run the commands to set variable directly from the command line instead of a script the variables stick however in the script they disappear the moment I move to the if statement.
Any input would extremely help!
Corrected version:
apacherelease=$(curl -s "https://httpd.apache.org" | grep Released | awk '{print $4}' | perl -p -e 's/2.4.54/2.4.54-1/g') &&
apacheinstallversion=$(dnf list installed | grep httpd.x86_64 | awk '{print $2}')
echo "$apacherelease"
echo "$apacheinstallversion"
if [[ $apacheinstallversion == $apacherelease ]]; then
echo "variables are the same"
else
echo "variables are different" >&2
fi
use the full featured bash test [[
use == instead of =
Assume an input table (intable.csv) that contains ID numbers in its second column, and a fresh output table (outlist.csv) into which the input file - extended by one column - is to be written line by line.
echo -ne "foo,NC_045043\nbar,NC_045193\nbaz,n.a.\nqux,NC_045054\n" > intable.csv
echo -n "" > outtable.csv
Further assume that one or more third-party commands (here: esearch, efetch; both part of Entrez Direct) are employed to retrieve additional information for each ID number. This additional info is to form the third column of the output table.
while IFS="" read -r line || [[ -n "$line" ]]
do
echo -n "$line" >> outtable.csv
NCNUM=$(echo "$line" | awk -F"," '{print $2}')
if [[ $NCNUM == NC_* ]]
then
echo "$NCNUM"
RECORD=$(esearch -db nucleotide -query "$NCNUM" | efetch -format gb)
echo "$RECORD" | grep "^LOCUS" | awk '{print ","$3}' | \
tr -d "\n" >> outtable.csv
else
echo ",n.a." >> outtable.csv
fi
done < intable.csv
Why does the while loop iterate only over the first input table entry under the above code, whereas it iterates over all input table entries if the code lines starting with RECORD and echo "$RECORD" are commented out? How can I correct this behavior?
This would happen if esearch reads from standard input. It will inherit the input redirection from the while loop, so it will consume the rest of the input file.
The solution is to redirect is standard input elsewhere, e.g. /dev/null.
while IFS="" read -r line || [[ -n "$line" ]]
do
echo -n "$line" >> outtable.csv
NCNUM=$(echo "$line" | awk -F"," '{print $2}')
if [[ $NCNUM == NC_* ]]
then
echo "$NCNUM"
RECORD=$(esearch -db nucleotide -query "$NCNUM" </dev/null | efetch -format gb)
echo "$RECORD" | grep "^LOCUS" | awk '{print ","$3}' | \
tr -d "\n" >> outtable.csv
else
echo ",n.a." >> outtable.csv
fi
done < intable.csv
I tried with below script. But, not working for cut the first field of each line and to be executed for "chmod".
#!/bin/bash
if [ -z "$1" ]; then
echo -e "Usage: $(basename $0) FILE\n"
exit 1
fi
if [ ! -e "$1" ]; then
echo -e "$1: File doesn't exist.\n"
exit 1
fi
while read -r line; do
awk '{print $1}'
[ -n "$line" ] && chown root "$line" && echo -e "$line Ownership changed"
done < "$1"
If field separator is space, try this:
while read -r line; do
FILE_TO_CHANGE=$(echo $line | awk '{print $1}')
[ -n "$line" ] && chown root "$FILE_TO_CHANGE" && echo -e "$line Ownership changed"
done < "$1"
awk read $line and print first token on standard output, the result is saved in FILE_TO_CHANGE variable and then it is used to run chown.
Another way could be:
awk '{print $1}' $1 | while read line; do
chown root "$line" && echo -e "$line Ownership changed"
done
awk read your file and print the first field of each line, in this case, while loop read awk output line by line and run chown on field.
You could extract the first word on each line with awk and pipe to xargs, invoking chown only as few times as possible:
awk '{print $1}' "$1" | xargs chown root
I have a problem
I Have a program general like this gene.sh
that for all file (es file: geneX.csv) make a directory with the name of gene (example: Genex/geneX.csv) next this program compile an other program inside gene.sh but this progrm need a varieble and I dont know how do it.
this is the program gene.sh
#!/bin/bash
# Create a dictory for each file *.xls and *.csv
for fname in *.xlsx *csv
do
dname=${fname%.*}
[[ -d $dname ]] || mkdir "$dname"
mv "$fname" "$dname"
done
# For each gene go inside the directory and compile the programs getChromosomicPositions.sh to have the positions, and getHapolotipeStings.sh to have the variants
for geni in */; do
cd $geni
z=$(tail -n 1 *.csv | tr ';' "\n" | wc -l)
cd ..
cp getChromosomicPositions.sh $geni --->
cp getHaplotypeStrings.sh $geni
cd $geni
export z
./getChromosomicPositions.sh *.csv
export z
./getHaplotypeStrings.sh *.csv
cd ..
done
This is the program getChromosomichPositions.sh:
rm chrPosRs.txt
grep '^Haplotype\ ID' $1 | cut -d ";" -f 4-61 | tr ";" "\n" | awk '{print "select chrom,chromStart,chromEnd,name from snp147 where name=\""$1"\";"}' > listOfQuery.txt
while read l; do
echo $l > query.txt
mysql -h genome-mysql.cse.ucsc.edu -u genome -A -D hg38 --skip-column-names < query.txt > queryResult.txt
if [[ "$(cat queryResult.txt)" == "" ]];
then
cat query.txt |
while read line; do
echo $line | awk '$6 ~/rs/ {print $6}' > temp.txt;
if [[ "$(cat temp.txt)" != "" ]];
then cat temp.txt | awk -F'name="' '{print $2}' | sed -e 's/";//g' > temp.txt;
./getHGSVposHG19.sh temp.txt ---> Hear the problem--->
else
echo $line | awk '{num=sub(/.*:g\./,"");num+=sub(/\".*/,"");if(num==2){print};num=""}' > temp2.txt
fi
done
cat query.txt >> varianti.txt
echo "Missing Data" >> chrPosRs.txt
else
cat queryResult.txt >> chrPosRs.txt
fi
done < listOfQuery.txt
rm query*
hear the problem:
I need to enter in the file temp.txt and put automatically at the beginning of the file the variable $geni of the program gene.sh
How can I do that?
Why not pass "$geni" as say the first argument when invoking your script, and treating the rest of the arguments as your expected .csv files.
./getChromosomicPositions.sh "$geni" *.csv
Alternatively, you can set it as environment variable for the script, so that it can be used there (or just export it).
geni="$geni" ./getChromosomicPositions.sh *.csv
In any case, once you have it available in the second script, you can do
if passed as the first argument:
echo "${1}:$(cat temp.txt | awk -F'name="' '{print $2}' | sed -e 's/";//g')
or if passed as environment variable:
echo "${geni}:$(cat temp.txt | awk -F'name="' '{print $2}' | sed -e 's/";//g')
I have the following code
for ip in $(ifconfig | awk -F ":" '/inet addr/{split($2,a," ");print a[1]}')
do
bytesin=0; bytesout=0;
while read line
do
if [[ $(echo ${line} | awk '{print $1}') == ${ip} ]]
then
increment=$(echo ${line} | awk '{print $4}')
bytesout=$((${bytesout} + ${increment}))
else
increment=$(echo ${line} | awk '{print $4}')
bytesin=$((${bytesin} + ${increment}))
fi
done < <(pmacct -s | grep ${ip})
echo "${ip} ${bytesin} ${bytesout}" >> /tmp/bwacct.txt
done
Which I would like to print the incremented values to bwacct.txt, but instead the file is full of zeroes:
91.227.223.66 0 0
91.227.221.126 0 0
127.0.0.1 0 0
My understanding of Bash is that a redirected for loop should preserve variables. What am I doing wrong?
First of all, simplify your script! Usually there are many better ways in bash. Also most of the time you can rely on pure bash solutions instead of running awk or other tools.
Then add some debbuging!
Here is a bit refactored script with debugging
#!/bin/bash
for ip in "$(ifconfig | grep -oP 'inet addr:\K[0-9.]+')"
do
bytesin=0
bytesout=0
while read -r line
do
read -r subIp _ _ increment _ <<< "$line"
if [[ $subIp == "$ip" ]]
then
((bytesout+=increment))
else
((bytesin+=increment))
fi
# some debugging
echo "line: $line"
echo "subIp: $subIp"
echo "bytesin: $bytesin"
echo "bytesout: $bytesout"
done <<< "$(pmacct -s | grep "$ip")"
echo "$ip $bytesin $bytesout" >> /tmp/bwacct.txt
done
Much clearer now, huh? :)