I run the below script in a reference env and get the correct output in the output file, but when I run the same in production the output file is empty.
I tried debugging the using set -x, and understand that the for loop is not getting executed in the prod env.
please suggest what might be the issue.
#!/bin/ksh
DIR=/some/log/dir/error
DATE=$1
OUTPUT=/some/logs/dir/scripts/output/output.csv
. /calling/env/setupscript.ksh
for file in $(find $DIR 2>/dev/null| grep $ASOF | grep -i something1 | grep -vi someotherthing2 | grep -iv someotherthing3 | grep -vi someotherthing4 | grep -vi someotherthing4 )
do
echo "Checking file $file..."
if [[ -z "$DONE" ]] then
head -1 $file | read line
echo "File name;Filter name;Type;Scenario;$line" > $OUTPUT.tmp
DONE=1
fi
fullfile=$file
file=$(basename $file)
file=${file#SPR_RPT_}
file=${file#SPR_BY__}
echo $file | awk -F_ '{ print $(NF-2)" "$(NF-1) }' | read type scen
filter=${file%%_$type\_*}
grep ERROR $fullfile | while read line
do
echo "$(basename $fullfile);$filter;$type;$scen;$line" >> ${OUTPUT}.tmp
done
done
Related
i'm trying to write a code which will print all files taking more than min_size (lets say 10G) in a directory. the problem is output off the below code is all files irrespective of the min_size. i will be getting other details like mtime , owner as well later in the code but this part itself doesnt work fine, whats wrong here ?
#!/bin/sh
if (( $# <3 )); then
echo "$0 dirname min_size count"
exit 1
else
dirname="$1";
min_size="$2";
count="$3";
#shift 3
fi
tmpfile=$(mktemp /lawdump/pulkit/files.XXXXXX)
exec 3> "$tmpfile"
find "${dirname}" -type f -print0 2>&1 | grep -v "Permission denied" | xargs -0 -I {} echo "{}" > "$tmpfile"
for i in `cat tmpfile`
do
x="`du -ah $i | awk '{print $1}' | grep G | sort -nr -k 1`"
size=$(echo $x | sed 's/[A-Za-z]*//g')
if [ size > $min_size ];then
echo $size
fi
done
Note : i know this can be done through find or du but i need to write a shell script to have an email sent out regularly with all the details.
I have a problem
I Have a program general like this gene.sh
that for all file (es file: geneX.csv) make a directory with the name of gene (example: Genex/geneX.csv) next this program compile an other program inside gene.sh but this progrm need a varieble and I dont know how do it.
this is the program gene.sh
#!/bin/bash
# Create a dictory for each file *.xls and *.csv
for fname in *.xlsx *csv
do
dname=${fname%.*}
[[ -d $dname ]] || mkdir "$dname"
mv "$fname" "$dname"
done
# For each gene go inside the directory and compile the programs getChromosomicPositions.sh to have the positions, and getHapolotipeStings.sh to have the variants
for geni in */; do
cd $geni
z=$(tail -n 1 *.csv | tr ';' "\n" | wc -l)
cd ..
cp getChromosomicPositions.sh $geni --->
cp getHaplotypeStrings.sh $geni
cd $geni
export z
./getChromosomicPositions.sh *.csv
export z
./getHaplotypeStrings.sh *.csv
cd ..
done
This is the program getChromosomichPositions.sh:
rm chrPosRs.txt
grep '^Haplotype\ ID' $1 | cut -d ";" -f 4-61 | tr ";" "\n" | awk '{print "select chrom,chromStart,chromEnd,name from snp147 where name=\""$1"\";"}' > listOfQuery.txt
while read l; do
echo $l > query.txt
mysql -h genome-mysql.cse.ucsc.edu -u genome -A -D hg38 --skip-column-names < query.txt > queryResult.txt
if [[ "$(cat queryResult.txt)" == "" ]];
then
cat query.txt |
while read line; do
echo $line | awk '$6 ~/rs/ {print $6}' > temp.txt;
if [[ "$(cat temp.txt)" != "" ]];
then cat temp.txt | awk -F'name="' '{print $2}' | sed -e 's/";//g' > temp.txt;
./getHGSVposHG19.sh temp.txt ---> Hear the problem--->
else
echo $line | awk '{num=sub(/.*:g\./,"");num+=sub(/\".*/,"");if(num==2){print};num=""}' > temp2.txt
fi
done
cat query.txt >> varianti.txt
echo "Missing Data" >> chrPosRs.txt
else
cat queryResult.txt >> chrPosRs.txt
fi
done < listOfQuery.txt
rm query*
hear the problem:
I need to enter in the file temp.txt and put automatically at the beginning of the file the variable $geni of the program gene.sh
How can I do that?
Why not pass "$geni" as say the first argument when invoking your script, and treating the rest of the arguments as your expected .csv files.
./getChromosomicPositions.sh "$geni" *.csv
Alternatively, you can set it as environment variable for the script, so that it can be used there (or just export it).
geni="$geni" ./getChromosomicPositions.sh *.csv
In any case, once you have it available in the second script, you can do
if passed as the first argument:
echo "${1}:$(cat temp.txt | awk -F'name="' '{print $2}' | sed -e 's/";//g')
or if passed as environment variable:
echo "${geni}:$(cat temp.txt | awk -F'name="' '{print $2}' | sed -e 's/";//g')
I wrote the following script:
for filename in `find . -name '*'.cpp | grep $IN_REGEX | grep -v $OUT_REGEX`
do
echo "Output file is $OUTPUT_FILE"
count=`git log --pretty=format: --name-only $filename | grep -v ^$ | wc -l`
echo "$count $filename" >> $OUTPUT_FILE
done
But nothing gets written into the output file.
Please note:
I have set the values for OUTPUT_FILE, IN_REGEX and OUT_REGEX.
The code inside the loop is being executed. I checked this with an sh -x invokation.
When I remove the >> $OUTPUT_FILE I get the output.
I tried a touch $OUTPUT_FILE inside the script and that is working fine.
Can someone please point out what is my mistake here?
This line of code
for filename in `find . -name '*'.cpp
is a
(break on space in file names)
You should instead do :
while IFS= read -r file; do
echo "Output file is $OUTPUT_FILE"
count=$(git log --pretty=format: --name-only "$file" | grep -v '^$' | wc -l)
echo "$count $file" >> "$OUTPUT_FILE"
done < <(find . -name '*.cpp' | grep "$IN_REGEX" | grep -v "$OUT_REGEX")
For this to work, ensure that $OUTPUT_FILE have a path in it.
This is my bash script:
#!/usr/local/bin/bash -x
touch /usr/local/p
touch /usr/local/rec
DATA_FULL=`date +%Y.%m.%d.%H`
CHECK=`netstat -an | grep ESTAB | egrep '(13001|13002|13003|13004|13061|13099|16001|16002|16003|16004|16061|16099|18001|18002|18003|18004|18061|18099|20001|20002|20003|20004|20061|20099|13000|16000|18000|20000)' | awk '{ print $5 }' | sort -u | wc -l`
netstat -an | grep ESTAB | egrep '(13001|13002|13003|13004|13061|13099|16001|16002|16003|16004|16061|16099|18001|18002|18003|18004|18061|18099|20001|20002|20003|20004|20061|20099|13000|16000|18000|20000)' | awk '{ print $5 }' | sort -u | wc -l > /usr/local/www/p
STAT=`cat /usr/local/www/rec`
if [ "$CHECK" -gt "$STAT" ]; then
echo $CHECK"\n"$DATA_FULL > /usr/local/p
fi
Ofcourse I've runned chmod +x script.sh and then sh script.sh, then I receive the following message: [: : bad number.
Why does it happends?
Run your script using
sh -x script.sh
It'll print every line it executes and the variable output.
Run the netstat command and stat command outside and check.
If these are integer for sure, use this syntax,
if [ "0$(echo $CHECK|tr -d ' ')" -gt "0$(echo $STAT|tr -d ' ')" ];
A simple hack. Only works if $STAT is always either empty or positive number.
Are you sure that both STAT and CHECK are numbers that can be compared with -gt?
probably your /usr/local/www/rec is empty. Try
STAT=`cat /usr/local/www/rec 2>/dev/null || echo 0`
maybe.
I have a folder with a bunch of files, the files only have a url in it i.e
http://itunes.apple.com/us/app/keynote/id361285480?mt=8
Here is my code. How can I get it to do this for each url in each file?
var='{"object":"App","action":"scrape","args":{"itunes_url":"!!!!HERE!!!!"}}'
string=$(echo "$var" | sed -e 's/"/\\"/g')
string='{"request":"'"$string"'"}'
api="http://api.lewis.com"
output=$(curl -s -d "request=$string" "$api")
code=$(echo "$output" | tr '{', '\n' | sed -n "2p" | sed -e 's/:/ /' | awk '{print $2}')
if [ "${code:0:1}" -ne "2" ]; then
# :(
echo "Error: response code $code was returned, "
else
string=$(echo "$output" | tr '{', '\n' | sed -e '/"signature":\(.*\)/d;/"data":\(.*\)/d;/"signature":\(.*\)/d;/"code":\(.*\)/d' |sed -e 's/\\"//g;s/\\\\\\\//\//g;s/\\//g' | tr '}', '\n' | sed -e 's/"//' | sed '/^$/d')
echo "$string"
fi
use a for loop
for filename in folder/*; do
-- your code where you do something using $filename --
done
og if you prefer to give the filenames as arguments to the script then:
for filename do
-- your code where you do something using $filename --
done
then run your script followed by the files
./script.sh folder/*
You could do:
for file in *; do
for line in $(cat $file); do
# Stuff goes here
done
done
Or even just:
for line in $(cat *); do
# Stuff goes here
done