I am trying to create:
a list containing files present in a but missing in b
a list containing files present in b but missing in a
This is my code so far:
#!/bin/sh
set -e
A_DIR_1=/tmp/1
A_DIR_2=/tmp/2
A_DIR_3=/tmp/3
B_DIR_1=/tmp/1
B_DIR_2=/tmp/2
B_DIR_3=/tmp/3
for i in {1..3}
do
A="A_DIR_$i"
B="B_DIR_$i"
if [ d ${!A} ]; then
bash -c 'diff -u <(find "${!B}" |sed "s:${!B}::") <(find "${!A}" |sed "s:${!A}::") |sed "/^+\//!d; s::${!A}/:"' >> /tmp/fileA
bash -c 'diff -u <(find "${!A}" |sed "s:${!A}::") <(find "${!B}" |sed "s:${!A}::") |sed "/^+\//!d; s::${!B}/:"' >> /tmp/fileB
fi
done
However, when I run it, I get the following error: find: cannot search : No such file or directory'. Why is this? The directories definitely exist.
mm, imho your code is difficult.
If I've understood you clearly, that can help:
Prepare:
mkdir /tmp/dir1
mkdir /tmp/dir2
touch /tmp/dir1/test{1..12}
touch /tmp/dir2/test{1..15}
touch /tmp/dir1/test{22..25}
Code:
#!/bin/bash
dir1=/tmp/dir1
dir2=/tmp/dir2
diff_result=$(diff -u "${dir1}" "${dir2}")
echo "${diff_result}" | grep "${dir1}" | awk '{print $NF}' > /tmp/files_only_in_dir1
echo "${diff_result}" | grep "${dir2}" | awk '{print $NF}' > /tmp/files_only_in_dir2
Output:
$ cat /tmp/files_only_in_dir1
test22
test23
test24
test25
$ cat /tmp/files_only_in_dir2
test13
test14
test15
Related
I run the below script in a reference env and get the correct output in the output file, but when I run the same in production the output file is empty.
I tried debugging the using set -x, and understand that the for loop is not getting executed in the prod env.
please suggest what might be the issue.
#!/bin/ksh
DIR=/some/log/dir/error
DATE=$1
OUTPUT=/some/logs/dir/scripts/output/output.csv
. /calling/env/setupscript.ksh
for file in $(find $DIR 2>/dev/null| grep $ASOF | grep -i something1 | grep -vi someotherthing2 | grep -iv someotherthing3 | grep -vi someotherthing4 | grep -vi someotherthing4 )
do
echo "Checking file $file..."
if [[ -z "$DONE" ]] then
head -1 $file | read line
echo "File name;Filter name;Type;Scenario;$line" > $OUTPUT.tmp
DONE=1
fi
fullfile=$file
file=$(basename $file)
file=${file#SPR_RPT_}
file=${file#SPR_BY__}
echo $file | awk -F_ '{ print $(NF-2)" "$(NF-1) }' | read type scen
filter=${file%%_$type\_*}
grep ERROR $fullfile | while read line
do
echo "$(basename $fullfile);$filter;$type;$scen;$line" >> ${OUTPUT}.tmp
done
done
i'm trying to write a code which will print all files taking more than min_size (lets say 10G) in a directory. the problem is output off the below code is all files irrespective of the min_size. i will be getting other details like mtime , owner as well later in the code but this part itself doesnt work fine, whats wrong here ?
#!/bin/sh
if (( $# <3 )); then
echo "$0 dirname min_size count"
exit 1
else
dirname="$1";
min_size="$2";
count="$3";
#shift 3
fi
tmpfile=$(mktemp /lawdump/pulkit/files.XXXXXX)
exec 3> "$tmpfile"
find "${dirname}" -type f -print0 2>&1 | grep -v "Permission denied" | xargs -0 -I {} echo "{}" > "$tmpfile"
for i in `cat tmpfile`
do
x="`du -ah $i | awk '{print $1}' | grep G | sort -nr -k 1`"
size=$(echo $x | sed 's/[A-Za-z]*//g')
if [ size > $min_size ];then
echo $size
fi
done
Note : i know this can be done through find or du but i need to write a shell script to have an email sent out regularly with all the details.
I have a problem
I Have a program general like this gene.sh
that for all file (es file: geneX.csv) make a directory with the name of gene (example: Genex/geneX.csv) next this program compile an other program inside gene.sh but this progrm need a varieble and I dont know how do it.
this is the program gene.sh
#!/bin/bash
# Create a dictory for each file *.xls and *.csv
for fname in *.xlsx *csv
do
dname=${fname%.*}
[[ -d $dname ]] || mkdir "$dname"
mv "$fname" "$dname"
done
# For each gene go inside the directory and compile the programs getChromosomicPositions.sh to have the positions, and getHapolotipeStings.sh to have the variants
for geni in */; do
cd $geni
z=$(tail -n 1 *.csv | tr ';' "\n" | wc -l)
cd ..
cp getChromosomicPositions.sh $geni --->
cp getHaplotypeStrings.sh $geni
cd $geni
export z
./getChromosomicPositions.sh *.csv
export z
./getHaplotypeStrings.sh *.csv
cd ..
done
This is the program getChromosomichPositions.sh:
rm chrPosRs.txt
grep '^Haplotype\ ID' $1 | cut -d ";" -f 4-61 | tr ";" "\n" | awk '{print "select chrom,chromStart,chromEnd,name from snp147 where name=\""$1"\";"}' > listOfQuery.txt
while read l; do
echo $l > query.txt
mysql -h genome-mysql.cse.ucsc.edu -u genome -A -D hg38 --skip-column-names < query.txt > queryResult.txt
if [[ "$(cat queryResult.txt)" == "" ]];
then
cat query.txt |
while read line; do
echo $line | awk '$6 ~/rs/ {print $6}' > temp.txt;
if [[ "$(cat temp.txt)" != "" ]];
then cat temp.txt | awk -F'name="' '{print $2}' | sed -e 's/";//g' > temp.txt;
./getHGSVposHG19.sh temp.txt ---> Hear the problem--->
else
echo $line | awk '{num=sub(/.*:g\./,"");num+=sub(/\".*/,"");if(num==2){print};num=""}' > temp2.txt
fi
done
cat query.txt >> varianti.txt
echo "Missing Data" >> chrPosRs.txt
else
cat queryResult.txt >> chrPosRs.txt
fi
done < listOfQuery.txt
rm query*
hear the problem:
I need to enter in the file temp.txt and put automatically at the beginning of the file the variable $geni of the program gene.sh
How can I do that?
Why not pass "$geni" as say the first argument when invoking your script, and treating the rest of the arguments as your expected .csv files.
./getChromosomicPositions.sh "$geni" *.csv
Alternatively, you can set it as environment variable for the script, so that it can be used there (or just export it).
geni="$geni" ./getChromosomicPositions.sh *.csv
In any case, once you have it available in the second script, you can do
if passed as the first argument:
echo "${1}:$(cat temp.txt | awk -F'name="' '{print $2}' | sed -e 's/";//g')
or if passed as environment variable:
echo "${geni}:$(cat temp.txt | awk -F'name="' '{print $2}' | sed -e 's/";//g')
I would like to change the file name from Sub****_Ses1 to HU_TT_12_****_UU;
(**** numbered from 0001 to 1600)
I did the below
#!/bin/sh
#Change file name
Subj_id=/Users/dave/biomark/dat
cd Subj_id
for abcd in Sub****_Ses1; do
mv Sub$a$b$c$d_Ses1 HU_TT_12_$a$b$c$d_UU;
done
for and wildcards don't work like this. Use cut to extract the number.
$ touch Sub000{1,2,3,4}_Ses1
$ for f in Sub????_Ses1
do
abcd=$(echo $f | cut -b4-7)
mv $f HU_TT_12_${abcd}_UU
done
$ ls HU_TT_12_000*
HU_TT_12_0001_UU HU_TT_12_0002_UU HU_TT_12_0003_UU HU_TT_12_0004_UU
You can use sed and mv
#!/bin/bash
set -x
Subj_id=/Users/dave/biomark/dat
cd $Subj_id
for i in Sub*_Ses1 ; do
#echo $i|sed -r 's/^.*\([[:digit:]]{4}\).*/HU_TT_12_\1_UU/'
mv $i $(echo $i|sed -rn 's/^.*([[:digit:]]{4}).*/HU_TT_12_\1_UU/ p')
done
I wrote the following script:
for filename in `find . -name '*'.cpp | grep $IN_REGEX | grep -v $OUT_REGEX`
do
echo "Output file is $OUTPUT_FILE"
count=`git log --pretty=format: --name-only $filename | grep -v ^$ | wc -l`
echo "$count $filename" >> $OUTPUT_FILE
done
But nothing gets written into the output file.
Please note:
I have set the values for OUTPUT_FILE, IN_REGEX and OUT_REGEX.
The code inside the loop is being executed. I checked this with an sh -x invokation.
When I remove the >> $OUTPUT_FILE I get the output.
I tried a touch $OUTPUT_FILE inside the script and that is working fine.
Can someone please point out what is my mistake here?
This line of code
for filename in `find . -name '*'.cpp
is a
(break on space in file names)
You should instead do :
while IFS= read -r file; do
echo "Output file is $OUTPUT_FILE"
count=$(git log --pretty=format: --name-only "$file" | grep -v '^$' | wc -l)
echo "$count $file" >> "$OUTPUT_FILE"
done < <(find . -name '*.cpp' | grep "$IN_REGEX" | grep -v "$OUT_REGEX")
For this to work, ensure that $OUTPUT_FILE have a path in it.