Creating file with Bash scripting for loop and if test - bash

Problem Statement
Find files using bash script
In this section, you are going to write a script named findJane.sh within the scripts directory.
This script should catch all "jane" lines and store them in another text file called oldFiles.txt. You will complete the script using the command we practiced in earlier sections. Don't worry, we'll guide you throughout the whole process.
Navigate to /scripts directory and create a new file named findJane.sh.
Mycode
#!/bin/bash
>oldFiles.txt
files=$(grep " jane " ../data/list.txt | cut -d ' ' -f 3)
for i in $files:do
do if test -e ~/data/"$i"; then
echo "$i" >> OldFiles.txt;
else
echo "File doesn't exist"; fi
done
output now
file does not exist
file does not exist
file does not exist
it should not print nothing
and cat oldFiles.txt should return all those files with name 'jane
Where i am coding wrong
Guide
Create the text file oldFiles.txt and make sure it's empty. This oldFiles.txt file should save files with username "jane".
Now, search for all lines that contain the name "jane" and save the file names into a variable. Let's call this variable files, we will refer to it with that name later in the lab.
Since none of the files present in the file list.txt are available in the file system, check if file names present in files variable are actually present in the file system. To do this, we'll use the test command that we practiced in the previous section.
Now, iterate over the files variable and add a test expression within the loop. If the item within the files variable passes the test, add/append it to the file oldFiles.txt.

I have the same problem, just don't call the files by $in grep
#!/bin/bash
files= grep ' jane ' ../data/list.txt | cut -d ' ' -f 3
for file in files; do
if test -e ~/data/$file; then
echo $file >> oldFiles.txt;
else
echo "File dosen't exsist"; fi
done
And make the new fie .oldFiles.txt in the command line
like
user#linux: ~/program$ ./findJane.sh > oldFiles.txt
The code shall work

If you're trying to do a Qwicklabs assignment, try this
#!/bin/bash
> oldFiles.txt
files=$(grep " jane " /home/<Student-id>/data/list.txt | cut -d ' ' -f 3)
for i in $files;do
if test -e ~/$i;then
echo $i>>oldFiles.txt;
else echo "not working"; fi
done

#!/bin/bash
>oldFiles.txt
files=$(grep " jane " ../data/list.txt | cut -d ' ' -f 3)
for i in $files; do
if test -e ~/$i; then
echo $i >> OldFiles.txt;
else
echo "File doesn't exist"; fi
done

you have just some typos. this is "working" code for your example
#!/bin/bash
>oldFiles.txt
files=$(grep " jane " ../data/list.txt | cut -d ' ' -f 3)
for i in $files; do
if test -e ~/data/"$i"; then
echo "$i" >> OldFiles.txt;
else
echo "File doesn't exist"; fi
done
however, this works only if list.txt doesn't contain any spaces except as delimiters between (at least) three columns
it won't work for Jane because it doesn't match jane
it won't work when there are more spaces like jane example.doc
it won't work for file names containing spaces like best movies.txt
it makes less sense to show how to do it right because the concept of list.txt is bad choice

#!/bin/bash
>oldFiles.txt
files=$(grep " jane " ../data/list.txt | cut -d ' ' -f 3)
for file in $files; do
if [ -e $HOME$file ] ; then
echo $HOME$file >> oldFiles.txt;
fi
done

#!/bin/bash
> oldFiles.txt
files=$(grep ' jane ' ../data/list.txt | cut -d ' ' -f 3)
#echo $files
for file in $files; do
if test -e ~/$file; then
echo "File path added !!"
echo $file >> oldFiles.txt;
fi
done

You need to check the path and then append it to the oldFile
#1/bin/bash
>oldFiles.txt
files=$(grep "jane " ../data/list.txt | cut -d' ' -f3)
for f in $files; do
if [ -e $HOME$f ];then
echo $HOME$f >> oldFiles.txt;
fi
done

I made a few fixes to your code and now works 😉
#!/bin/bash
> oldFiles.txt
files=$(grep " jane " ../data/list.txt | cut -d " " -f 3)
for file in $files; do
if test -e ~/$file; then echo $file >> oldFiles.txt; else echo "File doesn't exist"; fi done

Related

How to use if statements in bash scripting

I am trying to filter lines in a file using a for loop followed by an if statement with the following code:
#!/bin/bash
> oldFiles.txt
files=$(grep ' jane ' ~/data/list.txt | cut -d ' ' -f 3)
for file in $files; do
file="~${file}";
if [ -e "$file" ]; then
echo $file;
fi
done
Where list.txt is a list of names like this:
001 jane /data/jane_profile_07272018.doc
002 kwood /data/kwood_profile_04022017.doc
003 pchow /data/pchow_profile_05152019.doc
004 janez /data/janez_profile_11042019.doc
005 jane /data/jane_pic_07282018.jpg
006 kwood /data/kwood_pic_04032017.jpg
007 pchow /data/pchow_pic_05162019.jpg
008 jane /data/jane_contact_07292018.csv
009 kwood /data/kwood_contact_04042017.csv
010 pchow /data/pchow_contact_05172019.csv
I need to append the result in the file oldFiles.txt but i'm echo-ing it for now, yet it doesn't show anything on screen.
Everything looks perfect, for example when i replace "$file" in the if statement with an actual value of a file that exists (like this one /data/jane_profile_07272018.doc) then it works. Here's a list of files that exists in the file system:
jane_contact_07292018.csv
janez_profile_11042019.doc
kwood_profile_04022017.doc
pchow_pic_05162019.jpg
jane_profile_07272018.doc
kwood_pic_04032017.jpg
list.txt
test.txt
My script should echo only files that exist in the filesystem, hence the if statement.
What is there missing !?
You were almost there!!!
I would change a little bit your code to
#!/bin/bash
OLDFILES="odlfiles.txt"
FILES=$(egrep " jane " /data/list.txt | cut -d ' ' -f 3 )
#Alternative using awk with space as default delimiter to get the 3rd column ;)
#FILES=$(egrep " jane " /data/list.txt | awk '{print $3}' )
for file in $FILES; do
if [ -f $file ]; then
echo -e "[EXISTS]\tAppending to ${OLDFILES} path file: ${file}"
#Appending now to target file
echo $file >> $OLDFILES
fi
done
I think you are aware of the use of grep in order of filtering the content of the list.txt. I would recommend you to use in such cases egrep and then the pattern/regex which suits your needs the most.
In this case:
#!/bin/bash
OLDFILES="odlfiles.txt"
WORD2MATCH="jane"
FILES=$(egrep "\s${WORD2MATCH}\s" list.txt | awk '{print $3}')
for file in $FILES; do
if [ -f $file ]; then
echo -e "[EXISTS]\tAppending to ${OLDFILES} path file: ${file}"
echo $file >> $OLDFILES
fi
done
Or maybe in the compact form :)
#!/bin/bash
OLDFILES="odlfiles.txt"
WORD2MATCH="jane"
FILES=$(egrep "\s${WORD2MATCH}\s" list.txt | awk '{print $3}')
for file in $FILES; do
[ -f $file ] && echo $file >> $OLDFILES && echo -e "[APPENDED]\t${file}"
done

Use awk to print largest alphanumeric grep result and create variable next in sequence

I have a source file with the following information in it.
WABEL8499IPM101
WABEL8499IPM102
WABEL8499IPM103
WABEL8499IPM104
WABEL8499IPM105
WABEL8499IPM106
WABEL8499IPM107
WABEL8499IPM108
I need to be able to find the largest name in the sequence and then create a new variable with the next logical name in the sequence. I need to be able to create multiple if necessary. For example:
Use grep to search the file for WABEL8499IPM which shows all of the above results. I need to find WABEL8499IPM108 because it's the largest in the sequence and then create a new variable (how many depends on what the user inputs) with the value WABEL8499IPM109. If user inputs a quantity of 2 then I need both 109 and 110. My goal is to build a bash script to input the base name (without the last 3 digits), find the largest in the sequence and then output to a log file the next names in the sequence however many times the user needs.
I'm not really sure where to start. I can find all using grep but having difficulty finding only the largest value/sequence. The user will only input the base name because they won't know the last 3 digits. Currently I don't have any code that works.
SRCFILE="~/Desktop/deviceinfo.csv"
LOGDIR="~/Desktop/"
LOGFILE="$LOGDIR/DeviceNames.csv"
echo -e "\n"
echo "What is the base device name?"
read deviceName
echo "How many device names do you need?"
read quantityName
lines=$(grep -c "$deviceName" $SRCFILE)
echo -e "\n"
echo "There are $lines results."
deviceResults=$(grep -F "$deviceName" $SRCFILE)
echo -e "\n"
echo Device Name\'s Currently Enrolled:
echo "$deviceResults"
echo -e "\n"
echo "Your output file has been created."
CODE FOR CREATING OUTPUT FILE HERE
echo "$deviceName1" >> "$LOGFILE"
echo "$deviceName2" >> "$LOGFILE"
echo "$deviceName3" >> "$LOGFILE"
Would there be a way with this method to use a reference file for the input? For example if I had to research and create multiple names with different quantities could we use an input reference file for that so we don't have to type them each individually and run the script multiple times?
SRCFILE="~/Desktop/deviceinfo.csv"
LOGDIR="~/Desktop/"
LOGFILE="$LOGDIR/DeviceNames.csv"
# base name, such as "WABEL8499IPM"
device_name=$1
# quantity, such as "2"
quantityNum=$2
# the largest in sequence, such as "WABEL8499IPM108"
max_sequence_name=$(cat $SRCFILE | grep -o -e "$device_name[0-9]*" | sort --reverse | head -n 1)
# extract the last 3digit number (such as "108") from max_sequence_name
max_sequence_num=$(echo $max_sequence_name | rev | cut -c 1-3 | rev)
# creat a sequence of files starting from "WABEL8499IPM101" if there is not any "WABEL8499IPM".
if [ -z "$max_sequence_name" ];
then
max_sequence_name=device_name
max_sequence_num=100
fi
# create new sequence_name
# such as ["WABEL8499IPM109", "WABEL8499IPM110"]
array_new_sequence_name=()
for i in $(seq 1 $quantityNum);
do
cnum=$((max_sequence_num + i))
array_new_sequence_name+=($(echo $device_name$cnum))
done
#CODE FOR CREATING OUTPUT FILE HERE
#for fn in ${array_new_sequence_name[#]}; do touch $fn; done;
# write log
for sqn in ${array_new_sequence_name[#]};
do
echo $sqn >> $LOGFILE
done
Usage:
bash test.sh WABEL8499IPM 2
Result in the log file:
WABEL8499IPM109
WABEL8499IPM110
EDITED
The input reference file (input.txt) :
WABEL8499IPM,2
WABEL8555IPM,6
WABEL8444IPM,5
The driver shell script :
INPFIL="./input.txt"
PSRC="./test.sh"
cat $INPFIL | while read line;
do
device_name=`echo $line | cut -d "," -f 1`
quantity_num=`echo $line | cut -d "," -f 2`
bash $PSRC $device_name $quantity_num
done;
You can try
logdir="~/Desktop/"
srcfile="$logdir/deviceinfo.csv"
logfile="$logdir/DeviceNames.csv"
echo
read -p "What is the base device name? " deviceName
echo
read -p "How many device names do you need? " quantityName
echo
awk -v name="$deviceName" \
-v q="$quantityName" \
-v lelog="$logfile" '
$0 ~ "^"name {
sub(name,"")
a=a>$0?a:$0
}
END {
if ( a )
for ( i = 1 ; i <= q ; i++ )
print name ( a + i ) >> lelog
}
' "$srcfile"

compare 2 values within a while read loop, one from stdout and one from file

I have a while read line loop where i am assigning $user and $quota twhich are coming from stdout but i have also a file from another server where i am checking if $user is existing in this file, which is also containing the $user's quota.
I need to compare the $quota value which i am getting in stdout with the value within the file.
example:
zmsoap -z GetQuotaUsageRequest -v -u https://$sourceserver:7071/service/admin/soap/ |awk {'print $4" "$5'} | cut -d'"' -f2,4 | sed -r 's/["]+/ /g' | while read line
do {
user1=`echo $line|cut -f1 -d " "`
quota1=`echo $line|cut -f2 -d " "`
echo "$user1 has $quota1 on $sourceserver"
if grep -q "$user1" "$allremotequotasnew"; then
echo "OK: $user1 is EXISTING on $destserver"
else
echo "ERROR: $user1 NOT FOUND on $destserver" && echo "Creating $user1 on $destserver now..."
fi
}
done
any ideas? thanksa lot in advance
The file looks like: user33 quota and i need to check if $user is existing in the file, and if so, then also cut out it's quota
You can solve this question by the following script clips.
grep "$user1" "$allremotequotasnew" > /dev/null
if [ $? -eq 0 ];then
echo "OK: $user1 is EXISTING on $destserver"
dest_line=`grep "$user1" "$allremotequotasnew"`
dest_user=`echo $dest_line|cut -f1 -d " "`
dest_quota=`echo $dest_line|cut -f2 -d " "`
else
echo "ERROR: $user1 NOT FOUND on $destserver"
fi
dest_user and dest_quota are what you want to get from file.

Shell script print the path of all sub directory

My Script:
cd /var/www/try/
sort -u
files="$(find -L "/var/www/try/" -type d)"
echo "Count: $(echo -n "$files" | wc -l)"
echo "$files" | while read file; do
echo $file >> filename.csv
done
Output :
/var/www/try
/var/www/try/cat
Output Should be :
var-www-try
var-www-try-cat
Second Case :
any character except / as some fo my folder name contain / like for e.g.
"tv/dvd"
Output genrated :
/var/www/try/cat/tv/dvd
Output Should be :
var-www-try-cat-tv/dvd-
Have a look at sed, which is a popular tool for string replacing tasks.
>> x=$(echo '/var/www/try/cat/tv/' | sed 's/\//-/g')
>> echo $x
-var-www-try-cat-tv-
>> x=${x:1}
>> echo $x
var-www-try-cat-tv-
edit:
In response to your second case:
as some fo my folder name contain /
Maybe there's a misunderstanding here but you should not have a / in a filename. See Is it possible to use "/" in a filename?

shell get string

I have some lines have same structure like
1000 AS34_59329 RICwdsRSYHSD11-2-IPAAPEK-93 /ifshk5/BC_IP/PROJECT/T1
1073/T11073_RICekkR/Fq/AS34_59329/111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IP
AAPEK-93_1.fq.gz /ifshk5/BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_5932
9/111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IPAAPEK-93_2.fq.gz /ifshk5/
BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_59329/clean_111220_I631_FCC0E5EACXX_
L4_RICwdsRSYHSD11-2-IPAAPEK-93_1.fq.gz.total.info 11.824 0.981393
43.8283 95.7401 OK
And I want to get the Bold part to check whether in /home/jesse/ has this folder, if not create mkdir /home/jesse/AS34_59329
I use this code
! /bin/bash
myPath="/home/jesse/"
while read myline
do
dirname= echo "$myline" | awk -F ' ' '{print $2}'
echo $dirname
myPath= $myPath$dirname
echo $myPath
mkdir -p "$myPath"
done < T11073_all_3254.fq.list
But it can't mkdir and show the path name, it shows
-bash: /home/jesse/: is a directory
/home/jesse/
AS39_59324
read can read each field into a separate variable, and mkdir -p will create a dir only if it doesn't exist:
path="/home/jesse"
while read _ dir _
do
mkdir -p "$path/$dir"
done < T11073_all_3254.fq.list
for will iterate over each whitespace separated token. Try this instead.
#!/usr/bin/env bash
# Invoke with first arg as file containing the lines
# foo.sh <input_filename>
for i in `cat $1 | cut -d " " -f2`
do
if [ -d /home/jesse/$i ]
then
echo "Directory /home/jesse/$i exists"
else
mkdir /home/jesse/$i;
echo "Directory /home/jesse/$i created"
fi
done

Resources