I'm really struggling to see why this while-loop never ends, when the loop starts, my variable LOC is set to Testing/, which is a directory I created to test this program, it has the following layout:
I want the loop to end once all Directories have had the "count" function applied to them.
Here are the things I have tried;
I've checked my count function, and it doesn't produce an infinite loop
I've tried running through the algorithm by hand
PARSE=1
LOC=$LOC/
count
AVAILABLEDIR=$(ls $LOC -AFl | sed "1 d" | grep "/$" | awk '{ print $9 }')
while [ $PARSE = "1" ]
do
if [[ ${AVAILABLEDIR[#]} == '' ]]; then
PARSE=0
fi
DIRBASE=$LOC
for a in ${AVAILABLEDIR[#]}; do
LOC="${DIRBASE}${a}"
LOCLIST="$LOCLIST $LOC"
count
done
for a in ${LOCLIST[#]}; do
TMPAVAILABLEDIR=$(ls $a -AFl | sed "1 d" | grep "/$" | awk '{ print $9 }')
PREPEND=$a
if [[ ${TMPAVAILABLEDIR[#]} == '' ]]; then
continue
fi
for a in ${TMPAVAILABLEDIR[#]}; do
TMPAVAILABLEDIR2="$TMPAVAILABLEDIR2 ${PREPEND[#]}${a}"
done
NEWAVAILABLEDIR="$NEWAVAILABLEDIR $TMPAVAILABLEDIR2"
done
AVAILABLEDIR=$NEWAVAILABLEDIR
NEWAVAILABLEDIR=''
LOC=''
done
I am really struggling, and any input would be greatly appreciated, I've been trying to figure this out for the last couple of hours.
You should try to run the script with argument -x, or write it into the first line:
#!/bin/bash -x
Then it tells you everything it does.
In that case, you might notice two errors:
You never reset TMPAVAILABLEDIR2
You do ls on regular files as well.
If you really must avoid recursion, try this. It completely recursion-free:
#!/bin/bash
count() {
echo counting "$1"
}
todo=(Testing)
while test ${#todo[#]} != 0
do
doit=("${todo[#]}")
todo=()
for dir in "${doit[#]}"
do
for entry in "$dir"/* # If the directory is empty, this shows an entry named "*"
do
test -e "$entry" || continue # Skip the entry "*" of an empty directory
count "$entry"
test -d "$entry" || continue
todo+=("$entry")
done
done
done
You wrote you want to perform "count" on all directories.
Look at the options of find:
find $LOC -type d | while read dir; do
cd $LOC
cd ${dir}
count
done
Or shorter (when your function count accepts a directory as parameter 1):
find $LOC -type d | xargs count
I now see you do not want to use find or ls -R (recursive function). Then you should make your own recursive function, something like
function parseDir {
ls -d */ $1 | while read dir; do
count
parseDir $1/$dir
done
}
I have no idea if this will work, but it’s an interesting question I couldn't stop thinking about.
while true ; do
for word in "$(echo *)" ; do
if [[ -d "$word" ]] ; then
d[$((i++))]="$PWD"/"$word"
elif [[ -f "$word" ]] ;then
f[$((j++))]="$PWD"/"$word"
fi
done
[[ $k -gt $i ]] && cd ..
cd "$d[$((k++))]" || break
done
Related
I am looping through a folder and depending on the length of files do certain condition. I seem not to come right with that. I evaluate and output the length of a string in the terminal.
echo $file|wc -c gives me the answer of all files in the terminal.
But incorporating this into a loop is impossible
for file in `*.zip`; do
if [[ echo $file|wc -c ==9]]; then
some commands
where I want to operate on files that have a length of nine characters
Try this one:
for file in *.zip ; do
wcout=$(wc -c "$file")
if [[ ${wcout%% *} -eq 9 ]] ; then
# some commands
fi
done
The %% operator in variable expansion deletes everything that match the pattern after it. This is glob pattern, not regular expression.
Opposite to natural good sense of typical programmers the == operator in BASH compares strings, not numbers.
Alternatively (following the comment) you can:
for file in *.zip ; do
wcout=$(wc -c < "$file")
if [[ ${wcout} -eq 9 ]] ; then
# some commands
fi
done
Additional observation is that if BASH cannot expand *.zip as there is no ZIP files in the current directory it will pass "*.zip" into $file and let single iteration of the loop. That leads to the error reported by wc command. So it would be recommended to add:
if [[ -e ${file} ]] ; then ...
as a prevention mechanism.
Comments leads to another form of this solution (plus I added my safety mechanism):
for file in *.zip ; do
if [[ -e "$file" && (( $(wc -c < "$file") == 9 )) ]] ; then
# some commands
fi
done
using filter outside the loop
ls -1 *.zip \
| grep -E '^.{9}$' \
| while read FileName
do
# Your action
done
using filter inside loop
ls -1 *.zip \
| while read FileName
do
if [ ${#FileName} -eq 9 ]
then
# Your action
fi
done
alternative to ls -1 that is always a bit dangereous, find . -name '*.zip' -print [ but you neet to add 2 char length or filter the name form headin ./ and maybe limit to current folder depth ]
I have to write a shell script that creates a file containing the name of each text files from a folder (given as parameter) and it's subfolders that contain words longer than n characters (read n from keyboard).
I wrote the following code so far :
#!/bin/bash
Verifies if the first given parameter is a folder:
if [ ! -d $1 ]
then echo $1 is not a directory\!
exit 1
fi
Reading n
echo -n "Give the number n: "
read n
echo "You entered: $n"
Destination where to write the name of the files:
destinatie="destinatie"
the actual part that i think it makes me problems:
nr=0;
#while read line;
#do
for fisier in `find $1 -type f`
do
counter=0
for word in $(<$fisier);
do
file=`basename "$fisier"`
length=`expr length $word`
echo "$length"
if [ $length -gt $n ];
then counter=$(($counter+1))
fi
done
if [ $counter -gt $nr ];
then echo "$file" >> $destinatie
fi
done
break
done
exit
The script works but it does a few more steps that i don't need.It seems like it reads some files more than 1 time. If anyone can help me please?
Does this help?
egrep -lr "\w{$n,}" $1/* >$destinatie
Some explanation:
\w means: a character that words consist of
{$n,} means: number of consecutive characters is at least $n
Option -l lists files and does not print the grepped text and -r performs a recursive scan on your directory in $1
Edit:
a bit more complete version around the egrep command:
#!/bin/bash
die() { echo "$#" 1>&2 ; exit 1; }
[ -z "$1" ] && die "which directory to scan?"
dir="$1"
[ -d "$dir" ] || die "$dir isn't a directory"
echo -n "Give the number n: "
read n
echo "You entered: $n"
[ $n -le 0 ] && die "the number should be > 0"
destinatie="destinatie"
egrep -lr "\w{$n,}" "$dir"/* | while read f; do basename "$f"; done >$destinatie
This code has syntax errors, probably leftovers from your commented-out while loop: It would be best to remove the last 3 lines: done causes the error, break and exit are unnecessary as there is nothing to break out from and the program always terminates at its end.
The program appears to output files multiple times because you just append to $destinatie. You could simply delete that file when you start:
rm "$destinatie"
You echo the numbers to stdout (echo "$length") and the file names to $destinatie (echo "$file" >> $destinatie). I do not know if that is intentional.
I found the problem.The problem was the directory in which i was searching.Because i worked on the files from the direcotry and modified them , it seems that there remained some files which were not displayed in file explorer but the script would find them.i created another directory and i gived it as parameter and it works. Thank you for your answers
.
I currently have this code:
listing=$(find "$PWD")
fullnames=""
while read listing;
do
if [ -f "$listing" ]
then
path=`echo "$listing" | awk -F/ '{print $(NF)}'`
fullnames="$fullnames $path"
echo $fullnames
fi
done
For some reason, this script isn't working, and I think it has something to do with the way that I'm writing the while loop / declaring listing. Basically, the code is supposed to pull out the actual names of the files, i.e. blah.txt, from the find $PWD.
read listing does not read a value from the string listing; it sets the value of listing with a line read from standard input. Try this:
# Ignoring the possibility of file names that contain newlines
while read; do
[[ -f $REPLY ]] || continue
path=${REPLY##*/}
fullnames+=( $path )
echo "${fullnames[#]}"
done < <( find "$PWD" )
With bash 4 or later, you can simplify this with
shopt -s globstar
for f in **/*; do
[[ -f $f ]] || continue
path+=( "$f" )
done
fullnames=${paths[#]##*/}
I'm trying to write a small script that will count entries in a log file, and I'm incrementing a variable (USCOUNTER) which I'm trying to use after the loop is done.
But at that moment USCOUNTER looks to be 0 instead of the actual value. Any idea what I'm doing wrong? Thanks!
FILE=$1
tail -n10 mylog > $FILE
USCOUNTER=0
cat $FILE | while read line; do
country=$(echo "$line" | cut -d' ' -f1)
if [ "US" = "$country" ]; then
USCOUNTER=`expr $USCOUNTER + 1`
echo "US counter $USCOUNTER"
fi
done
echo "final $USCOUNTER"
It outputs:
US counter 1
US counter 2
US counter 3
..
final 0
You are using USCOUNTER in a subshell, that's why the variable is not showing in the main shell.
Instead of cat FILE | while ..., do just a while ... done < $FILE. This way, you avoid the common problem of I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?:
while read country _; do
if [ "US" = "$country" ]; then
USCOUNTER=$(expr $USCOUNTER + 1)
echo "US counter $USCOUNTER"
fi
done < "$FILE"
Note I also replaced the `` expression with a $().
I also replaced while read line; do country=$(echo "$line" | cut -d' ' -f1) with while read country _. This allows you to say while read var1 var2 ... varN where var1 contains the first word in the line, $var2 and so on, until $varN containing the remaining content.
Always use -r with read.
There is no need to use cut, you can stick with pure bash solutions.
In this case passing read a 2nd var (_) to catch the additional "fields"
Prefer [[ ]] over [ ].
Use arithmetic expressions.
Do not forget to quote variables! Link includes other pitfalls as well
while read -r country _; do
if [[ $country = 'US' ]]; then
((USCOUNTER++))
echo "US counter $USCOUNTER"
fi
done < "$FILE"
minimalist
counter=0
((counter++))
echo $counter
You're getting final 0 because your while loop is being executed in a sub (shell) process and any changes made there are not reflected in the current (parent) shell.
Correct script:
while read -r country _; do
if [ "US" = "$country" ]; then
((USCOUNTER++))
echo "US counter $USCOUNTER"
fi
done < "$FILE"
I had the same $count variable in a while loop getting lost issue.
#fedorqui's answer (and a few others) are accurate answers to the actual question: the sub-shell is indeed the problem.
But it lead me to another issue: I wasn't piping a file content... but the output of a series of pipes & greps...
my erroring sample code:
count=0
cat /etc/hosts | head | while read line; do
((count++))
echo $count $line
done
echo $count
and my fix thanks to the help of this thread and the process substitution:
count=0
while IFS= read -r line; do
((count++))
echo "$count $line"
done < <(cat /etc/hosts | head)
echo "$count"
USCOUNTER=$(grep -c "^US " "$FILE")
Incrementing a variable can be done like that:
_my_counter=$[$_my_counter + 1]
Counting the number of occurrence of a pattern in a column can be done with grep
grep -cE "^([^ ]* ){2}US"
-c count
([^ ]* ) To detect a colonne
{2} the colonne number
US your pattern
Using the following 1 line command for changing many files name in linux using phrase specificity:
find -type f -name '*.jpg' | rename 's/holiday/honeymoon/'
For all files with the extension ".jpg", if they contain the string "holiday", replace it with "honeymoon". For instance, this command would rename the file "ourholiday001.jpg" to "ourhoneymoon001.jpg".
This example also illustrates how to use the find command to send a list of files (-type f) with the extension .jpg (-name '*.jpg') to rename via a pipe (|). rename then reads its file list from standard input.
I am trying to look for a certain file in multiple folders. When I hit the file, I want to stop going into subdirectories.
For example:
/foo/.target
/bar/buz/.target
/foo/bar/.target
I want only the first two:
/foo/.target
/bar/buz/.target
Your requirements are not completely clear. I understand them as: look for “wanted” files inside a directory tree; if a directory directly contains at least one match, then just print them, otherwise recurse into that directory.
I can't think of a pure find solution. You could write an awk or perl script to parse the output of find.
Here's a shell script that I think does what you're looking for. Warning: I've only minimally tested it.
#!/bin/sh
## Return 0 if $1 is a matching file, 1 otherwise.
## Note that $1 is the full path to the file.
wanted () {
case ${1##*/} in
.target) true;;
esac
}
## Recurse into the directory $1. Print all wanted files in this directory.
## If there is no wanted file, recurse into each subdirectory in turn.
traverse () {
found=0
for x in "$1"/.* "$1"/*; do
if [ "$x" = "$1/." ] || [ "$x" = "$1/.." ]; then
continue # skip '.' and '..' entries
fi
if ! [ -e "$x" ]; then
continue # skip spurious '.*', '*' from non-matching patterns
fi
if wanted "$x"; then
printf '%s\n' "$x"
found=$(($found+1))
fi
done
if [ $found -eq 0 ]; then # no match here, so recurse
for x in "$1"/.*/ "$1"/*/; do
x=${x%/}
if [ "$x" = "$1/." ] || [ "$x" = "$1/.." ]; then
continue
fi
if [ -d "$x" ]; then # only actual subdirs, not symlinks or '.*' or '*'
found_stack=$found:$found_stack # no lexical scoping in sh
traverse "${x%/}"
found=${found_stack%%:*}
found_stack=${found_stack#*:}
fi
done
fi
}
found_stack=:
for x; do
if wanted "$x"; then
printf '%s\n' "$x"
else
traverse "$x"
fi
done
Use sed or perhaps awk or anything that could break the pipe once it reads a line from input. It may stop find's execution quickly or at least soon enough.
find ... | sed 1q
find ... | awk '1; { exit }"
It would just show a single line.
For the first two:
find ... | sed 2q
find ... | awk '1; NR == 2 { exit }"