I searched and couldn't find anything, maybe I can't understand the problem properly.
I have a bash function who read files in current dir and sub dir's, I'm trying to arrange the text and analyze the data but somehow I'm losing lines if I'm using pipeline.
the code:
function recursiveFindReq {
for file in *.request; do
if [[ -f "$file" ]]; then
echo handling "$file"
echo ---------------with pipe-----------------------
cat "$file" | while read -a line; do
if (( ${#line} > 1 )); then
echo ${line[*]}
fi
done
echo ----------------without pipe----------------------
cat "$file"
echo
echo num of lines: `cat "$file" | wc -l`
echo --------------------------------------
fi
done
for dir in ./*; do
if [[ -d "$dir" ]]; then
echo cd to $dir
cd "$dir"
recursiveFindReq "$1"
cd ..
fi
done
}
the output is:
losing lines even when they meet requirements
I marked with 2 red arrows the place I'm losing info
Related
I'm writing code to exchange the upper and lower alphabet of file's name in certain directory.
If directory is unable to access, it shows err message.
#!/bin/bash
if [ $# -eq 0 ];then
path=$(pwd)
for a in "$path"/*
do
mv "$i" "`echo $i | tr '[:upper:][:lower:]' '[:lower:][:upper:]'`"
done
else
if [ ! -d "$1" ];then
echo "Unable to access directory!"
else
for i in "$1"/*
do
mv "$i" "`echo $i | tr '[:upper:][:lower:]' '[:lower:][:upper:]'`"
done
fi
fi
The problem is that, when I echo $i, it doesn't express ONLY filename, but filename with directory!
So, when I try to mv the file, not only file's name is changed but ALSO directory's name is changed so I can't mv the file.
Like this:
mv: cannot move 'test3/Ipad.txt' to 'TEST3/iPAD.TXT': No such file or directory
mv: cannot move 'test3/iPhone' to 'TEST3/IpHONE': No such file or directory
mv: cannot move 'test3/macOS' to 'TEST3/MACos': No such file or directory
how can I change the file names in certain directory?
Any help would be appreciable and thanks in advance.
I'd use a single loop for handling both cases.
#! /bin/sh -
case $# in
( 0 ) path=. ;;
( * ) path=$1
esac
for fpath in "$path"/*; do
echo mv -- "$fpath" "${fpath%/*}/$(
printf '%s\n' "${fpath##*/}" |
tr '[a-zA-Z]' '[A-Za-z]')"
done
If the path given by user is not present or accessible, this'll error out without causing any harm. But if you insist on handling that yourself, add a check before loop, or, use nullglob with bash and the script will exit silently on such an occasion.
Also note that the possibility that files having neither upper nor lower case letters in their names may exist is ignored here. Nothing will happen to them but mv will complain that source and target are the same.
You may want to cd to the directory first, then work on filenames in the directory:
#!/bin/bash
if [ $# -eq 0 ]; then
path=$(pwd)
cd "$path"
for a in *
do
mv "$i" "`echo $i | tr '[:upper:][:lower:]' '[:lower:][:upper:]'`"
done
cd -
else
if [ ! -d "$1" ]; then
echo "Unable to access directory!"
else
cd "$1"
for i in *
do
mv "$i" "`echo $i | tr '[:upper:][:lower:]' '[:lower:][:upper:]'`"
done
cd -
fi
fi
I have a homework using for loop but I'm not quite understand the task that I have to do in there. I wrote a script but I feel like it's not a correct script. Please help!
Here is the question:
Write a shell script to list out the contents of any directory, and indicate for each file (including invisible ones) whether the file is a directory, a plain file, and whether it is public and/or executable to this process
#!/bin/bash
if [ $# -lt 1 ] ; then
echo " file doesn't exist"
echo
echo " variable needed to run a command"
fi
echo ---------------------------------------------
echo ---------------------------------------------
for i in $*
do
if [ -f $i ]; then
echo " it's a file";
echo "THIS IS A LIST OF FILE and DIRECTORY in $i"
ls -a $i
fi
done
echo -----------------------------------------
if [ -d $i ]; then
echo "directory" ;
echo "THIS IS A LIST OF FILES AND DIRETORY in $i"
ls -a $i
fi
echo ------------------------------------------
if [ -x $i ]; then
echo "executable"
echo "THIS IS A LIST OF EXECUTABLE FILE IN $i"
ls -x $i
fi
echo -----------------------------------------
if [ -r $i ]; then
echo "this file is a public file"
else "this is a private file"
fi
#!/bin/bash
if [ $# -lt 1 ] ; then
echo " file doesn't exist"
echo
echo " variable needed to run a command"
fi
echo ---------------------------------------------
echo ---------------------------------------------
for i in $*
do
if [ -f $i ]; then
echo " it's a file";
echo "THIS IS A LIST OF FILE and DIRECTORY in $i"
ls -a $i
fi
done
echo -----------------------------------------
if [ -d $i ]; then
echo "directory" ;
echo "THIS IS A LIST OF FILES AND DIRETORY in $i"
ls -a $i
fi
echo ------------------------------------------
if [ -x $i ]; then
echo "executable"
echo "THIS IS A LIST OF EXECUTABLE FILE IN $i"
ls -x $i
fi
echo -----------------------------------------
if [ -r $i ]; then
echo "this file is a public file"
else "this is a private file"
fi
Poorly written specifications are the bane of education. "Public" sounds like the wrong word here. I'll assume it means "readable".
You check if there's an argument, but you don't exit the program if there is not. I'd also confirm it's a directory, and readable.
The manual will do you a lot of good. Expect to do a lot of reading till you learn this stuff, and then reference it a lot to be sure.
Read this section carefully, create some tests for yourself to prove they work and that you understand them, and your job will be more than half done.
Don't use [. Generally it's just better to always use [[ instead, unless you are using (( or case or some other construct.
I don't see that a for loop was specified, but it ought to be fine. Just be aware that you might have to specify $1/* and $1/.* separately.
Put all your tests in one loop, though. For each file, test for whether it's a directory - if it is, report it. Test if it's a plain file - if it is, report it.
I do NOT like doing homework for someone, but it looks like you could use an example that simplifies this. I recommend you not use this as written - break it out and make it clearer, but this is a template for the general logic.
#! /bin/env bash
(( $# )) && [[ -d "$1" ]] && [[ -r "$1" ]] || {
echo "use: $0 <dir>" >&2
exit 1
}
for e in "$1"/.* "$1"/*
do echo "$e:"
[[ -d "$e" ]] && echo " is a directory"
[[ -f "$e" ]] && echo " is a plain file"
[[ -r "$e" ]] && echo " is readable"
[[ -x "$e" ]] && echo " is executable"
done
If you read the links I provided you should be able to break this apart and understand it.
Generally, your script is long and a bit convoluted. Simpler is easier to understand and maintain. For example, be very careful about block indentation to understand scope.
$: for i in 1 2 3
> do echo $i
> done
1
2
3
$: echo $i
3
Compare this to -
for i in $*
do if [ -f $i ]; then
echo " it's a file";
echo "THIS IS A LIST OF FILE and DIRECTORY in $i"
ls -a $i
fi
done
echo -----------------------------------------
if [ -d $i ]; then
echo "directory" ;
echo "THIS IS A LIST OF FILES AND DIRETORY in $i"
ls -a $i
fi
You are testing each entry to see if it is a file, and if it is, reporting "THIS IS A LIST OF FILE and DIRECTORY in $i" every time...
but then only testing the last one to see if it's a directory, because the [ -d $i ] is after the done.
...did you run this somewhere to try it, and look at the results?
I am writing a shell script to read all the files in the give directory by the user input then count how many files with that extension. I just started learning Bash and I am not sure why it this not locating the files or reading the directory. I am only putting 2 example but my count is always 0.
This is how I run my script
$./check_ext.sh /home/user/temp
my script check_ext.sh
#!/bin/bash
count1=0
count2=0
for file in "ls $1"
do
if [[ $file == *.sh ]]; then
echo "is a txt file"
(( count1++ ))
elif [[ $file == *.mp3 ]]; then
echo "is a mp3 file"
(( count2++ ))
fi
done;
echo $count $count2
"ls $1" doesn't execute ls on $1, it just a plain string. Command substitution syntax is $(ls "$1")
However there is no need to use ls, just use globbing:
count1=0
count2=0
for file in "$1"/*; do
if [[ $file == *.sh ]]; then
echo "is a txt file"
(( count1++ ))
elif [[ $file == *.mp3 ]]; then
echo "is a mp3 file"
(( count2++ ))
fi
done
echo "counts: $count1 $count2"
for file in "$1"/* will iterate through all the files/directories in the directory denoted by $1
EDIT: For doing it recursively inside a directory:
count1=0
count2=0
while IFS= read -r -d '' file; do
if [[ $file == *.sh ]]; then
echo "is a txt file"
(( count1++ ))
elif [[ $file == *.mp3 ]]; then
echo "is a mp3 file"
(( count2++ ))
fi
done < <(find "$1" -type f -print0)
echo "counts: $count1 $count2"
POSIXly:
count1=0
count2=0
for f in "$1"/*; do
case $f in
(*.sh) printf '%s is a txt file\n' "$f"; : "$((count1+=1))" ;;
(*.mp3) printf '%s is a mp3 file\n' "$f"; : "$((count2+=1))" ;;
esac
done
printf 'counts: %d %d\n' "$count1" "$count2"
You can use Bash arrays for this too: if you only want to deal with extensions sh and mp3:
#!/bin/bash
shopt -s nullglob
shs=( "$1"/*.sh )
mp3s=( "$1"/*.mp3 )
printf 'counts: %d %d\n' "${#shs[#]}" "${#mp3s[#]}"
If you want to deal with more extensions, you can generalize this process:
#!/bin/bash
shopt -s nullglob
exts=( .sh .mp3 .gz .txt )
counts=()
for ext in "${exts[#]}"; do
files=( "$1"/*."$ext" )
counts+=( "${#files[#]}" )
done
printf 'counts:'
printf ' %d' "${counts[#]}"
echo
If you want to deal with all extensions (using associative arrays, available in Bash ≥4)
#!/bin/bash
shopt -s nullglob
declare -A exts
for file in "$1"/*.*; do
ext=${file##*.}
((++'exts[$ext]'))
done
for ext in "${!exts[#]}"; do
printf '%s: %d\n' "$ext" "${exts[$ext]}"
done
Sorry for asking this question again. I have already received answer but with using find but unfortunately I need to write it without using any predefined commands.
I am trying to write a script that will loop recursively through the subdirectories in the current directory. It should check the file count in each directory. If file count is greater than 10 it should write all names of these file in file named "BigList" otherwise it should write in file "ShortList". This should look like:
---<directory name>
<filename>
<filename>
<filename>
<filename>
....
---<directory name>
<filename>
<filename>
<filename>
<filename>
....
My script only works if subdirectories don't include subdirectories in turn.
I am confused about this because it doesn't work as I expect.
Here is my script
#!/bin/bash
parent_dir=""
if [ -d "$1" ]; then
path=$1;
else
path=$(pwd)
fi
parent_dir=$path
loop_folder_recurse() {
local files_list=""
local cnt=0
for i in "$1"/*;do
if [ -d "$i" ];then
echo "dir: $i"
parent_dir=$i
echo before recursion
loop_folder_recurse "$i"
echo after recursion
if [ $cnt -ge 10 ]; then
echo -e "---"$parent_dir >> BigList
echo -e $file_list >> BigList
else
echo -e "---"$parent_dir >> ShortList
echo -e $file_list >> ShortList
fi
elif [ -f "$i" ]; then
echo file $i
if [ $cur_fol != $main_pwd ]; then
file_list+=$i'\n'
cnt=$((cnt + 1))
fi
fi
done
}
echo "Base path: $path"
loop_folder_recurse $path
How can I modify my script to produce the desired output?
This bash script produces the output that you want:
#!/bin/bash
bigfile="$PWD/BigList"
shortfile="$PWD/ShortList"
shopt -s nullglob
loop_folder_recurse() {
(
[[ -n "$1" ]] && cd "$1"
for i in */; do
[[ -d "$i" ]] && loop_folder_recurse "$i"
count=0
files=''
for j in *; do
if [[ -f "$j" ]]; then
files+="$j"$'\n'
((++count))
fi
done
if ((count > 10)); then
outfile="$bigfile"
else
outfile="$shortfile"
fi
echo "$i" >> "$outfile"
echo "$files" >> "$outfile"
done
)
}
loop_folder_recurse
Explanation
shopt -s nullglob is used so that when a directory is empty, the loop will not run. The body of the function is within ( ) so that it runs within a subshell. This is for convenience, as it means that the function returns to the previous directory when the subshell exits.
Hopefully the rest of the script is fairly self-explanatory but if not, please let me know and I will be happy to provide additional explanation.
I have a text file (ListOfAllFiles.txt) that has a list of 500 files some of which exist and some don't.
I'd like to make two texts files that indicate which files exist and which don't.
This is my code thus far:
#!/bin/bash
for f in $(cat /path/to/ListOfAllFiles.txt)
do
if [[ -f $f ]]; then
echo $f > /path/to/FilesFound.txt
else
echo $f > /path/to/FilesNOTFound.txt
fi
done
What am I doing wrong??
Your biggest problem is that each pass through the loop will overwrite either /path/to/FilesFound.txt or /path/to/FilesNOTFound.txt; instead of using >, you should be using >>. Fixing that, and making other improvements for robustness, we get:
#!/bin/bash
echo -n > /path/to/FilesFound.txt # reset to empty file
echo -n > /path/to/FilesNOTFound.txt # reset to empty file
while IFS= read -r f ; do
if [[ -f "$f" ]]; then
echo "$f" >> /path/to/FilesFound.txt
else
echo "$f" >> /path/to/FilesNOTFound.txt
fi
done < /path/to/ListOfAllFiles.txt