I am trying to build a bash script capable of comparing two directories given as arguments $1 and $2, and changing the files' timestamps from the second directory ( if they are not different than a given timestamp $3 ) to be the same as the files with the same name in the first directory. I'm doing okay with that, but I don't see how to access the folders inside the given directories, and compare the files inside those folders.
For example, if I have Directory1 and Directory2 given as arguments:
Directory1 contains:
-text.txt
-folder1/secondfile.txt
-folder2/thirdfile.txt
and Directory2 contains:
-text.txt
-folder1/secondfile.txt
-folder3/thirdfile.txt
so in this case I want my script to modify the files text.txt and secondfile.txt, but not the thirdfile.txt because the relative paths are not the same. How would my script access folders in the directory and how would it compare relative paths? I have managed to do what I wanted with files from the directory, but not folders, and I have no idea how to compare relative paths, even though I searched around I couldn't find it.
So far I've done this script (with help from SO):
#!/bin/bash
cd "$1"
function check {
for i in *; do
if [[-d "$i"]]; then
cd "$i"
check
cd -
fi
if [[-f "$i"]]; then
if [[stat %y "$i" == "$3"]]; then
#if [[path check]];then
#touch -r "$i" "$2/path/$i"
fi
}
and I don't know how to do the [[path check]] which should check if both files have the same relative path (relative to the directories given as arguments).
EDIT:
As the answer suggests, is the following code the right way to do it?
#!/bin/bash
cd "$1"
shopt -s globstar
for i in **; do
if [[stat %y "$i" == "$3"]]; then
if [["$1/$i" == "$2/$i"]];then
touch -r "$i" "$2/$i"
fi
There was an answer here before, which I wanted to reply to, suggesting using shopt -s globstar and ** instead of *.
The additional question was something along the lines of "Would I be able to compare the relative paths?".
Yes, you would. With shopt -s globstar, ** expands to include the relative path to each file. So it would return text.txt folder1/secondfile.txt folder2/thirdfile.txt.
EDIT:
You should not need to cd "$1" either, for "$1" and "$2" would not exist inside dir "$1". Try something along the lines of:
#!/usr/bin/env bash
shopt -s globstar
for i in $(cd "$1"; echo **); do
if [[ $(stat -c %y "$1/$i") == "$3" ]]; then
if [[ -f "$1/$i" ]] && [[ -f "$2/$i" ]]; then
touch -r "$1/$i" "$2/$i"
fi
fi
done
Related
I want my script to delete all zero length files in the directory that is shown as an argument as well as all of the zero length files in the sub directory.
I wrote a script that can delete all of the zero length files in the current directory. I need it to be able to enter sub directories and delete those zero length files too.
What can I do to solve this problem?
DIR=$1
TRAV= touch -c "$DIR"/*
for d in ./$(DIR)/*; do
if [ -f "${d}" ] && [ -s "${d}" ]; then
echo "$d has some data"
else
echo "$d has no data, we're deleting it."
rm -d -R ${d}
fi
done
~ ~ ~
The program works perfectly fine so far, it just doesn't delete the zero length files in the sub directories of the specified directory.
add shopt -s globstar to the script to enable globstar, change line four
from
for d in ./$(DIR)/*; do
to
for d in ./$(DIR)/**/*; do
then run the script with your directory as an argument
Why don't you use "find"?
If you don't use "find", you can do by using a recursive call and ls command instead.
#!/bin/bash
function del(){
local dir=$1
cd ${dir}
for f in $(ls -1)
do
if [ -d "${f}" ]; then
(del ${f})
else
if [ ! -s ${f} ]; then
rm -f ${f}
fi
fi
done
}
del .
I would want to iterate over contents of a directory and list only ordinary files.
The path of the directory is given as an user input. The script works if the input is current directory but not with others.
I am aware that this can be done using ls.. but i need to use a for .. in control structure.
#!/bin/bash
echo "Enter the path:"
read path
contents=$(ls $path)
for content in $contents
do
if [ -f $content ];
then
echo $content
fi
done
ls is only returning the file names, not including the path. You need to either:
Change your working directory to the path in question, or
Combine the path with the names for your -f test
Option #2 would just change:
if [ -f $content ];
to:
if [ -f "$path/$content" ];
Note that there are other issues here; ls may make changes to the output that break this, depending on wrapping. If you insist on using ls, you can at least make it (somewhat) safer with:
contents="$(command ls -1F "$path")"
You have two ways of doing this properly:
Either loop through the * pattern and test file type:
#!/usr/bin/env bash
echo "Enter the path:"
read -r path
for file in "$path/"*; do
if [ -f "$file" ]; then
echo "$file"
fi
done
Or using find to iterate a null delimited list of file-names:
#!/usr/bin/env bash
echo "Enter the path:"
read -r path
while IFS= read -r -d '' file; do
echo "$file"
done < <(
find "$path" -maxdepth 1 -type f -print0
)
The second way is preferred since it will properly handle files with special characters and offload the file-type check to the find command.
Use file, set to search for files (-type f) from $path directory:
find "$path" -type f
Here is what you could write:
#!/usr/bin/env bash
path=
while [[ ! $path ]]; do
read -p "Enter path: " path
done
for file in "$path"/*; do
[[ -f $file ]] && printf '%s\n' "$file"
done
If you want to traverse all the subdirectories recursively looking for files, you can use globstar:
shopt -s globstar
for file in "$path"/**; do
printf '%s\n' "$file"
done
In case you are looking for specific files based on one or more patterns or some other condition, you could use the find command to pick those files. See this post:
How to loop through file names returned by find?
Related
When to wrap quotes around a shell variable?
Why you shouldn't parse the output of ls
Is double square brackets [[ ]] preferable over single square brackets [ ] in Bash?
I've been trying to make a bash script that starts at a folder, namely my home folder, then gets each file recursively and checks for some properties. Say I want to check to see if my files have a certain size and have text (not binary data) in them. It should take care of the special cases where the files are hidden or starting with a hyphen. This is what I came up with:
for i in $(cd "/home/user" && ls -aR);
do
if [[ $(file ./"$i") == "./\"$i\": ASCII text" ]] && [[ $(du -b ./"$i" | grep -oE "[0-9]+") == "1015" ]]; then
echo ./"$i"
fi
done
I don't know how many subfolders there are, and I need it to echo the path of the files that meet the criteria. It works ok for files in /home/user/ but it doesn't seem to find (and thus check) the files in any subfolder. How may I fix this?
I'm going to assume you are using bash 4 unless otherwise stated.
shopt -s globstar
for f in /home/user/**/*:
if [[ $(file -- "$f") != *": ASCII text" ]]; then
continue
fi
# This is the syntax for GNU stat; consult your manual for
# other implementations
size=$(stat -c %s -- "$f")
if (( size != 1015 )); then
continue
fi
echo "$f"
done
I would separate traversing the file tree from checking the individual files.
Start by writing a script which examines a single file and prints the file name to stdout if the file matches your criteria. Let's call this script check_file. Now use, for instance,
find /home/user -type f -exec check_file {} \;
I have a problem with this script. The script is supposed to go trough all the files and all sub-directories and sub-files (recursively). If the file ends with the extension .txt i need to replace a char/word in the text with a new char/word and then copy it into a existing directory. The first argument is the directory i need to start the search, the second is the old char/word, third the new char/word and fourth the directory to copy the files to. The script goes trough the files but only does the replacement and copies the files from the original directory. Here is the script
#!/bin/bash
funk(){
for file in `ls $1`
do
if [ -f $file ]
then
ext=${file##*.}
if [ "$ext" = "txt" ]
then
sed -i "s/$2/$3/g" $file
cp $file $4
fi
elif [ -d $file ]
then
funk $file $2 $3 $4
fi
done
}
if [ $# -lt 4 ]
then
echo "Need more arg"
exit 2;
fi
cw=$1
a=$2
b=$3
od=$4
funk $cw $a $b $od
You're using a lot of bad practices here: lack of quotings, you're parsing the output of ls... all this will break as soon as a filename contains a space of other funny symbol.
You don't need recursion if you either use bash's globstar optional behavior, or find.
Here's a possibility with the former, that will hopefully show you better practices:
#!/bin/bash
shopt -s globstar
shopt -s nullglob
funk() {
local search=${2//\//\\/}
local replace=${3//\//\\/}
for f in "$1"/**.txt; do
sed -i "s/$search/$replace/g" -- "$f"
cp -nvt "$4" -- "$f"
done
}
if (($#!=4)); then
echo >&2 "Need 4 arguments"
exit 1
fi
funk "$#"
The same function funk using find:
#!/bin/bash
funk() {
local search=${2//\//\\/}
local replace=${3//\//\\/}
find "$1" -name '*.txt' -type f -exec sed -i "s/$search/$replace/g" -- {} \; -exec cp -nvt "$4" -- {} \;
}
if (($#!=4)); then
echo >&2 "Need 4 arguments"
exit 1
fi
funk "$#"
In cp I'm using
the -n switch: no clobber, so as to not overwrite an existing file. Use it if your version of mv supports it, unless you actually want to overwrite files.
the -v switch: verbose, will show you the moved files (optional).
the -t switch: -t followed by a directory tells to copy into this directory. It's a very good thing to use cp this way: imagine instead of giving an existing directory, you give an existing file: without this feature, this file will get overwritten several times (well, this will be the case if you omit the -n option)! with this feature the existing file will remain safe.
Also notice the use of --. If your cp and sed supports it (it's the case for GNU sed and cp), use it always! it means end of options now. If you don't use it and if a filename start with a hyphen, it would confuse the command trying to interpret an option. With this --, we're safe to put a filename that may start with a hyphen.
Notice that in the search and replace patterns I replaced all slashes / by their escaped form \/ so as not to clash with the separator in sed if a slash happens to appear in search or replace.
Enjoy!
As pointed out, looping over find output is not a good idea. It also doesn't support slashes in search&replace.
Check gniourf_gniourf's answer.
How about using find for that?
#!/bin/bash
funk () {
local dir=$1; shift
local search=$1; shift
local replace=$1; shift
local dest=$1; shift
mkdir -p "$dest"
for file in `find $dir -name *.txt`; do
sed -i "s/$search/$replace/g" "$file"
cp "$file" "$dest"
done
}
if [[ $# -lt 4 ]] ; then
echo "Need 4 arguments"
exit 2;
fi
funk "$#"
Though you might have files with the same names in the subdirectories, then those will be overwritten. Is that an issue in your case?
I'm trying to do a bash script that will find & copy similar files to a destination directory.
For example, I'm passing a parameter 12300 to a script and I want to copy all files that start with 12300... to a new directory.
like this:
sh script.sh 12300
and here's the script:
if [ -f /home/user/bashTest/$#*.jpg ]
then
cp /home/user/bashTest/$#*.jpg /home/user/bashTest/final/
fi
This just doesn't work. I have tried all kinds of solutions but nothing has worked.
The question is: How can I use wildcard with parameter?
When you're checking for multiple files with -f or -e it can get nasty. I recommend kenfallon's blog. This is something like what he would recommend:
#! /bin/bash
ls -l /home/user/bashTest/$1*.jpg > /dev/null
if [ "$?" = "0" ]
then
cp /home/user/bashTest/$1*.jpg /home/user/bashTest/final/
fi
Not sure how the $# would play in here, or if it's required.
Enclose the thing that expands to the parameters in {}, i.e. /home/user/bashTest/${#}*.jpg. You should use $1 instead of $# in your case however as you only seem to be able to handle the first argument given to the script. $1 expands to the first argument, $2 to the second etc.
You also need a loop to iterate over all files that this glob expands to, e.g.
for file in /tmp/${#}*.jpg
do
if [ -f $file ]
then
echo $file
fi
done
Here is a solution:
#!/bin/bash
cp /home/user/bashTest/${1}*.jpg /home/user/bashTest/final/
Discussion
In this case, a simple cp command will do
I have tested it with files that have embedded spaces
Write this in script.sh:
cp /home/user/bashTest/$1*.jpg /home/user/bashTest/final/
That's all.
UPD. #macduff solution usefull too.
This will find all of them in your $HOME directory and subdirectories (you may wish to tweak find to follow/not follow symlinks and/or adjust the $HOME base directory where it starts the search)
#!/bin/sh
DEST=/your/dest/folder
for FILE in `find "$HOME" -iname "$1"*`;do
[ -f "$FILE" ] && mv "$FILE" "$DEST/$FILE"
#or ln -s ...if you want to keep it in its original location
done
if you want to do multiple patterns using $#
for PATTERN in $#; do
for FILE in `find "$HOME" -iname "$PATTERN"*`;do
[ -f "$FILE" ] && mv "$FILE" "$DEST/$FILE"
done
done