I'm working on this script but the option -x isn't working, it's supposed to move only the executable files.
This is the error I'm receiving:
$ sh wizlast.sh u555 -x
mv: target ‘./u555/ud’ is not a directory
it targets the right file (ud) but doesn't move it. I tried different types of combinations.
#!/bin/bash
dir=$1
if [ $# -lt 1 ] ; then
echo "ERROR: no argument"
exit 1 # pas 0
else
case $2 in
-d)
mv $dir/* /tmp/*
echo 'moving with -d'
;;
-x)
find -executable -type f | xargs mv -t "$dir"/* /tmp
echo 'moving executables'
;;
*)
mv $dir/* /tmp/
echo 'no flag passed so moving all'
echo "mv $dir/* /tmp/"
;;
esac
fi
man mv shows:
-t, --target-directory=DIRECTORY
You can't use $dir/* as a target directory, as the shell expands it and treats the first file in the list as the target (hence the error).
Use this format
For example to move files into $dir
find -executable -type f | xargs -I{} mv {} "$dir"/
The I{} tell xargs to replace and occurence of {} with the strings from pipe, so after mv each string is substituted before the directory "$dir"/ and the command works like normal.
The reason yours wasn't working was the strings from the find were read last and so treated as the directory to move into.
As you're working with Bash you should leverage it's tools and syntax improvement.
Solution for loop and globbing
So instead of using find you can use globbing and [[ -x ]] to test if the current file is executable:
for f in "$dir"/*; do
if [[ -x $f ]]; then
mv "$f" /tmp
fi
done
It use the conditionnal expression -x in [[ … ]]:
-x file
True if file exists and is executable
As a one-liner
You can rewrite it like: for f in "$dir"/*; do [[ -x $f ]] && mv "$f" /tmp; done
Deeper search (d>1)
Current loop is limited to what is directly in your "$dir/", if you want to explore deeper like "$dir///*" you will need:
to toggle the globstar shell option using shopt built-in ;
update your glob in the for loop to use it: "$dir"/**
shopt -s globstar # enable/set
for f in "$dir"/**/*; do [[ -x $f ]] && mv "$f" /tmp; done
shopt -u globstar # disable/unset
Arithmethic context
Bash has syntactic sugar, that let you replace:
if [ $# -lt 1 ] ; then … fi
with
if (( $# < 1 )); then … fi
More about Arithmetic Expression read articles at:
1. wooledge's wiki ;
2. bash-hackers' wiki.
Don't use wildcards in the destination part of a mv command, so instead of
mv $dir/* /tmp/*
do
mv $dir/* /tmp/
Related
I am trying to traverse all files in /home directory recursively. I want to do some linux command with each file . So, I am making use of for loop as below:
for i in /home/**/*
I have put below statements as start of script as well:
shopt -s globstar
shopt -s nullglob
But its getting stuck in for loop. It might be the problem with handling so many files. If I give some another directory(with less no of files) to for loop loop, then it traverse properly.
What else I can try.
Complete code:
#!/bin/bash
shopt -s globstar
shopt -s nullglob
echo "ggg"
for i in /home/**/*
do
NAME=${i}
echo "It's there." $NAME
if [ -f "$i" ]; then
echo "It's there." $NAME
printf "\n\n"
fi
done
Your code isn't getting stuck. It will just be very, very slow since it needs to build up the list of all files before entering the for loop. The standard alternative is to use find, but you need to be careful about what exactly you want to do. If you want it to behave exactly like your for loop, which means i) ignore hidden files (those whose name starts with .) and ii) follow symlinks, you can do this (assuming GNU find since you are on Linux):
find -L . -type f -not -name '.*' -printf '.\n' | wc -l
That will print a . for each file found, so wc -l will give you the number of files. The -L makes find dereference symlinks and the -not -name '.*' will exclude hidden files.
If you want to iterate over the output and do something to each file, you would need to use this:
find -L . -type f -not -name '.*' -print0 |
while IFS= read -r -d '' file; do
printf -- "FILE: %s\n" "$file"
done
Perhaps this approach may help:
#!/bin/bash
shopt -s globstar
shopt -s nullglob
echo "ggg"
for homedir in /home/*/
do
for i in "$homedir"**
do
NAME=${i}
echo "It's there." "$NAME"
if [ -f "$i" ]; then
echo "It's there." "$NAME"
printf "\n\n"
fi
done
done
Update: Another approach in pure bash might be
#!/bin/bash
shopt -s nullglob
walktree() {
local file
for file in *; do
[[ -L $file ]] && continue
if [[ -f $file ]]; then
# Do something with the file "$PWD/$file"
echo "$PWD/$file"
elif [[ -d $file ]]; then
cd "./$file" || exit
walktree
cd ..
fi
done
}
cd /home || exit
walktree
I want to implement incremental backup in Ubuntu, so I am thinking of finding md5sum of all files from source and target and check if any two files have same md5sum then keep that file in destination else if different copy the file from source into directory.
I am thinking of doing this in bash
Can anyone help me with the commands of how to check md5sum of two files in different directories ?
Thanks in advance!!
#!/bin/bash
#
SOURCE="/home/pallavi/backup1"
DEST="/home/pallavi/BK"
count=1
TODAY=$(date +%F_%H%M%S)
cd "${DEST}" || exit 1
mkdir "${TODAY}"
while [ $count -le 1 ]; do
count=$(( $count + 1 ))
cp -R $SOURCE/* $DEST/$TODAY
mkdir "MD5"
cd ${DEST}/${TODAY}
for f in *;do
md5sum "${f}" >"${TODAY}${f}.md5"
echo ${f}
done
if [ $? -ne 0 ] && [[ $IGNORE_ERR -eq 0 ]]; then
#error or eof
echo "end of source or error"
break
fi
done
This is reinventing the wheel sort of thing.
There are some utility written for this kind of purpose, to name a few.
rsync
GNU cp(1) has the -u flag.
cp
For comparing files
cmp
diff
For finding duplicates
fdupes
rmlint
Here is what I've come up with re-inventing the wheel sort of thing.
#!/usr/bin/env bash
shopt -s extglob
declare -A source_array
while IFS= read -r -d '' files; do
read -r source_hash source_files < <(sha512sum "$files")
source_array["$source_hash"]="$source_files"
done < <(find source/ -type f -print0)
source=$( IFS='|'; printf '%s' "#(${!source_array[*]})" )
while IFS= read -r -d '' files0 ; do
read -r destination_hash destination_files < <(sha512sum "$files0")
if [[ $destination_hash == $source ]]; then
echo "$destination_files" FOUND from source/ directory
else
echo "$destination_files" NOT-FOUND from source/ directory
fi
done < <(find destination/ -type f -print0)
Should be safe enough from files with spaces and tabs and new lines, but since I don't have files with new lines so I can't really say.
Change the action from the if-else statement depending on what you want to do.
Ok maybe sha512sum is a bit over kill, change it to md5sum
Add set -x after the shebang to see what's actually being executed, good luck.
I need a bash script to recursively rename files with blank extensions to append .txt at the end. I found the following script, but I can't figure out how to make it recursive:
#!/bin/sh
for file in *; do
test "${file%.*}" = "$file" && mv "$file" "$file".txt;
done
Thanks.
Thanks.
You can delegate the heavy lifting to find
$ find . -type f ! -name "*.*" -print0 | xargs -0 -I file mv file file.txt
assumption is without extension means without a period in name.
If you don't mind using a recursive function, then you can do it in older Bash versions with:
shopt -s nullglob
function add_extension
{
local -r dir=$1
local path base
for path in "$dir"/* ; do
base=${path##*/}
if [[ -f $path && $base != *.* ]] ; then
mv -- "$path" "$path.txt"
elif [[ -d $path && ! -L $path ]] ; then
add_extension "$path"
fi
done
return 0
}
add_extension .
The mv -- is to protect against paths that begin with a hyphen.
running the following script to rename all .jpg files in the current folder works well sometimes, but it often deletes one or more files it is renaming. How would I write a script to rename files without deleting them in the process? This is running on Mac OSX 10.8 using GNU bash, version 3.2.48
this is an example file listing I would change for illustration:
original files
red.jpg
blue.jpg
green.jpg
renamed files if counter is set to 5
file_5.jpg
file_6.jpg
file_7.jpg
instead I get usually lose one or more files
#!/bin/bash
counter=5
for file in *.jpg; do
echo renaming "$file" to "file_${counter}.jpg";
mv "$file" "file_${counter}.jpg";
let "counter+=1";
done
** UPDATE **
it no longer seems to be deleting files, but the output is still not as expected. for example:
file_3.jpg
file_4.jpg
turns into
file_3.jpg
file_5.jpg
when counter is set to 4, when the expected output is
file_4.jpg
file_5.jpg
-
#!/bin/bash
counter=3
for file in *.jpg; do
if [[ -e file_${counter}.jpg ]] ; then
echo Skipping "$file", file exists.
else
echo renaming "$file" to "file_${counter}.jpg"
mv "$file" "file_${counter}.jpg"
fi
let "counter+=1"
done
The problem is that some of the files already have names corresponding to the target names. For example, if there are files
file_1.jpg
file_7.jpg
and you start with counter=7, you overwrite file_7.jpg with file_1.jpg in the first step, and then rename it to file_8.jpg.
You can use mv -n to prevent clobbering (if supported), or test for existence before running the command
if [[ -e file_${counter}.jpg ]] ; then
echo Skipping "$file", file exists.
else
mv "$file" "file_${counter}.jpg"
fi
I think you are glazing over an obvious problem with the glob. If the glob matches file_2.jpg, it will try and create file_file_2.jpg (I don't mean that in the literal sense, just that you will be reprocessing files you already processed). To solve this, you need to make sure your initial glob expression doesn't match the files you have already moved:
shopt -s extglob
i=0
for f in !(file_*).jpg ; do
while [[ -e "file_${i}.jpg" ]] ; do
(( i++ ))
done
mv -v "$f" "file_$i.jpg"
(( i++ ))
done
What choroba said is correct. You can also use:
mv "$file" "file_${counter}.jpg" -n
to simply neglect the move when the destination filename already exists, or
mv "$file" "file_${counter}.jpg" -i
to ask whether it should overwrite or not.
Instead of iterating over *.jpg you should skip your already renamed files i.e. file_[0-9]*.jpg and run your loop like this:
counter=5
while read file; do
echo renaming "$file" to "file_${counter}.jpg";
mv -n "$file" "file_${counter}.jpg";
let "counter+=1";
done < <(find . -maxdepth 1 -name "*.jpg" -not -name "file_[0-9]*.jpg")
Another way is to continue your counting until a file does not exist:
#!/bin/bash
counter=1
shopt -s extglob
for file in *.jpg; do
[[ $file == ?(*/)file_+([[:digit:]]).jpg ]] && continue
until
newname=file_$(( counter++ )).jpg
[[ ! -e $newname ]]
do
continue
done
echo "renaming $file to $newname.";
mv -i "$file" "$newname" ## Remove the -i option if you think it's safe already.
done
When doing things recursively:
#!/bin/bash
shopt -s extglob
counter=1
while read file; do
dirprefix=${file%%+([^/])
until
newfile=$dirprefix/file_$(( counter++ )).jpg
[[ ! -e $newfile ]]
do
continue
done
echo "renaming $file to $newfile."
mv -i "$file" "$newfile" ## Remove the -i option if you think it's safe already.
done < <(find -type f -name '*.jpg' -and -not -regex '^.*/file_[0-9]\+$')
I have a problem with this script. The script is supposed to go trough all the files and all sub-directories and sub-files (recursively). If the file ends with the extension .txt i need to replace a char/word in the text with a new char/word and then copy it into a existing directory. The first argument is the directory i need to start the search, the second is the old char/word, third the new char/word and fourth the directory to copy the files to. The script goes trough the files but only does the replacement and copies the files from the original directory. Here is the script
#!/bin/bash
funk(){
for file in `ls $1`
do
if [ -f $file ]
then
ext=${file##*.}
if [ "$ext" = "txt" ]
then
sed -i "s/$2/$3/g" $file
cp $file $4
fi
elif [ -d $file ]
then
funk $file $2 $3 $4
fi
done
}
if [ $# -lt 4 ]
then
echo "Need more arg"
exit 2;
fi
cw=$1
a=$2
b=$3
od=$4
funk $cw $a $b $od
You're using a lot of bad practices here: lack of quotings, you're parsing the output of ls... all this will break as soon as a filename contains a space of other funny symbol.
You don't need recursion if you either use bash's globstar optional behavior, or find.
Here's a possibility with the former, that will hopefully show you better practices:
#!/bin/bash
shopt -s globstar
shopt -s nullglob
funk() {
local search=${2//\//\\/}
local replace=${3//\//\\/}
for f in "$1"/**.txt; do
sed -i "s/$search/$replace/g" -- "$f"
cp -nvt "$4" -- "$f"
done
}
if (($#!=4)); then
echo >&2 "Need 4 arguments"
exit 1
fi
funk "$#"
The same function funk using find:
#!/bin/bash
funk() {
local search=${2//\//\\/}
local replace=${3//\//\\/}
find "$1" -name '*.txt' -type f -exec sed -i "s/$search/$replace/g" -- {} \; -exec cp -nvt "$4" -- {} \;
}
if (($#!=4)); then
echo >&2 "Need 4 arguments"
exit 1
fi
funk "$#"
In cp I'm using
the -n switch: no clobber, so as to not overwrite an existing file. Use it if your version of mv supports it, unless you actually want to overwrite files.
the -v switch: verbose, will show you the moved files (optional).
the -t switch: -t followed by a directory tells to copy into this directory. It's a very good thing to use cp this way: imagine instead of giving an existing directory, you give an existing file: without this feature, this file will get overwritten several times (well, this will be the case if you omit the -n option)! with this feature the existing file will remain safe.
Also notice the use of --. If your cp and sed supports it (it's the case for GNU sed and cp), use it always! it means end of options now. If you don't use it and if a filename start with a hyphen, it would confuse the command trying to interpret an option. With this --, we're safe to put a filename that may start with a hyphen.
Notice that in the search and replace patterns I replaced all slashes / by their escaped form \/ so as not to clash with the separator in sed if a slash happens to appear in search or replace.
Enjoy!
As pointed out, looping over find output is not a good idea. It also doesn't support slashes in search&replace.
Check gniourf_gniourf's answer.
How about using find for that?
#!/bin/bash
funk () {
local dir=$1; shift
local search=$1; shift
local replace=$1; shift
local dest=$1; shift
mkdir -p "$dest"
for file in `find $dir -name *.txt`; do
sed -i "s/$search/$replace/g" "$file"
cp "$file" "$dest"
done
}
if [[ $# -lt 4 ]] ; then
echo "Need 4 arguments"
exit 2;
fi
funk "$#"
Though you might have files with the same names in the subdirectories, then those will be overwritten. Is that an issue in your case?