How to remove files starting with #! or ending with .sh in the name - shell

I am new to shell programming. I want to move any executable file, any file starting with shebang(#!), and any file whose name ends with .sh from a directory to /tmp/backup and log the names of the files moved.
This is what I have done till now
Searching for files with #^
grep -ircl --exclude=*.{png,jpg,gif,html,jar} "^#" /home
Finding executables
find . -type f -perm +111 or find . -type f -perm -u+x
Now I am struggling how to club these two commands get a final output which I can pass to perform backup and remove from current directory
Thanks

Use the xargs command
"find command" | xargs "grep command"

You could put everything in a file, sort it, then process it with Awk:
# Select all files to move
grep -ircl --exclude=*.{png,jpg,gif,html,jar} '^#\!' /home > list.txt
find /home -type f \( -perm -u+x -o -name "*.sh" \) -print >> list.txt
# Feed them to Awk that will log and move the file
sort list.txt | uniq | awk -v LOGFILE="mylog.txt" '
{ print "Moving " $0 >> LOGFILE
"mv -v --backup \"" $0 "\" /tmp/backup" | getline
print >> LOGFILE }'
EDIT: you can make a formal script out of this skeleton, by adding some variables and some additional checks:
#!/bin/bash
LIST="$( mktemp || exit 1 )"
LOG="/tmp/mylog.txt"
SOURCE="/home"
TARGET="/tmp/backup"
mkdir -p "${TARGET}"
cd "${SOURCE}" || exit 1
# Select all files to move
grep -ircl --exclude=*.{png,jpg,gif,html,jar} '^#\!' "${SOURCE}" > "${LIST}"
find "${SOURCE}" -type f \( -perm -u+x -o -name "*.sh" \) -print >> "${LIST}"
# Feed them to Awk that will log and move the file
sort "${LIST}" | uniq | awk -v LOGFILE="${LOG}" -v TARGET="${TARGET}" '
{ print "Moving " $0 >> LOGFILE
"mv -v --backup \"" $0 "\" " TARGET | getline
print >> LOGFILE }'

Related

Find the file path of a specific file inside a zipped folder

I'm struggling with my script. I want to find the full file path of a specific file, just like this example:
/path/folder_06may2017.zip/file_B.txt
I hope you can help me solving this problem. It will be really useful.
Directory and file samples
- Folder_01may2017.zip
+ file_A.txt
+ file_B.txt
- Folder_06may2017.zip
+ file_A.txt
+ file_B.txt
I have used this commands with no success at all:
1st attempt:
find "/path/folder/" -name "*06may2017*" -print -exec unzip -l {} \; | grep -i 'file_B'
1st Output:
182118 2017-05-06 11:20 file_B.txt
2nd attempt:
find "/path/folder/" -name "*06may2017*" -print -exec unzip -l {} \; | grep -i 'file_B'| awk '{ print $4 }' ${PWD}
2nd output:
awk: warning: command line argument '/path/folder' is a directory: skipped
3rd attempt:
find "/path/folder" -name "*06may2017*" -exec grep -l "file_B" /dev/null '{}' \;
3rd output
/path/folder/Folder_06may2017.zip
What about:
$ find "/path/folder" -name "*06may2017*" -exec unzip -l {} \; | awk '$1 ~ /Archive/{zipname = $2}; $4 ~ /file_B/ {printf "%s/%s\n", zipname, $4}'

Rename files to unique names and move them into a single destination directory

i have 100s of directories with same filename of content.html along with other files.
I am trying to copy all these content.html files under 1 directory, but since they have same name, it overwrites each other
so how can i rename and move all these under 1 directory
Eg:
./0BD3D9D2-F8B1-4472-95C2-13319650A45C:
card.png content.html note.xhtml quickLook.png snippet.txt
./0EA34DB4-CD56-42BE-91DA-F631E44FB6E0:
card.png content.html note.xhtml quickLook.png related snippet.txt
./1A33F29E-3938-4C2F-BA99-6B98FD045742:
card.png content.html note.xhtml quickLook.png snippet.txt
command i tried:
rename content.html to content
find . -type f | grep content.html | while read f; do mv $f ${f/.html/}; done
append number to filename "content" to make it unique
find . -type f | grep content | while read f; do i=1; echo mv $f $f$i.html; i=i+1; done
MacBook-Pro$ find . -type f | grep content | while read f; do i=1; echo mv $f $f$i.html; i=i+1; done
mv ./0BD3D9D2-F8B1-4472-95C2-13319650A45C/content ./0BD3D9D2-F8B1-4472-95C2-13319650A45C/content1.html
mv ./0EA34DB4-CD56-42BE-91DA-F631E44FB6E0/content ./0EA34DB4-CD56-42BE-91DA-F631E44FB6E0/content1.html
mv ./1A33F29E-3938-4C2F-BA99-6B98FD045742/content ./1A33F29E-3938-4C2F-BA99-6B98FD045742/content1.html
once above step is successful, i should be able do this to achieve my desired output:
find . -type f | grep content | while read f; do mv $f ../; done
however, i am sure i can do this in 1 step command and also my step 2 is not working (incrementing i)
any idea why step2 is not working??
bash script:
#!/bin/bash
find . -type f -name content.html | while IFS= read -r f; do
name=$(basename $f)
((++i))
mv "$f" "for_content/${name%.*}$i.html"
done
replace for_content with your destination folder name
Suppose in your base directory, you create a folder named final for storing
content.html files, then do something like below
find . -path ./final -prune -o -name "content.html" -print0 |
while read -r -d '' name
do
mv "$name" "./final/content$(mktemp -u XXXX).html"
# mktemp with -u option just creates random characters, or it is just a dry run
done
At the end you'll get all the content.html files under ./final folder in the format contentXXXX.html where XXXX are random characters.
Note:-path ./final -prune -o in find prevents it from descending to our results folder.
The inode of the of the files should be unique and so you could use the following:
find $(pwd) -name "content.html" -printf %f" "%i" "%p"\n" | awk '{ system("mv "$3" <directorytomoveto>"$2$1) }'
I'd use something like this:
find . -type f -name 'test' | awk 'BEGIN{ cnt=0 }{ printf "mv %s ./output-dir/content_%03d.txt\n", $0, cnt++ }' | bash;
You can replace ./output-dir/ with your destination directory
Example:
[root#sl7-o2 test]# ls -R
.:
1 2 3 output-dir
./1:
test
./2:
test
./3:
test
./output-dir:
[root#sl7-o2 test]# find . -type f -name 'test' | awk 'BEGIN{ cnt=0 }{ printf "mv %s ./output-dir/content_%03d.txt\n", $0, cnt++ }' | bash;
[root#sl7-o2 test]# ls ./output-dir/
content_000.txt content_001.txt content_002.txt
You can use shopt -s globstar to grab all content.html files recursively and then use a loop to rename them:
#!/bin/bash
set -o globstar
counter=0
dest_dir=/path/to/destination
for f in **/content.html; do # pick up all content.html files
[[ -f "$f" ]] || continue # skip if not a regular file
mv "$f" "$dest_dir/content_$((++counter).html"
done

Using awk to print ALL spaces within filenames which have a varied number of spaces

I'm executing the following using bash and awk to get the potentially space-full filename, colon, file size. (Column 5 contains the space delimited size, and 9 to EOL the file name):
src="Desktop"
echo "Constructing $src files list. `date`"
cat /dev/null > "$src"Files.txt
find -s ~/"$src" -type f -exec ls -l {} \; |
awk '{for(i=9;i<=NF;i++) {printf("%s", $i " ")} print ":" $5}' |
grep -v ".DS_Store" | grep -v "Icon\r" |
while read line ; do filespacesize=`basename "$line"`; filesize=`echo "$filespacesize" |
sed -e 's/ :/:/1'`
path=`dirname "$line"`; echo "$filesize:$path" >> "$src"Files.txt ;
done
And it works fine, BUT…
If a filename has > 1 space between parts, I only get 1 space between filename parts, and the colon, followed by the filesize.
How can I get the full filename, :, and then the file size?
It seems you want the following (provided your find handles the printf option with the %f, %s and %h modifiers):
src=Desktop
echo "Constructing $src files list. $(date)"
find ~/"$src" -type f -printf '%f:%s:%h\n' > "$src"Files.txt
Much shorter and much more efficient than your method!
This will not discard the .DS_STORE and Icon\r things… but I'm not really sure what you really want to discard. If you want to discard the .DS_STORE directory altogether:
find ~/"$src" -name '.DS_STORE' -type d -prune -o -type f -printf '%f:%s:%h\n' > "$src"Files.txt
#guido seems to have guessed what you mean by grep -v "Icon\r": ignore files ending with Icon; if this his guess is right, then this will do:
find ~/"$src" -name '.DS_STORE' -type d -prune -o ! -name '*Icon' -type f -printf '%f:%s:%h\n' > "$src"Files.txt

bash - padding find results

I'm running the following command to get a directory listing:
find ./../ \
-type f -newer ./lastsearchstamp -path . -prune -name '*.txt' -o -name '*.log' \
| awk -F/ '{print $NF " - " $FILENAME}'
Is there some way I can format the output in a 2 column left indented layout so that the output looks legible?
The command above always adds a constant spacing between the filename and the path.
Expected output:
abc.txt /root/somefolder/someotherfolder/
helloworld.txt /root/folder/someotherfolder/
a.sh /root/folder/someotherfolder/scripts
I nice tool for this kind of thing is column -t. You just add the command on to the end of the pipeline:
find ... | awk -F/ '{print $NF " - " $FILENAME}' | column -t

Bash , append line at the end of each file

I have files in a dir.
I need to append a new line and the file name at the end of each file.
This should do:
for f in *; do echo >> $f; echo $f >> $f; done
First echo a new-line, then echo the filename.
The >> says "append at the end of the file".
Edit: aioobe's answer has been updated to show the -e flag that I didn't see when I first answered this. Thus I'm now just showing an example which includes a directory and how to eliminate the directory name:
#!/bin/bash
for fn in dir/*
do
shortname=${fn/#*\//}
echo -e "\n$shortname" >> $fn
done
If you want the directory name, take out the shortname=${fn/#*\//} line and replace $shortname with $fn in the echo.
Let xargs do the looping:
# recursive, includes directory name
find -type f -print0 | xargs -0 -I% bash -c 'echo -e "\n%" >> %'
or
# non-recursive, doesn't include directory name
find -maxdepth 1 -type f -exec basename {} \; | xargs -I% bash -c 'echo -e "\n%" >> %'
or
# non-recursive, doesn't include directory name
find -maxdepth 1 -type f -printf "%f\0" | xargs -0 -I% bash -c 'echo -e "\n%" >> %'
or
# recursive, doesn't include directory name
find -type f -print0 | xargs -0 -I% bash -c 'f=%; echo -e "\n${f##*/}" >> %'
Another method using ex (or vim) :
ex -c 'args **/*' -c 'set hidden' -c 'argdo $put =bufname(".")' -c 'wqa'

Resources