I need generate list of files and directories in macOS terminal like find / -ls > list.txt but adding the columns: creation time date, modified time date in the begin. Is possible to separate every field with tab char on output?
Finally I got it!
echo -e "inode birth\tmodified\tcreated\ttype and permissions\tsize in bytes\tuser id\tgroup id\tmd5 hash\tfile name with full path" | tee "$(date "+%Y-%m-%d_%H-%M") list of files and dirs.txt";find ~ | while read file; do (stat -f %SB "$file";stat -f %Sm "$file"; stat -f %Sc "$file"; stat -f %Sp "$file"; stat -f %Dz "$file"; stat -f %Su "$file"; stat -f %Sg "$file"; md5 -q -s "$file"; echo $file;)|tr '\n' '\t';echo ; done | tee -a "$(date "+%Y-%m-%d_%H-%M") list of files and dirs.txt"
Related
I use youtube-dl to archive specific blogs. I use a custom bash script (called tvify) to help me organize my content into Plex-ready filenames for later replay via my home Plex server.
Archiving the content works fine, unless a blogger posts more than one video on the same date - if that happens my script creates more than one file for a given month/date and plex sees a duplicate episode. In the plex app, it stuffs them together as distinct 'versions' of the same episode. The result is that the description of the video no longer matches its contents, and only one 'version' appears unless I access an additional sub menu.
The videos get downloaded by you tube-dl kicked off from a cron-job, and that downloader script runs the following to help format their filenames and stuff them into appropriate folders for 'seasons'.
The season is the year when the video was released, and the episode is the combination of the month and date in MMDD format.
Below is my 'tvify' script, which helps perform the filename manipulation and stuffs the file into the proper folder for the season.
#!/bin/bash
mySuff="$1"
echo mySuff="$mySuff"
if [ -z "$1" ]; then
mySuff="*.mp4"
fi
for i in $mySuff
do
prb=`ffprobe -- "$i" 2>&1`
myDate=`echo "$prb" | grep -E 'date\s+:' | cut -d ':' -f 2`
myartist=`echo "$prb" | grep -E 'artist\s+:' | cut -d ':' -f 2`
myTitle=`echo "$prb" | grep -E 'title\s+:' | cut -d ':' -f 2 | sed 's/\//_/g'`
cwd_stub=`pwd | awk -F'/' '{print $NF}'`
if [ -d "s${myDate:1:4}" ]; then echo "Directory found" > /dev/null; else mkdir "s${myDate:1:4}"; fi
[ -d "s${myDate:1:4}" ] && mv -- "$i" "s${myDate:1:4}/${myartist[#]:1} - s${myDate:1:4}e${myDate:5:8} - ${myTitle[#]:1:40} _$i" || mv -- "$i" "${myartist[#]:1} - s${myDate:1:4}e${myDate:5:8} - ${myTitle[#]:1:40} _$i"
done
How can I modify that script to identify if a conflicting year/MMDD file exists, and if so, append an appropriate suffix to the episode number so that plex will interpret them as distinct episodes?
I ended up implementing an array, counting the number of elements in the array, and using that to append the integer:
#!/bin/bash
mySuff="$1"
echo mySuff="$mySuff"
if [ -z "$1" ]; then
mySuff="*.mp4"
fi
for i in $mySuff
do
prb=`ffprobe -- "$i" 2>&1`
myDate=`echo "$prb" | grep -E 'date\s+:' | cut -d ':' -f 2`
myartist=`echo "$prb" | grep -E 'artist\s+:' | cut -d ':' -f 2`
myTitle=`echo "$prb" | grep -E 'title\s+:' | cut -d ':' -f 2 | sed 's/\//_/g'`
cwd_stub=`pwd | awk -F'/' '{print $NF}'`
readarray -t conflicts < <(find . -maxdepth 2 -iname "*s${myDate:1:4}e${myDate:5:8}*" -type f -printf '%P\n')
[ ${#conflicts[#]} -gt 0 ] && _inc=${#conflicts[#]} || _inc=
if [ -d "s${myDate:1:4}" ]; then echo "Directory found" > /dev/null; else mkdir "s${myDate:1:4}"; fi
[ -d "s${myDate:1:4}" ]
&& mv -- "$i" "s${myDate:1:4}/${myartist[#]:1} - s${myDate:1:4}e${myDate:5:8}$_inc - ${myTitle[#]:1:40} _$i"
|| mv -- "$i" "${myartist[#]:1} - s${myDate:1:4}e${myDate:5:8}$_inc - ${myTitle[#]:1:40} _$i"
done
I need to write a bash script that finds and lists symlinks from one directory (lets say some "Directory1") but only the ones pointing to files in certain another directory (lets say "Directory2"). I can`t use "find".
I have tried something like this but it's apparently wrong:
if [[ -d $1 ]]&&[[ -d $2 ]]
then
current_dir='pwd'
cd $1
do
for plik in *
if[[-L $file] && ["$(readlink -- "$file")" = "$2"] ]
then
#ls -la | grep ^l
echo "$(basename "$file")"
fi
done
fi
How about a simple ls with grep :
ls -l Directory1/ | grep "\->" | grep "Directory2"
I have found a solution:
for file1 in $_cat1/* do
if [ -L $file1]; then
dir1="$(dirname `readlink -f $file1`)"
dir2="$(dirname `readlink -f $_cat2`)"
if [dir1 == dir2]
echo $dir1
fi
fi
done
I have two folders with some common files, I want to delete duplicate files from xyz folder.
folder1:
/abc/file1.csv
/abc/file2.csv
/abc/file3.csv
/abc/file4.csv
folder2:
/xyz/file1.csv
/xyz/file5.csv
I want to compare both folders and remove duplicate from /xyz folder. Output should be: file5.csv
For now I am using :
find "/xyz" "/abc" "/abc" -printf '%P\n' | sort | uniq -u | -exec rm {} \;
But it failing with reason : if -exec is not a typo you can run the following command to lookup the package that contains the binary:
command-not-found -exec
-bash: -exec: command not found
-exec is an option to find, you've already exited the command find when you started the pipes.
Try xargs instead, it take all the data from stdin and appends to the program.
UNTESTED
find "/xyz" "/abc" "/abc" -printf '%P\n' | sort | uniq -u | xargs rm
Find every file in 234 and 123 directory get filename by -printf, sort them, uniq -d give list of duplications, give back path by sed, using 123 directory to delete the duplications from, and pass files to xargs rm
Command:
find ./234 ./123 -type f -printf '%P\n' | sort | uniq -d | sed 's/^/.\/123\//g' | xargs rm
sed don't needed if you are in the ./123 directory and using full path for folders in find.
Another approach: just find the files in abc and attempt to remove them from xyz:
UNTESTED
find /abc -type f -printf 'rm -f /xyz/%P' | sh
Remove Duplicate Files From Particular Directory
FileList=$(ls)
for D1 in $FileList ;do
if [[ -f $D1 ]]; then
for D2 in $FileList ;do
if [[ -f $D2 ]]; then
if [[ $D1 == $D2 ]]; then
: 'Skip Orignal File'
else
if [[ $(md5sum $D1 | cut -d'=' -f 2 | cut -d ' ' -f 1 ) == $(md5sum $D2 | cut -d'=' -f 2 | cut -d ' ' -f 1 ) ]]; then
echo "Duplicate File Found : $D2"
rm -rf $D2
fi #Detect Duplicate Using MD5
fi #Skip Orginal File
fi #D2 File available Then Next
done
fi #D1 File available Then Next
done
AIM: To find files with a word count less than 1000 and move them another folder. Loop until all under 1k files are moved.
STATUS: It will only move one file, then error with "Unable to move file as it doesn't exist. For some reason $INPUT_SMALL doesn't seem to update with the new file name."
What am I doing wrong?
Current Script:
Check for input files already under 1k and move to Split folder
INPUT_SMALL=$( ls -S /folder1/ | grep -i reply | tail -1 )
INPUT_COUNT=$( cat /folder1/$INPUT_SMALL 2>/dev/null | wc -l )
function moveSmallInput() {
while [[ $INPUT_SMALL != "" ]] && [[ $INPUT_COUNT -le 1003 ]]
do
echo "Files smaller than 1k have been found in input folder, these will be moved to the split folder to be processed."
mv /folder1/$INPUT_SMALL /folder2/
done
}
I assume you are looking for files that has the word reply somewhere in the path. My solution is:
wc -w $(find /folder1 -type f -path '*reply*') | \
while read wordcount filename
do
if [[ $wordcount -lt 1003 ]]
then
printf "%4d %s\n" $wordcount $filename
#mv "$filename" /folder2
fi
done
Run the script once, if the output looks correct, then uncomment the mv command and run it for real this time.
Update
The above solution has trouble with files with embedded spaces. The problem occurs when the find command hands its output to the wc command. After a little bit of thinking, here is my revised soltuion:
find /folder1 -type f -path '*reply*' | \
while read filename
do
set $(wc -w "$filename") # $1= word count, $2 = filename
wordcount=$1
if [[ $wordcount -lt 1003 ]]
then
printf "%4d %s\n" $wordcount $filename
#mv "$filename" /folder2
fi
done
A somewhat shorter version
#!/bin/bash
find ./folder1 -type f | while read f
do
(( $(wc -w "$f" | awk '{print $1}' ) < 1000 )) && cp "$f" folder2
done
I left cp instead of mv for safery reasons. Change to mv after validating
I you also want to filter with reply use #Hai's version of the find command
Your variables INPUT_SMALL and INPUT_COUNT are not functions, they're just values you assigned once. You either need to move them inside your while loop or turn them into functions and evaluate them each time (rather than just expanding the variable values, as you are now).
How do I list newest directory and add as variable to bash script to process files recursively
ls -t1 | head -n1
Works perfectly to list the latest directory, but I want to add that directory name to my script so I can process the files within using the following script:
#!/bin/bash
ls | while read -r FILE
do
mv -v "$FILE" `echo $FILE | tr ' ' '_' `
done
ls | while read -r FILE
do
mv -v "$FILE" `echo $FILE | tr '\*.JPEG' '\*.jpg' `
done
mogrify -resize 750 *.jpg
wait
jpegoptim *.jpg –max=70 --strip-all
exit
I also want to process the files recursively, there might be at most one level of sub directories.
Basically keep the bash script at the root of the directory and process all latest directories and sub directories files.
OK I modified the script to this:
#!/bin/bash
DIR=ls -t1 | head -n1
ls $DIR | while read -r FILE
do
mv -v "$FILE" `echo $FILE | tr ' ' '_' `
done
ls $DIR | while read -r FILE
do
mv -v "$FILE" `echo $FILE | tr '\*.JPEG' '\*.jpg' `
done
mogrify -resize 750 $DIR/*.jpg
wait
jpegoptim $DIR/*.jpg –max=70 --strip-all
exit
But it does not seem to recognise the $DIR variable.
This bash script will rename and convert the jpg files in the newest directory in the current directory and the files in the first level of directories under that directory.
#!/bin/bash
FIRST_DIR=`ls -t1F | grep / | head -n1`
DIR="./${FIRST_DIR}"
ls -t1F $DIR | while read -r FILE
do
if [ "$FILE" ]
then
if [[ $FILE = */ ]]
then
echo "here ${DIR}${FILE}."
DEEP_DIR="${DIR}${FILE}"
ls -t1 $DEEP_DIR | while read -r FILE2
do
if [ "$FILE2" ]
then
if [[ $FILE2 != */ ]]
then
RENAME=`echo ${FILE2//\*/} | tr ' ' '_' `
mv -v "${DEEP_DIR}${FILE2//\*/}" "${DEEP_DIR}${RENAME}"
FILE2=$RENAME
RENAME2=`echo ${FILE2//\*/} | tr '\*.JPEG' '\*.jpg' `
mv -v "${DEEP_DIR}${FILE2//\*/}" "${DEEP_DIR}${RENAME2}"
FILE2=$RENAME2
fi
fi
done
mogrify -resize 750 "$DEEP_DIR*.jpg"
wait
jpegoptim "$DEEP_DIR*.jpg" –max=70 --strip-all
fi
if [[ $FILE != */ ]]
then
RENAME=`echo ${FILE//\*/} | tr ' ' '_' `
mv -v "${DIR}${FILE//\*/}" "${DIR}${RENAME}"
FILE=$RENAME
RENAME2=`echo ${FILE//\*/} | tr '\*.JPEG' '\*.jpg' `
mv -v "${DIR}${FILE//\*/}" "${DIR}${RENAME2}"
FILE=$RENAME2
fi
fi
done
if [ "$FIRST_DIR" ]
then
mogrify -resize 750 "$DIR*.jpg"
wait
jpegoptim "$DIR*.jpg" –max=70 --strip-all
fi
Here are a couple of good links about bash programming:
http://tldp.org/LDP/abs/html/comparison-ops.html
http://tldp.org/LDP/abs/html/string-manipulation.html