unix home directories without entries in /etc/passwd - shell

I am able to get both listings ( /etc/passwd and /home ) but how to script something like read line of /etc/passwd, parse home directory, then look for that in /home . If it doesn't exist, throw an error, if it does exist, move along.
/etc/passwd home dir listing for users
cut -d":" -f6 /etc/passwd | grep home | sort
user listing from /home
ls -1 /home | (while read line; do echo "/home/"$line; done)
Maybe right out output from first command to a file, then read each line into a find command and...or, test with
if [ -d "$DIRECTORY" ]; then
echo "directory found for user that doesn't exist"
fi
Now how to put it all together...
EDIT: isedev had exactly what I needed. I may have mis-worded my original message...we have been cleaning up users, but not cleaning up their /home directory. So I want to know what /home directories still exist that don't have /etc/passwd entries.
this is what worked to a T
for name in /home/*; do
if [ -d "$name" ]; then
cut -d':' -f6 /etc/passwd | egrep -q "^$name$"
if [ $? -ne 0 ]; then
echo "directory $name does not correspond to a valid user"
fi
fi
done
from now on, we will be running
userdel -r login

This will report all home directories from /etc/passwd that should be in /home but aren't:
cut -d":" -f6 /etc/passwd | grep home | sort |
while read dir; do [ -e "$dir" ] || echo Missing $dir; done
And this one reports all that don't exist:
cut -d":" -f6 /etc/passwd | while read dir; do
[ -e "$dir" ] || echo Missing $dir
done

as 1st approximation:
perl -F: -lane 'next if m/^#/;print "$F[5] for user $F[0] missing\n" unless(-d $F[5])' /etc/passwd
if you want find the differences between the /etc/passwd and the /home
comm <(find /home -type d -maxdepth 1 -mindepth 1 -print|sort) <(grep -v '^#' /etc/passwd | cut -d: -f6| grep '/home' | sort)
in an narrow form
comm <(
find /home -type d -maxdepth 1 -mindepth 1 -print |sort
) <(
grep -v '^#' /etc/passwd |cut -d: -f6 |grep /home |sort
)
if you will use
comm ... (without args as above) will show 3 colums 1.) only in /home 2.)only in /etc/passwd 3.) common
comm -23 .... - will show directories what are only in the /home (and not in the /etc/passwd)
comm -13 .... - will show dirs what are only in the /etc/passwd and not in the /home
comm -12 .... - will show correct directories (exists in the /etc/passwd and the /home too)
I'm not sure with the -{max|min}depth on the AIX..

So, assuming you want to know if there are directories under /home which do not correspond to existing users:
for name in /home/*; do
if [ -d "$name" ]; then
cut -d':' -f6 /etc/passwd | egrep -q "^$name$"
if [ $? -ne 0 ]; then
echo "directory $name does not correspond to a valid user"
fi
fi
done
Then again, this assumes you are not using a name service such as LDAP or NIS, in which case, change the line starting with cut to:
getent passwd | cut -d':' -f6 | egrep -q "^$name$"

Related

bash script to scan for repeated episode numbers, append episode modifier

I use youtube-dl to archive specific blogs. I use a custom bash script (called tvify) to help me organize my content into Plex-ready filenames for later replay via my home Plex server.
Archiving the content works fine, unless a blogger posts more than one video on the same date - if that happens my script creates more than one file for a given month/date and plex sees a duplicate episode. In the plex app, it stuffs them together as distinct 'versions' of the same episode. The result is that the description of the video no longer matches its contents, and only one 'version' appears unless I access an additional sub menu.
The videos get downloaded by you tube-dl kicked off from a cron-job, and that downloader script runs the following to help format their filenames and stuff them into appropriate folders for 'seasons'.
The season is the year when the video was released, and the episode is the combination of the month and date in MMDD format.
Below is my 'tvify' script, which helps perform the filename manipulation and stuffs the file into the proper folder for the season.
#!/bin/bash
mySuff="$1"
echo mySuff="$mySuff"
if [ -z "$1" ]; then
mySuff="*.mp4"
fi
for i in $mySuff
do
prb=`ffprobe -- "$i" 2>&1`
myDate=`echo "$prb" | grep -E 'date\s+:' | cut -d ':' -f 2`
myartist=`echo "$prb" | grep -E 'artist\s+:' | cut -d ':' -f 2`
myTitle=`echo "$prb" | grep -E 'title\s+:' | cut -d ':' -f 2 | sed 's/\//_/g'`
cwd_stub=`pwd | awk -F'/' '{print $NF}'`
if [ -d "s${myDate:1:4}" ]; then echo "Directory found" > /dev/null; else mkdir "s${myDate:1:4}"; fi
[ -d "s${myDate:1:4}" ] && mv -- "$i" "s${myDate:1:4}/${myartist[#]:1} - s${myDate:1:4}e${myDate:5:8} - ${myTitle[#]:1:40} _$i" || mv -- "$i" "${myartist[#]:1} - s${myDate:1:4}e${myDate:5:8} - ${myTitle[#]:1:40} _$i"
done
How can I modify that script to identify if a conflicting year/MMDD file exists, and if so, append an appropriate suffix to the episode number so that plex will interpret them as distinct episodes?
I ended up implementing an array, counting the number of elements in the array, and using that to append the integer:
#!/bin/bash
mySuff="$1"
echo mySuff="$mySuff"
if [ -z "$1" ]; then
mySuff="*.mp4"
fi
for i in $mySuff
do
prb=`ffprobe -- "$i" 2>&1`
myDate=`echo "$prb" | grep -E 'date\s+:' | cut -d ':' -f 2`
myartist=`echo "$prb" | grep -E 'artist\s+:' | cut -d ':' -f 2`
myTitle=`echo "$prb" | grep -E 'title\s+:' | cut -d ':' -f 2 | sed 's/\//_/g'`
cwd_stub=`pwd | awk -F'/' '{print $NF}'`
readarray -t conflicts < <(find . -maxdepth 2 -iname "*s${myDate:1:4}e${myDate:5:8}*" -type f -printf '%P\n')
[ ${#conflicts[#]} -gt 0 ] && _inc=${#conflicts[#]} || _inc=
if [ -d "s${myDate:1:4}" ]; then echo "Directory found" > /dev/null; else mkdir "s${myDate:1:4}"; fi
[ -d "s${myDate:1:4}" ]
&& mv -- "$i" "s${myDate:1:4}/${myartist[#]:1} - s${myDate:1:4}e${myDate:5:8}$_inc - ${myTitle[#]:1:40} _$i"
|| mv -- "$i" "${myartist[#]:1} - s${myDate:1:4}e${myDate:5:8}$_inc - ${myTitle[#]:1:40} _$i"
done

Bash filename expansion identifies items in file tree, called command not

..poky/build$ for SUBPATH in $(bitbake -e alsa-lib | grep -P -e '(?<=^)FILES_alsa-lib(?==)' | cut -d= -f2 | tr -d \") ; do ls ./tmp-glibc/work/armv7a-vfp-neon-oe-linux-gnueabi/alsa-lib/1.0.29-r0/package$SUBPATH 2>&1 ; done | grep -e "No such file or directory" | wc -l
2855
..poky/build$ for SUBPATH in $(bitbake -e alsa-lib | grep -P -e '(?<=^)FILES_alsa-lib(?==)' | cut -d= -f2 | tr -d \") ; do ls ./tmp-glibc/work/armv7a-vfp-neon-oe-linux-gnueabi/alsa-lib/1.0.29-r0/package$SUBPATH 2>&1 ; done | grep -v -e "No such file or directory" | wc -l
15
Here one of all those No such file or directory
ls: cannot access ./tmp-glibc/work/armv7a-vfp-neon-oe-linux-gnueabi/alsa-lib/1.0.29-r0/package/usr/lib/libicalss.so.1.0.0: No such file or directory
where
..poky/build$ bitbake -e alsa-lib | grep -P -e '(?<=^)FILES_alsa-lib(?==)' | cut -d= -f2 | tr -d \"
/usr/bin/* /usr/sbin/* /usr/lib/alsa-lib/* /usr/lib/lib*.so.* /etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d /usr/lib/udev/rules.d /usr/share/alsa-lib /usr/lib/alsa-lib/* /usr/share/pixmaps /usr/share/applications /usr/share/idl /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers /usr/lib/alsa-lib/smixer/*.so
and
poky/build$ echo $SHELL
/bin/bash
Apparently Bash file name expansion finds 2855 items in identified sub-paths the called ls command can't identify.
Actually in every iteration instead of ls ... I need to do find with search root point set to ./tmp-glibc/work/armv7a-vfp-neon-oe-linux-gnueabi/alsa-lib/1.0.29-r0/package$SUBPATH and -nameargument set few times (logical OR) to some patterns.
Where is my mistake?
Is this that file name expansion takes place in the for-loop instead of on invoking ls command (as programmer wishes it)?
#
Following alternative found under my limited expertise level and time resources
cd ./tmp-glibc/work/armv7a-vfp-neon-oe-linux-gnueabi/alsa-lib/1.0.29-r0/package && echo "/usr/bin/* /usr/sbin/* /usr/lib/alsa-lib/* /usr/lib/lib*.so.* /etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d /usr/lib/udev/rules.d /usr/share/alsa-lib /usr/lib/alsa-lib/* /usr/share/pixmaps /usr/share/applications /usr/share/idl /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers" | sed -r 's/(^\/)/.\//g' | sed -r 's/( \/)/ .\//g' | ls -Ralh $(awk '{print $0}') ; cd -
Glue for pasting long string with sub-paths to this command pipe from building block presented earlier out of scope as for this Q.
If possible please review. Thanks.
Does this qualify to be this Q's answer?

Need to remove the extra empty lines from the output of shell script

i'm trying to write a code which will print all files taking more than min_size (lets say 10G) in a directory. the problem is output off the below code is all files irrespective of the min_size. i will be getting other details like mtime , owner as well later in the code but this part itself doesnt work fine, whats wrong here ?
#!/bin/sh
if (( $# <3 )); then
echo "$0 dirname min_size count"
exit 1
else
dirname="$1";
min_size="$2";
count="$3";
#shift 3
fi
tmpfile=$(mktemp /lawdump/pulkit/files.XXXXXX)
exec 3> "$tmpfile"
find "${dirname}" -type f -print0 2>&1 | grep -v "Permission denied" | xargs -0 -I {} echo "{}" > "$tmpfile"
for i in `cat tmpfile`
do
x="`du -ah $i | awk '{print $1}' | grep G | sort -nr -k 1`"
size=$(echo $x | sed 's/[A-Za-z]*//g')
if [ size > $min_size ];then
echo $size
fi
done
Note : i know this can be done through find or du but i need to write a shell script to have an email sent out regularly with all the details.

Is there any command to step down one directory in shell?(when there is only one sub-directory)

To go up one directory I write cd ..
Is there any command that would work for the reverse situation in which there is only one subdirectory?
Let's say I am in:
dir1/dir2/
dir2 has only one subdirectory dir3
Is there any short cut to step down one directory from dir2 to dir3 without writing the name of subdirectory(dir3)?
There isn't such command per se, but you can trick the cd command by typing cd */ ;-)
At the time of my question I was not aware of shells's auto-complete with Tab key.
In this scenario I just type cd, press Tab and the name of the directory shows up so I can press Enter to move to the directory.
I had a similar thought when I was learning shell and wrote a wrapper around cd that does what you want. It grew into something a bit more complicated. If you have folders called folder1 and folder2 you can type: cdd 2
If there is only one folder you can just type: cdd
It also has a similar functionality to ksh's cd paths substitution with 2 args (if at /home/tom, you can type the following to get to /home/bob: cdd tom bob).
It also works like the normal cd command if you pass a folder that exists.
It was written a while ago so it might not be the prettiest, but it works.
It also does an ls at the end which you can remove.
One other thing to note is you can (in bash at least) type the following to go the the previous directory you were in: cd -
function cdd()
{
if [[ $3 != "" ]]; then
printf "~~~ cdd can only take 1 or 2 arguments, you specified 3 or more\n";
return;
else
if [[ $2 != "" ]]; then
ARG=$(pwd | sed "s/$1/$2/g");
cd $ARG;
else
if [[ $1 == "" ]]; then
cd $(ls -d */ | head -1);
else
if [[ -d $1 ]]; then
cd $1;
else
if [[ -d $(ls -F | grep "/$" | grep "^$1" | head -1) ]]; then
cd $(ls -F | grep "/$" | grep "^$1" | head -1);
else
if [[ -d $(ls -F | grep "/$" | grep "$1/$" | head -1) ]]; then
cd $(ls -F | grep "/$" | grep "$1/$" | head -1);
else
if [[ -d $(ls -F | grep "/$" | grep "$1" | head -1) ]]; then
cd $(ls -F | grep "/$" | grep "$1" | head -1);
else
if [[ -d $(ls -a -F | grep "/$" | grep "$1" | head -1) ]]; then
cd $(ls -a -F | grep "/$" | grep "$1" | head -1);
else
printf "~~~ Folder not found...\n";
return 3;
fi;
fi;
fi;
fi;
fi;
fi;
fi;
fi;
if [[ $? == 0 ]]; then
ls --color=auto -a --color;
fi
}

Find duplicated nested directories

I have a large directory tree with this nested directories duplicates (but not all):
data/home/home/
data/banners/banners/
resources/users/documents/documents/
How can I merge only duplicated directories with this actions:
copy (without replace) data/home/home/ contents to data/home/
delete data/home/home
My current code:
#/bin/bash
for folder in $(find httpdocs -type d); do
n=$(echo $folder | tr "/" "\n" | wc -l)
nuniq=$(echo $folder | tr "/" "\n" | sort | uniq | wc -l)
[ $n -eq $nuniq ] || echo "Duplicated folder $folder"
done
But have a problem, because data/home/es/home is a valid folder, but detected as duplicated.
Thanks.
you can use uniq command as below;
#/bin/bash
for folder in $(find httpdocs -type d); do
nuniq=$(echo $folder | tr "/" "\n" | uniq -d | wc -l)
if [ "$nuniq" -gt "0" ]
then
echo "Duplicated folder $folder"
fi
done
man uniq;
-d, --repeated
only print duplicate lines
you can try the following script for copy and delete folder. I can not test this, so take a backup your httpdocs folder before run this.
#/bin/bash
for folder in $(find httpdocs -type d); do
nuniq=$(echo $folder | tr "/" "\n" | uniq -d | wc -l)
if [ "$nuniq" -gt "0" ]
then
dest=$(echo $folder | tr '/' '\n' | awk '!a[$0]++' | tr '\n' '/')
mv -i $folder/* $dest
rmdir $folder
fi
done
For example;
user#host $ echo "data/home/es/home" | tr "/" "\n"
data
home
es
home
user#host $ echo "data/home/es/home" | tr "/" "\n" | uniq -d | wc -l
0
user#host $ echo "data/home/home" | tr "/" "\n"
data
home
home
user#host $ echo "data/home/home" | tr "/" "\n" | uniq -d
home
user#host $ echo "data/home/home" | tr "/" "\n" | uniq -d | wc -l
1

Resources