How can I write a bash script on Linux to determine which files in two directories have different permissions?
For example, I have two directories:
fold1 having two files:
1- file1 (-rw-rw-r--)
2- file2 (-rw-rw-r--)
fold2 having same-name files with different permissions:
1- file1 (-rwxrwxr-x)
2- file2 (-rw-rw-r--)
I need a script to output the file names that have different permissions,
so the script will print only file1
I am currently checking the permissions manually by displaying the files with:
for i in `find .`; do ls -l $i ls -l ../file2/$i; done
Parsing find . output with: for i in $(find .) is going to give you trouble for any filenames with spaces, newlines, or other perfectly normal characters:
$ touch "one file"
$ for i in `find .` ; do ls -l $i ; done
total 0
-rw-r--r-- 1 sarnold sarnold 0 2012-02-08 17:30 one file
ls: cannot access ./one: No such file or directory
ls: cannot access file: No such file or directory
$
Since permissions can also differ by owner or by group, I think you should include those as well. If you need to include the SELinux security label, the stat(1) program makes that easy to get as well via the %C directive:
for f in * ; do stat -c "%a%g%u" "$f" "../scatman/${f}" |
sort | uniq -c | grep -q '^\s*1' && echo "$f" is different ; done
(Do whatever you want for the echo command...)
Example:
$ ls -l sarnold/ scatman/
sarnold/:
total 0
-r--r--r-- 1 sarnold sarnold 0 2012-02-08 18:00 funky file
-rw-r--r-- 1 sarnold sarnold 0 2012-02-08 18:01 second file
-rw-r--r-- 1 root root 0 2012-02-08 18:05 third file
scatman/:
total 0
-rw-r--r-- 1 sarnold sarnold 0 2012-02-08 17:30 funky file
-rw-r--r-- 1 sarnold sarnold 0 2012-02-08 18:01 second file
-rw-r--r-- 1 sarnold sarnold 0 2012-02-08 18:05 third file
$ cd sarnold/
$ for f in * ; do stat -c "%a%g%u" "$f" "../scatman/${f}" | sort | uniq -c | grep -q '^\s*1' && echo "$f" is different ; done
funky file is different
third file is different
$
Related
I have a file called samples.list with sample IDs. I have same files in my directory that I want to pattern match with my sample.list and get the output of unmatched sample.list.
samples.list
SRR1369385
SRR1352799
SRR1377262
SRR1400622
ls -lh
-rw-rw----+ 1 gen dbgap_6109 2.2G Jul 29 02:44 SRR1369385_1.fastq.gz
-rw-rw----+ 1 gen dbgap_6109 2.2G Jul 29 02:44 SRR1369385_2.fastq.gz
-rw-rw----+ 1 gen dbgap_6109 1.2G Jul 29 03:34 SRR1352799_1.fastq.gz
-rw-rw----+ 1 gen dbgap_6109 1.2G Jul 29 03:34 SRR1352799_2.fastq.gz
-rw-rw----+ 1 gen tnt_pipeli 2.2G Jul 29 01:44 sometxt.txt
The output I want (samples that did not match with the file names in the directory):
SRR1377262
SRR1400622
code I tried:
grep -oFf `cat samples.list` ls -lh | grep -vFf - `cat samples.list`
I would really appreciate if someone could guide me through the solution.
# find all files named in the way you want and print filenames
find . -maxdepth 1 -type f -name '*_*.fastq.gz' -printf "%f\n" |
# Remove all everything except the SRR=numbers
sed 's/_.*//' |
# Sort the list, remove duplicate elements
sort -u |
# join the list with samples and print only unmatched elements from samples
join -v1 -o 1.1 <(sort samples.list) -
Tested on repl.
Notes:
do not use backticks `. Prefer $(...) instead. obsolete and deprecated syntax bashhackers wiki
greps -f options takes filename not content of the file. You could do grep -f some_file.txt to grep with all regexes stored in some_file.txt.
ls -lh produces output to stdout. Running grep ls -lh would make grep want to search file named ls (and what for -l and -h if you want to search filenames?). While you could ls -1 | grep, but it's better to find . -maxdepth 1 -mindepth 1 | grep ...
Try this:
awk -F_ 'NR==FNR{a[$1]=1;next}!($0 in a)' <(ls) samples.list
First that will index everything until _ from ls for each output line (NR==FNR is true for these lines), and then find all unmatched lines in samples.list (»if a line is not indexed, print it«).
I have two directory which name are very strange as below:
"?????+ "BizComponent
"?+ "BizComponent
and now I try to remove them but failed. when I type rm -r ' then type tab it give me:
"^J^I^I^I^I+ "BizComponent/BizComponent/
"^J+ "BizComponent/
And then, when I type rm -r "^J^I^I^I^I+ "BizComponent/BizComponent/ it gives me No such file or directory
The problem is that your filename contains a special character, but this is not necessarily the character you see. Example (taken from here):
$ touch zzz yyy $'zzz\nyyy'
$ ls
yyy zzz zzz?yyy
As you see, a filename with a new line was created, but ls printed it as a ?. So how do we remove this?
method 1:
An option available to ls is --quoting-style=shell-escape, this allows you to see how to type the filename for removal (works on files and directories):
$ ls --quoting-style=shell-escape
yyy zzz 'zzz'$'\n''yyy'
$ rm 'zzz'$'\n''yyy'
$ ls
yyy zzz
method 2: The second method is to use the inode number using find (works on files and directories):
$ touch zzz yyy $'zzz\nyyy'
$ ls -li
total 2
3886009 -rw-r--r-- 1 user group 0 Jul 23 12:56 yyy
3886008 -rw-r--r-- 1 user group 0 Jul 23 12:56 zzz
662083 -rw-r--r-- 1 user group 0 Jul 23 12:56 zzz?yyy
$ find . -inum 662083 -delete
$ ls -li
total 1
3886009 -rw-r--r-- 1 user group 0 Jul 23 12:56 yyy
3886008 -rw-r--r-- 1 user group 0 Jul 23 12:56 zzz
Looks like you have some special characters in your directory names. You could use a loop with a glob:
for dir in *BizComponent; do
rm -r -- "$dir"
done
Due to the shell expansion you need to escape the special characters ? + " ^ and space. So to reference your files, you need to remove/reference them likes this: rm -rf \"\?\?\?\?\?+\ \"BizComponent/
rm -rf \"\?\+\ \"BizComponent rm -rf \"\^J\^I\^I\^I\^I+\ \"BizComponent/BizComponent/
and so on.
Why don't you try rm -rf "?????+ BizComponent" | echo $?
I am trying to output the number of directories in a given path on a SINGLE line. My desire is to output this:
X-many directories
Currently, with my bash sript, I get this:
X-many
directories
Here's my code:
ARGUMENT=$1
ls -l $ARGUMENT | egrep -c '^drwx'; echo -n "directories"
How can I fix my output? Thanks
I suggest
echo "$(ls -l "$ARGUMENT" | egrep -c '^drwx') directories"
This uses the shell's feature of final newline removal for command substitution.
Do not pipe to ls output and count directories as you can get wrong results if special characters have been used in file/directory names.
To count directories use:
shopt -s nullglob
arr=( "$ARGUMENT"/*/ )
echo "${#arr[#]} directories"
/ at the end of glob will make sure to match only directories in "$ARGUMENT" path.
shopt -s nullglob is to make sure to return empty results if glob pattern fails (no directory in given argument).
as alternative solution
$ bc <<< "$(find /etc -maxdepth 1 -type d | wc -l)-1"
116
another one
$ count=0; while read curr_line; do count=$((count+1)); done < <(ls -l ~/etc | grep ^d); echo ${count}
116
Would work correctly with spaces in the folder name
$ ls -la
total 20
drwxrwxr-x 5 alex alex 4096 Jun 30 18:40 .
drwxr-xr-x 11 alex alex 4096 Jun 30 16:41 ..
drwxrwxr-x 2 alex alex 4096 Jun 30 16:43 asdasd
drwxrwxr-x 2 alex alex 4096 Jun 30 16:43 dfgerte
drwxrwxr-x 2 alex alex 4096 Jun 30 16:43 somefoler with_space
$ count=0; while read curr_line; do count=$((count+1)); done < <(ls -l ./ | grep ^d); echo ${count}
3
I have a directory of files:
/home/user/files/1.txt
/home/user/files/2.txt
/home/user/files/3.txt
I'd like to zip up the files directory into files.zip so when extracted I get:
files/1.txt
files/2.txt
files/3.txt
I know I can do:
# bash
cd /home/user; zip -r files.zip files/
Is there a way to do this without cding to the user directory?
I know that the --junk-paths flag will store just the filenames and junk the path but I'd like to keep the files directory as a container.
Couldn't find direct way using zip command but you can try "tar" command with -C option.
$ pwd
/home/shenzi
$ ls -l giga/files
total 3
-rw-r--r-- 1 shenzi Domain Users 3 Aug 5 11:24 1.txt
-rw-r--r-- 1 shenzi Domain Users 4 Aug 5 11:25 2.txt
-rw-r--r-- 1 shenzi Domain Users 9 Aug 5 11:25 3.txt
$ tar -C giga -cvf files.zip files/*
files/1.txt
files/2.txt
files/3.txt
$ tar -tvf files.zip
-rw-r--r-- shenzi/Domain Users 3 2014-08-05 11:24 files/1.txt
-rw-r--r-- shenzi/Domain Users 4 2014-08-05 11:25 files/2.txt
-rw-r--r-- shenzi/Domain Users 9 2014-08-05 11:25 files/3.txt
USE: -xvf to extract
I have a ruby one-liner ruby1.9 -ine '#some statement' src/**. I assumed, like perl does, ruby skips the directories ( well that's how I remember it ). But I get this error e:1:in 'gets': Is a directory. Besides giving it a list of files, is there a quick way of getting round this?
I don't think it ever skipped directories, at least even 1.8.6 does not. So I suppose the only quick way is to give it a list of files, or to manipulate ARGV, but it'd hardly be a proper one-liner then anymore.
Something like this:
ruby -ne 'ARGV.delete_if{|s| File.ftype(s) == "directory"}; do_stuff_here' src/**
So yeah, giving it a proper file list seems to be the nicer solution.
If you want only files recursively, then find(1) will be your best bet :
find ./src -type f | ruby1.9 -ne '#some statement'
I believe your assumption is actually about your shell's handling of **. The shell interprets un-escaped meta-characters in commands.
bash(1) by default will not expand ** recursively. You need to set the globstar option for this behavior:
$ ls -l /tmp
total 20
drwx------ 2 sarnold sarnold 4096 2011-11-17 15:43 keyring-9mdW7p
drwx------ 2 gdm gdm 4096 2011-11-17 15:43 orbit-gdm
drwx------ 2 sarnold sarnold 4096 2011-11-17 15:44 orbit-sarnold
drwx------ 2 sarnold sarnold 4096 2011-11-17 15:46 plugtmp
drwx------ 2 sarnold sarnold 4096 2011-11-17 15:43 ssh-ZriaCoWL2248
$ shopt -u globstar
$ echo /tmp/**
/tmp/keyring-9mdW7p /tmp/orbit-gdm /tmp/orbit-sarnold /tmp/plugtmp /tmp/ssh-ZriaCoWL2248
$ shopt -s globstar
$ echo /tmp/**
/tmp/ /tmp/keyring-9mdW7p /tmp/keyring-9mdW7p/control /tmp/orbit-gdm /tmp/orbit-sarnold /tmp/orbit-sarnold/linc-9a5-0-240e051029b41 /tmp/orbit-sarnold/linc-9ad-0-1b1412421b16c /tmp/plugtmp /tmp/ssh-ZriaCoWL2248 /tmp/ssh-ZriaCoWL2248/agent.2248