Why extglob except breaking except condition? - bash

I run in Debian 8.1, GNU bash, version 4.3.30(1)-release (x86_64-pc-linux-gnu) and Lenovo G50-30 with 500 GB SSD and 8 GB flash memory:
shopt -s extglob
cp -r !(Backups.backupdb/) /home/masi/Documents/
but it will copy also everything from the directory Backups.backupdb/, confirmed at the end of copying.
Messages during the copying process
After 2h copying
cp: cannot stat ‘Backups.backupdb/masi\’s MacBook Air/2015-06-25-233115/Macintosh HD/System/Library/Image Capture/Automatic Tasks/MakePDF.app/Contents/Resources/ko.lproj/3x5로 자르기.mkpdf’: No such file or directory
cp: cannot stat ‘Backups.backupdb/masi\’s MacBook Air/2015-06-25-233115/Macintosh HD/System/Library/Image Capture/Automatic Tasks/MakePDF.app/Contents/Resources/ko.lproj/3x5에 맞추기.mkpdf’: No such file or directory
...
cp: cannot stat ‘Camera Uploads/2015-06-29 11.51.36.jpg’: Invalid argument
cp: cannot stat ‘Camera Uploads/2015-06-29 11.51.53.jpg’: Invalid argument
cp: cannot stat ‘Camera Uploads/Icon\r’: Invalid argument
cp: cannot stat ‘Cancer’: Invalid argument
cp: cannot stat ‘cardio bad/atria-en-ventrikels.swf’: Invalid argument
cp: cannot stat ‘cardio bad/extreme_90_180.swf’: Invalid argument
cp: cannot stat ‘Cardio databases tools’: Invalid argument
cp: cannot stat ‘Cardiology’: Invalid argument
...
where I keep unexpected browsing through the backup folder.
The challenge here is that the Backups.backupdb is several TBs and seeing that it goes browsing it does not feel good.
No symbolic links
I run
ls -lR /media/masi/eb807ed8-7f45-35f8-a345-9da6692b228a/ |grep Backup
getting
ls: cannot access /media/masi/eb807ed8-7f45-35f8-a345-9da6692b228a/Cancer: Invalid argument
ls: cannot access /media/masi/eb807ed8-7f45-35f8-a345-9da6692b228a/Cardio databases tools: Invalid argument
ls: cannot access /media/masi/eb807ed8-7f45-35f8-a345-9da6692b228a/Cardiology: Invalid argument
drwxr-xr-x 1 root root 7 Jul 8 20:25 Backups.backupdb
ls: cannot open directory /media/masi/eb807ed8-7f45-35f8-a345-9da6692b228a/animations/Embryology/e17_files: Permission denied
^C
so the directory is not a symlink.
Why is extglob except condition broken here by browsing the folder?

Try removing the backslash (I am adding echo for test purposes):
shopt -s extglob
echo cp -r !(Backups.backupdb) /home/masi/Documents/
It works in my environment (GNU bash, version 4.3.30, OS X)

Related

Cannot remove symbolic link to directory

I have a directory reference in my Downloads directory that contains a symbolic link (created with ln -s) to another directory. I get conflicting error message when trying to remove the symlink:
rm returns "Is a directory"
rmdir returns "Not a directory"
This only occurs with cellranger/ (followed by a forward slash) and not with cellranger.
[tom#rlgsw68 cellranger]$ pwd
/home/tom/Downloads/reference
[tom#rlgsw68 cellranger]$ ls -lth
lrwxrwxrwx 1 tom genome 33 Apr 4 14:52 cellranger -> /analysisdata/genomes/cellranger/
[tom#rlgsw68 cellranger]$ rm cellranger/
rm: cannot remove directory `cellranger/': Is a directory
[tom#rlgsw68 cellranger]$ rmdir cellranger/
rmdir: cellranger/: Not a directory
[tom#rlgsw68 cellranger]$ rm cellranger
Why does neither of these commands to remove the symlink work and why do these conflicting errors occur? What is the recommended way to remove symbolic links without removing the content in the source directory. rm -rf cellranger/ also does not remove the symlink (but does not return an error).
Information: I'm running a linux server (Debian 9.0). These errors occur with both bash and zsh. Ambiguous names have been removed from the example. I encountered this when a directory included a link to the parent directory in addition to the contents:
/home/tom/Downloads/reference/cellranger/cellranger/ -> /analysisdata/genomes/cellranger/
By putting a trailing slash you are referring to the directory the symlink points to, no longer the symlink itself. Printing the inode number (the number that a path refers to in the file system) shows the difference between dereferencing the symlink and the directory:
$ cd "$(mktemp --directory)"
$ mkdir a
$ stat --format %i a/
9
$ ln --symbolic a b
$ stat --format %i b
10
$ stat --format %i b/
9
This may be related to the fact that a symlink is never a directory, it is always just a file containing a path.

My loop bash command doesn't work

I'm trying to run a command called bedtools shuffle on a file 5 times and save each run as with its counter after the filename (i.e. SHUFFLE1... SHUFFLE 5).
Here is my code but I keep getting the error message below:
for i in {1..5}; do bedtools shuffle -i '/PATH/wgEncodeBroadHistoneOsteoblH3k27acStdPk.broadPeak_use' -g '/PATH/chrom.sizes'> rm -f '/PATH/wgEncodeBroadHistoneOsteoblH3k27acStdPk.broadPeak_use_SHUFFLED${i}.bed'; done
-bash: rm: cannot overwrite existing file
-bash: rm: cannot overwrite existing file
-bash: rm: cannot overwrite existing file
-bash: rm: cannot overwrite existing file
-bash: rm: cannot overwrite existing file
Please advise!

How to delete a file with a "Symbol for NULL" character (0x2400) in filename

ATTN: shell gods ;)
I can't seem to figure out how to delete a file with a unicode character 0x2400 in the filename on OSX (Example: ␀.test).
It's not a NULL character per-se, but a "symbol for null". (See: http://unicodelookup.com/#null/1)
Script - How to Reproduce
#!/usr/bin/env bash
dir="${HOME}/test_dir"
# Create Directory: ~/test_dir
if [ ! -d "${dir}" ]; then
printf "\nCreating Directory: ${dir}\n"
mkdir ${dir}
fi
# Create our character
char=$'\xE2\x90\x80'
# Create filename
file="${dir}/${char}.test"
# Create the File
printf "\nCreating File: ${file}\n"
touch ${file}
Delete the file... NOPE!
# Attempt 1 - Delete File
printf "\nDeleting File: ${file}\n"
rm -rf ${file}
Delete the whole directory... NOPE!
# Attempt 2 - Delete Directory
printf "\nDeleting Directory: ${file}\n"
rm -rf ${dir}
Delete the file via inode... NOPE!
# Attempt 3 - Delete File
inode=$(ls -i1 ${dir} | awk '{print $1}')
printf "\nDeleting via Inode: ${inode}\n"
find ${dir} -inum ${inode} -exec rm -i {} \;
The whole script should output something like this:
Creating File: /Users/bsmith/test_dir/␀.test
Deleting File: /Users/bsmith/test_dir/␀.test
rm: /Users/bsmith/test_dir/␀.test: Invalid argument
Deleting Directory: /Users/bsmith/test_dir/␀.test
rm: /Users/bsmith/test_dir/␀.test: Invalid argument
rm: /Users/bsmith/test_dir: Directory not empty
Deleting via Inode: 68592933
remove /Users/bsmith/test_dir/␀.test? y
rm: /Users/bsmith/test_dir/␀.test: Invalid argument
This command works for me:
rm ?.test
But sadly it is very probable that it will NOT work for you.
It is a known bug of osx:
Is it impossible to delete/move a file named “␀” on mac?
Rename folder with odd characters
The sure bet is to boot from a pen drive with some Linux OS, mount the file system in such Linux, and erase the file. It is sure that files with such names could be erased in Linux.

File globbing without word splitting?

This is a simplified example to hopefully illustrate my problem.
I have a script that takes a parameter to be used as a wildcard. Sometimes this wildcard contains whitespace. I need to be able to use the wildcard for globbing, but word splitting is causing it to fail.
For example, consider the following example files:
$ ls -l "/home/me/dir with whitespace"
total 0
-rw-r--r-- 1 me Domain Users 0 Jun 25 16:58 file_a.txt
-rw-r--r-- 1 me Domain Users 0 Jun 25 16:58 file_b.txt
My script - simplified to use a hard coded pattern variable - looks like this:
#!/bin/bash
# Here this is hard coded, but normally it would be passed via parameter
# For example: pattern="${1}"
# The whitespace and wildcard can appear anywhere in the pattern
pattern="/home/me/dir with whitespace/file_*.txt"
# First attempt: without quoting
ls -l ${pattern}
# Result: word splitting AND globbing
# ls: cannot access /home/me/dir: No such file or directory
# ls: cannot access with: No such file or directory
# ls: cannot access whitespace/file_*.txt: No such file or directory
####################
# Second attempt: with quoting
ls -l "${pattern}"
# Result: no word splitting, no globbing
# ls: cannot access /home/me/dir with whitespace/file_*.txt: No such file or directory
Is there a way to enable globbing, but disable word splitting?
Do I have any options except manually escaping whitespace in my pattern?
Don't keep glob inside the quote to be able to expand it:
pattern="/home/me/dir with whitespace/file_"
ls -l "${pattern}"*
EDIT:
Based on edited question and comment you can use find:
find . -path "./$pattern" -print0 | xargs -0 ls -l
I finally got it!
The trick is modifying the internal field separator (IFS) to be null. This prevents word splitting on unquoted variables until IFS is reverted to its old value or until it becomes unset.
Example:
$ pattern="/home/me/dir with whitespace/file_*.txt"
$ ls -l $pattern
ls: cannot access /home/me/dir: No such file or directory
ls: cannot access with: No such file or directory
ls: cannot access whitespace/file_*.txt: No such file or directory
$ IFS=""
$ ls -l $pattern
-rw-r--r-- 1 me Domain Users 0 Jun 26 09:14 /home/me/dir with whitespace/file_a.txt
-rw-r--r-- 1 me Domain Users 0 Jun 26 09:14 /home/me/dir with whitespace/file_b.txt
$ unset IFS
$ ls -l $pattern
ls: cannot access /home/me/dir: No such file or directory
ls: cannot access with: No such file or directory
ls: cannot access whitespace/file_*.txt: No such file or directory
I found out the hard way that you cannot set and use IFS with ls. For example, this doesn't work:
$ IFS="" ls -l $pattern
This is because the command has already undergone word splitting before IFS changes.

blank space in directory name causing error

Hi I have many directories which have blank space in their names.I have some scripts to be run on them.
Here is an example script
#!/bin/bash
docid='/home/deel/PDF/rant/spr 2008 Shree rose Visheshank'
p=1
while [ $p -lt 117 ]; do
cp $docid/$p.pdf ./
p=$p+1
done
Upon executing the above script I get following errors
cp: cannot stat `/home/deel/PDF/rant/spr': No such file or directory
cp: cannot stat `2008': No such file or directory
cp: cannot stat `Shree': No such file or directory
cp: cannot stat `rose': No such file or directory
cp: cannot stat `Visheshank/1.pdf': No such file or directory
./d2.sh: line 5: [: 1+1: integer expression expected
I have tried using double quotes "" but the results are same.
What changes should I do?
This line:
cp $docid/$p.pdf ./
should read:
cp "$docid/$p.pdf" ./
Example:
$ a='abc def'
$ ls $a
ls: abc: No such file or directory
ls: def: No such file or directory
$ ls "$a"
ls: abc def: No such file or directory
And arithmetic goes like:
p=$[p+1]
Example:
$ p=1
$ q=$[p+1]
$ echo $p $q
1 2
You need to quote the name containing spaces with double quotes. Moreover, p=$p+1 does not do what you expect. Here is the corrected loop:
for p in `seq 116`; do
cp "$docid/$p.pdf" ./
done

Resources