hopefully a simple question and the last piece in my puzzle... :-) I have a shell script running in terminal under os x. It contains among other things:
name=$(basename "$file")
printf "%s" "\"$name\";"
... which is fine ... but lets say that the file name contains a double quote - IMA"G09%'27.jpg - then the output would be:
"IMA"G09%'27.jpg;"
... and that would "break" my line intended for putting into a db later (the double quote). So I need to escape it so I get the output:
"IMA\"G09%'27.jpg;"
... but I couldn't figure out how ... anyone? :-)
EDIT - RESULT: With the help of anubhava this is what I use (to get file info incl. type/creator):
#!/bin/bash
find . -type f -name '*' -print0 | while IFS= read -r -d '' file
do
name=$(basename "$file")
path=$(dirname "$file")
# full_path=$(readlink -f "$file") # This only works on Linux
full_path=$(echo "$PWD/${file#./}")
extension=${name##*.}
size_human_readable=$(ls -lh "$file" | awk -F' ' '{print $5}')
size_in_bytes=$(stat -f "%z" "$file")
creation_date=$(stat -f "%SB" "$file")
last_access=$(stat -f "%Sa" "$file")
last_modification=$(stat -f "%Sm" "$file")
last_change=$(stat -f "%Sc" "$file")
creator=$(mdls -name kMDItemFSCreatorCode "$file")
printf "\"%q\";" "$name"
printf "%s" "\"$full_path\";"
printf "%s" "\"$extension\";"
printf "\"$size_human_readable\";"
printf "\"$size_in_bytes\";"
printf "\"$last_modification\";"
printf "%s" "\"$creator\""
printf "\n"
done
Using printf with %q:
name='file"naeme.txt'
printf "\"%q;\"" "$name"
"file\"naeme.txt;"
Here's another approach, using sed to control the escaping:
printquoted() {
printf '"%s";' "$(LC_ALL=C sed 's/["]/\\&/g' <<<"$1")"
}
printquoted "$name"
printquoted "$full_path"
printquoted "$extension"
...etc
If it turns out there are other things you need to escape besides double-quotes (like, say, backslashes themselves), you can add them to the sed [] expression (e.g. ["\] would escape double-quotes and backslashes).
Note that this approach will fail horribly if the string contains any newlines (which is legal in filenames).
Related
In a bash script I have a variable containing a shell glob expression that I want to expand into an array of matching file names (nullglob turned on), like in
pat='dir/*.config'
files=($pat)
This works nicely, even for multiple patterns in $pat (e.g., pat="dir/*.config dir/*.conf), however, I cannot use escape characters in the pattern. Ideally, I would like to able to do
pat='"dir/*" dir/*.config "dir/file with spaces"'
to include the file *, all files ending in .config and file with spaces.
Is there an easy way to do this? (Without eval if possible.)
As the pattern is read from a file, I cannot place it in the array expression directly, as proposed in this answer (and various other places).
Edit:
To put things into context: What I am trying to do is to read a template file line-wise and process all lines like #include pattern. The includes are then resolved using the shell glob. As this tool is meant to be universal, I want to be able to include files with spaces and weird characters (like *).
The "main" loop reads like this:
template_include_pat='^#include (.*)$'
while IFS='' read -r line || [[ -n "$line" ]]; do
if printf '%s' "$line" | grep -qE "$template_include_pat"; then
glob=$(printf '%s' "$line" | sed -nrE "s/$template_include_pat/\\1/p")
cwd=$(pwd -P)
cd "$targetdir"
files=($glob)
for f in "${files[#]}"; do
printf "\n\n%s\n" "# FILE $f" >> "$tempfile"
cat "$f" >> "$tempfile" ||
die "Cannot read '$f'."
done
cd "$cwd"
else
echo "$line" >> "$tempfile"
fi
done < "$template"
Using the Python glob module:
#!/usr/bin/env bash
# Takes literal glob expressions on as argv; emits NUL-delimited match list on output
expand_globs() {
python -c '
import sys, glob
for arg in sys.argv[1:]:
for result in glob.iglob(arg):
sys.stdout.write("%s\0" % (result,))
' _ "$#"
}
template_include_pat='^#include (.*)$'
template=${1:-/dev/stdin}
# record the patterns we were looking for
patterns=( )
while read -r line; do
if [[ $line =~ $template_include_pat ]]; then
patterns+=( "${BASH_REMATCH[1]}" )
fi
done <"$template"
results=( )
while IFS= read -r -d '' name; do
results+=( "$name" )
done < <(expand_globs "${patterns[#]}")
# Let's display our results:
{
printf 'Searched for the following patterns, from template %q:\n' "$template"
(( ${#patterns[#]} )) && printf ' - %q\n' "${patterns[#]}"
echo
echo "Found the following files:"
(( ${#results[#]} )) && printf ' - %q\n' "${results[#]}"
} >&2
I am trying to write a bash script that convert all file names to lowercase, but I have a problem because it does not work for one case.
When you have your file1 and FILE1, and you will use it on the FILE1 it will replace letters file1.
#!/bin/bash
testFILE=""
FLAG="1"
for FILE in *
do
testFILE=`echo FILE | tr '[A-Z]' '[a-z]'`
for FILE2 in *
do
if [ `echo $testFILE` = `echo $FILE2` ]
then
FLAG="0"
fi
done
if [ $FLAG = "1" ]
then
mv $FILE `echo $FILE | tr '[A-Z]' '[a-z]'`
fi
FLAG="1"
done
Looks like
testFILE=`echo FILE | tr '[A-Z]' '[a-z]'`
should be
testFILE=`echo "$FILE" | tr '[A-Z]' '[a-z]'`
Re-writing your script to fix some other minor things
#!/bin/bash
testFILE=
FLAG=1
for FILE in *; do
testFILE=$(tr '[A-Z]' '[a-z]' <<< "$FILE")
for FILE2 in *; do
if [ "$testFILE" = "$FILE2" ]; then
FLAG=0
fi
done
if [ $FLAG -eq 1 ]; then
mv -- "$FILE" "$(tr '[A-Z]' '[a-z]' <<< "$FILE")"
fi
FLAG=1
done
Quote variables to prevent word-splitting ("$FILE" instead of $FILE)
Generally preferable to use $() instead of tildes
Don't use string comparison where you don't have to
Use -- to delimit arguments in commands that accept it (in order to prevent files like -file from being treated as options)
By convention, you should really only use capital variable names for environment variables, though I kept them in above.
Pipes vs here strings (<<<) doesn't matter so much here, but <<< is slightly faster and generally safer.
Though more simply, I think you want
#!/bin/bash
for file in *; do
testFile=$(tr '[A-Z]' '[a-z]' <<< "$file")
[ -e "$testFile" ] || mv -- "$file" "$testFile"
done
Or on most modern mv implementations (not technically posix)
#!/bin/bash
for file in *; do
mv -n -- "$file" "$(tr '[A-Z]' '[a-z]' <<< "$file")"
done
From the man page
-n, --no-clobber
do not overwrite an existing file
Below script :
find . -mindepth 1 -maxdepth 1 -print0 | while read -d '' filename
do
if [ -e ${filename,,} ]
then
mv --backup ${filename} ${filename,,} 2>/dev/null
# create a backup of the desination only if the destination already exist
# suppressing the error caused by moving the file to itself
else
mv ${filename} ${filename,,}
fi
done
may do the job for you.
Advantages of this script
It will parse files containing newlines.
It avoids a prompt by doing selective backup destination that already exists.
Have looked around on stackoverflow, googled ... but nothing fits my requirement... :-) What I want/need is a flat txt file containing info on files in a certain dir and subdirs. There are ways to do this with 3rd party programs and from terminal/unix. Both ways would be fine by me. BUT ... I need all of the following info on the file:
Path and filename
Type (we are talking macintosh here, so it might come from the resource fork, many very old files, back to mid 90's!)
Filesize
Creation date
Modification date (this is not essential to have, just good to have).
Any good ideas? As mentioned, it could be a one-liner for the terminal that writes to a file or suggestion for a small app that can do it.
Thanks in advance :-)
EDIT: the output format doesn't matter much, can be tab or semicolon seperated or whatever, as long as it's a textfile - I can always modify it after. The important thing is the information is there... :-)
RESULT - what I ended up using (thanks for the help) :-)
#!/bin/bash
find . -type f -name '*' -print0 | while IFS= read -r -d '' file
do
name=$(basename "$file")
path=$(dirname "$file")
# full_path=$(readlink -f "$file") # This only works on Linux
full_path=$(echo "$PWD/${file#./}")
extension=${name##*.}
size_human_readable=$(ls -lh "$file" | awk -F' ' '{print $5}')
size_in_bytes=$(stat -f "%z" "$file")
creation_date=$(stat -f "%SB" "$file")
last_access=$(stat -f "%Sa" "$file")
last_modification=$(stat -f "%Sm" "$file")
last_change=$(stat -f "%Sc" "$file")
creator=$(mdls -name kMDItemFSCreatorCode "$file")
printf "\"%q\";" "$name"
printf "\"%q\";" "$full_path"
printf "%s" "\"$extension\";"
printf "\"$size_human_readable\";"
printf "\"$size_in_bytes\";"
printf "\"$last_modification\";"
printf "%s" "\"$creator\""
printf "\n"
done
Maybe something like this:
#!/bin/bash
find . -type f -name '*' -print0 | while IFS= read -r -d '' file
do
name=$(basename "$file")
path=$(dirname "$file")
# full_path=$(readlink -f "$file") # This only works on Linux
full_path=$(echo "$PWD/${file#./}")
extension=${name##*.}
size_human_readable=$(ls -lh "$file" | awk -F' ' '{print $5}')
size_in_bytes=$(stat -f "%z" "$file")
creation_date=$(stat -f "%SB" "$file")
last_access=$(stat -f "%Sa" "$file")
last_modification=$(stat -f "%Sm" "$file")
last_change=$(stat -f "%Sc" "$file")
printf "[$file]:\n"
printf "\tfile name:\t\t$name\n"
printf "\tfile path:\t\t$path\n"
printf "\tfull file path:\t\t$full_path\n"
printf "\tfile extension:\t\t$extension\n"
printf "\tfile size:\t\t$size_human_readable\n"
printf "\tfile size in bytes:\t$size_in_bytes\n"
printf "\tfile creation date:\t$creation_date\n"
printf "\tlast file access:\t$last_access\n"
printf "\tlast file modification:\t$last_modification\n"
printf "\tlast file change:\t$last_change\n"
printf "\n"
done
Of course you can modify the output! Copy this script into a file (e.g. recursive_file_info.sh). Then make it executable with chmod +x recursive_file_info.sh. If you want the output in a text file you can redirect it with ./recursive_file_info.sh > files.txt.
I have several files which all start with this line:
CREATE PROCEDURE **CHANGING_NAME**
I want to be able to pull the name of the procedure and use it to the rename the file. There is content to each file below this first line.
Has anyone done something like this before?
Thanks
Assuming you have all files in one directory :
#!/bin/bash
for i in *.extension :
do
# Assuming 3rd column of the first line is the new name of the file
# And **CHANGING_NAME** doesn't contain any space or meta characters
newname=$(awk 'NR==1 && /PROCEDURE/ {print $3}' "$i")
if [ "$newname" == "" ]; then
echo "There is no PROCEDURE in the first line";
echo "No new name for file $i";
else
mv "$i" "$newname"
fi
done
With a lot of care and pretending that the **CHANGING_NAME** is well-formed:
for file in *.files; do mv -i -- "$file" "$(awk '{print $3; exit}' $file)" ; done
The -i option is to prevent accidental overriding existing files.
This version works with spaces (and many other strange characters except for /):
for file in *.files; do mv -i -- "$file" "$(sed -n '1s/^CREATE\ PROCEDURE\ \(.*\)$/\1/p' $file)"; done
Since I was never great with awk I might suggest:
#! /bin/bash
#
for i in *.extension
do echo $i
newname=$(head -1 "${i}" | cut -d ' ' -f2)
mv -i "${i}" "${newname}"
done
This assumes all files you're looking for have the same extension. If not, and you need the extension, you could use:
#! /bin/bash
#
for i in *
do echo $i
ext="${i##*.}"
newname=$(head -1 "${i}" | cut -d ' ' -f2)
mv -i "${i}" "${newname}"."${ext}"
done
Both assume all the files are in a single directory.
You can try the next:
perl -lanE 'if($.==1&&/PROCEDURE/){close ARGV;say "$ARGV,$F[2]"}' files*
and if satisfied, change it to
perl -lanE 'if($.==1&&/PROCEDURE/){close ARGV;rename $ARGV,$F[2]}' files*
mv myfile `sed '1 s/.*PROCEDURE\s*//' myfile`
(the sed command will delete the text to the left of the word proceeding PROCEDURE regardless of how many spaces on only the first line and print it out the backticks make it execute in place so it is used as the filename to the mv command)
to move them all and add an extension .ext:
ls *.ext | xargs -I {} mv {} `sed '1 s/.*PROCEDURE\s*//' {}`.ext
This is what I came up with. It works perfectly -- I'm just curious if there's a smaller/crunchier way to do it. (wondering if possible without a loop)
files='file1|file2|file3|file4|file5'
path='/my/path'
found=$(find "$path" -regextype posix-extended -type f -regex ".*\/($files)")
for file in $(echo "$files" | tr '|', ' ')
do
if [[ ! "$found" =~ "$file" ]]
then
echo "$file"
fi
done
You can do this without invoking any external tools:
IFS="|"
for file in $files
do
[ -f "$file" ] || printf "%s\n" "$file"
done
Your code will break if you have file names with whitespace. This is how I would do it, which is a bit more concise.
echo "$files" | tr '|' '\n' | while read file; do
[ -e "$file" ] || echo "$file"
done
You can probably play around with xargs if you want to get rid of the loop all together.
$ eval "ls $path/{${files//|/,}} 2>&1 1>/dev/null | awk '{print \$4}' | tr -d :"
Or use awk
$ echo -n $files | awk -v path=$path -v RS='|' '{printf("! [[ -e %s ]] && echo %s\n", path"/"$0, path"/"$0) | "bash"}'
without whitespace in filenames:
files=(mbox todo watt zoff xorf)
for f in ${files[#]}; do test -f $f || echo $f ; done