I am trying to preserve special characters in my variables when setting them. I am trying to save file paths as variables. For example:
prompt
user input
click and drag your file here
/Users/leetbacon/Desktop/My\ Stuff/time\ to\ fly\ \&\ soar.png
You chose /Users/leetbacon/Desktop/My\ Stuff/time\ to\ fly\ \&\ soar.png
Instead, whenever I input the file it always outputs like this (which I DON'T want):
You chose /Users/leetbacon/Desktop/My Stuff/time to fly & soar.png
Any way to get it to store the variable how I would like it?
Here's the code I have right now:
echo 'click and drag your file here'
read -p " " FilepatH
echo 'You chose '"$FilepatH"
I would like for it to preserve ALL special characters. I'm just trying to write a script that can cover all possibilities of file names.
And I'm using OS X Yosemite
--Todd
I would like for it to preserve ALL special characters.
Done. In the script you posted, all characters are preserved.
You can verify that they are really preserved by running:
ls "$FilepatH"
This will work only because all special characters are preserved. If they were not preserved it wouldn't work, the file would not be found.
However, you might want to clarify the intent with the output:
echo "You chose '$FilepatH'"
This will print:
You chose '/Users/leetbacon/Desktop/My Stuff/time to fly & soar.png'
You can tell read to skip parsing (and removing) escapes and quotes by using its -r ("raw") option. But, as everyone has said, you do not want to do this. Having escapes and/or quotes embedded in the values assigned to shell variables doesn't do anything useful, because the shell does not parse them when it expands variables. See this question for an example of someone having trouble specifically because they had escapes embedded in the filenames they were trying to use.
Here's an example of doing this right:
$ cat t1.sh
#!/bin/bash
echo 'click and drag your file here'
read -p " " FilepatH
echo 'You chose '"$FilepatH"
echo
echo "Trying to use the variable with double-quotes:"
ls -l "$FilepatH"
$ ./t1.sh
click and drag your file here
/Users/gordon/weird\ chars\:\ \'\"\\\(\)\&\;.txt
You chose /Users/gordon/weird chars: '"\()&;.txt
Trying to use the variable with double-quotes:
-rw-r--r-- 1 gordon staff 0 Jul 19 22:56 /Users/gordon/weird chars: '"\()&;.txt
And here's doing it wrong (with read -r):
$ cat t2.sh
#!/bin/bash
echo 'click and drag your file here'
read -r -p " " FilepatH
echo 'You chose '"$FilepatH"
echo
echo "Trying to use the variable with double-quotes:"
ls -l "$FilepatH"
echo
echo "Trying to use the variable without double-quotes:"
ls -l $FilepatH
$ ./t2.sh
click and drag your file here
/Users/gordon/weird\ chars\:\ \'\"\\\(\)\&\;.txt
You chose /Users/gordon/weird\ chars\:\ \'\"\\\(\)\&\;.txt
Trying to use the variable with double-quotes:
ls: /Users/gordon/weird\ chars\:\ \'\"\\\(\)\&\;.txt: No such file or directory
Trying to use the variable without double-quotes:
ls: /Users/gordon/weird\: No such file or directory
ls: \'\"\\\(\)\&\;.txt: No such file or directory
ls: chars\:\: No such file or directory
Note that with the variable in double-quotes, it tried to treat the escapes as literal parts of the filename. Without them, it split the file path into separate items based on spaces, and then still treated the escapes as literal parts of the filenames.
Related
Hello I am trying to get all files with Jane's name to a separate file called oldFiles.txt. In a directory called "data" I am reading from a list of file names from a file called list.txt, from which I put all the file names containing the name Jane into the files variable. Then I'm trying to test the files variable with the files in list.txt to ensure they are in the file system, then append the all the files containing jane to the oldFiles.txt file(which will be in the scripts directory), after it tests to make sure the item within the files variable passes.
#!/bin/bash
> oldFiles.txt
files= grep " jane " ../data/list.txt | cut -d' ' -f 3
if test -e ~data/$files; then
for file in $files; do
if test -e ~/scripts/$file; then
echo $file>> oldFiles.txt
else
echo "no files"
fi
done
fi
The above code gets the desired files and displays them correctly, as well as creates the oldFiles.txt file, but when I open the file after running the script I find that nothing was appended to the file. I tried changing the file assignment to a pointer instead files= grep " jane " ../data/list.txt | cut -d' ' -f 3 ---> files=$(grep " jane " ../data/list.txt) to see if that would help by just capturing raw data to write to file, but then the error comes up "too many arguments on line 5" which is the 1st if test statement. The only way I get the script to work semi-properly is when I do ./findJane.sh > oldFiles.txt on the shell command line, which is me essentially manually creating the file. How would I go about this so that I create oldFiles.txt and append to the oldFiles.txt all within the script?
The biggest problem you have is matching names like "jane" or "Jane's", etc. while not matching "Janes". grep provides the options -i (case insensitive match) and -w (whole-word match) which can tailor your search to what you appear to want without having to use the kludge (" jane ") of appending spaces before an after your search term. (to properly do that you would use [[:space:]]jane[[:space:]])
You also have the problem of what is your "script dir" if you call your script from a directory other than the one containing your script, such as calling your script from your $HOME directory with bash script/findJane.sh. In that case your script will attempt to append to $HOME/oldFiles.txt. The positional parameter $0 always contains the full pathname to the current script being run, so you can capture the script directory no matter where you call the script from with:
dirname "$0"
You are using bash, so store all the filenames resulting from your grep command in an array, not some general variable (especially since your use of " jane " suggests that your filenames contain whitespace)
You can make your script much more flexible if you take the information of your input file (e.g list.txt), the term to search for (e.g. "jane"), the location where to check for existence of the files (e.g. $HOME/data) and the output filename to append the names to (e.g. "oldFile.txt") as command line [positonal] parameters. You can give each default values so it behaves as you currently desire without providing any arguments.
Even with the additional scripting flexibility of taking the command line arguments, the script actually has fewer lines simply filling an array using mapfile (synonymous with readarray) and then looping over the contents of the array. You also avoid the additional subshell for dirname with a simple parameter expansion and test whether the path component is empty -- to replace with '.', up to you.
If I've understood your goal correctly, you can put all the pieces together with:
#!/bin/bash
# positional parameters
src="${1:-../data/list.txt}" # 1st param - input (default: ../data/list.txt)
term="${2:-jane}" # 2nd param - search term (default: jane)
data="${3:-$HOME/data}" # 3rd param - file location (defaut: ../data)
outfn="${4:-oldFiles.txt}" # 4th param - output (default: oldFiles.txt)
# save the path to the current script in script
script="$(dirname "$0")"
# if outfn not given, prepend path to script to outfn to output
# in script directory (if script called from elsewhere)
[ -z "$4" ] && outfn="$script/$outfn"
# split names w/term into array
# using the -iw option for case-insensitive whole-word match
mapfile -t files < <(grep -iw "$term" "$src" | cut -d' ' -f 3)
# loop over files array
for ((i=0; i<${#files[#]}; i++)); do
# test existence of file in data directory, redirect name to outfn
[ -e "$data/${files[i]}" ] && printf "%s\n" "${files[i]}" >> "$outfn"
done
(note: test expression and [ expression ] are synonymous, use what you like, though you may find [ expression ] a bit more readable)
(further note: "Janes" being plural is not considered the same as the singular -- adjust the grep expression as desired)
Example Use/Output
As was pointed out in the comment, without a sample of your input file, we cannot provide an exact test to confirm your desired behavior.
Let me know if you have questions.
As far as I can tell, this is what you're going for. This is totally a community effort based on the comments, catching your bugs. Obviously credit to Mark and Jetchisel for finding most of the issues. Notable changes:
Fixed $files to use command substitution
Fixed path to data/$file, assuming you have a directory at ~/data full of files
Fixed the test to not test for a string of files, but just the single file (also using -f to make sure it's a regular file)
Using double brackets — you could also use double quotes instead, but you explicitly have a Bash shebang so there's no harm in using Bash syntax
Adding a second message about not matching files, because there are two possible cases there; you may need to adapt depending on the output you're looking for
Removed the initial empty redirection — if you need to ensure that the file is clear before the rest of the script, then it should be added back, but if not, it's not doing any useful work
Changed the shebang to make sure you're using the user's preferred Bash, and added set -e because you should always add set -e
#!/usr/bin/env bash
set -e
files=$(grep " jane " ../data/list.txt | cut -d' ' -f 3)
for file in $files; do
if [[ -f $HOME/data/$file ]]; then
if [[ -f $HOME/scripts/$file ]]; then
echo "$file" >> oldFiles.txt
else
echo "no matching file"
fi
else
echo "no files"
fi
done
Trying to write a bash script to copy a large number of files from an external drive into separate directories based on a subject id.
I've included the script I've written below.
I get the following error:
cat: /Volumes/Seagate: No such file or directory
cat: Backup: No such file or directory
cat: Plus: No such file or directory
cat: Drive/Subject_List.txt: No such file or directory
When I try to copy a single file at a time using the terminal, it copies using the exact command I've put in this script. I'm not sure why it's not recognizing the directory when I try and use it in the script below. Any help is greatly appreciated!
#!/bin/bash
#A bash script to copy the structural and functional files into the HCP_Entropy folder
#subject list
SUBJECT_LIST="/Volumes/Seagate/Backup/Plus/Drive/Subject_List.txt
for j in $(cat ${SUBJECT_LIST}); do
echo ${j}
cp /Volumes/Seagate\ Backup\ Plus\ Drive/HCP_DATA/Structural/{j}/unprocessed/3T/T1w_MPR1/${j}_3T_T1w_MPR1.nii.gz /Users/myname/Box/HCP_Entropy/BEN/${j}/anat
done
the line
$SUBJECT_LIST=/Volumes/Seagate\ Backup\ Plus\ Drive/Subject_List.txt
is bogus.
to assign values to a variable, you must not add the $ specifier.
a token starting with $ will be expanded, so $SUBJECT_LIST=... will first be expanded to =... (since you haven't assigned anything to the SUBJECT_LIST variable yet it is empty).
the proper way would be:
SUBJECT_LIST="/Volumes/Seagate Backup Plus Drive/Subject_List.txt"
(this uses quotes instead of escaping each space, which i find much more readable)
you also need to quote variables in case they contain spaces, else they might be interpreted by the command (cp) as multiple arguments.
for j in $(cat "${SUBJECT_LIST}"); do
# ...
done
and of course, you should check whether the source file actually exists, just like the destination directory.
indir="/Volumes/Seagate Backup Plus Drive"
SUBJECT_LIST="${indir}/Subject_List.txt"
cat "${SUBJECT_LIST}" | while read j; do
infile="${indir}/HCP_DATA/Structural/${j}/unprocessed/3T/T1w_MPR1/${j}_3T_T1w_MPR1.nii.gz"
outdir="/Users/myname/Box/HCP_Entropy/BEN/${j}/anat"
mkdir -p "${outdir}"
if [ -e "${infile}" ]; then
cp -v "${infile}" "${outdir}"
else
echo "missing file ${infile}" 1>&2
fi
done
Hello so today i was playing around with my shell script and figured to make it more user friendly i would make it so the file extension of file was automatically added.
for example say the user wants to search a file using grep but first they must type in thhe name of the file in this case lets say file.txt what i want to do is automatically add on the .txt so the user only needs to type in "file"
here is what i have so far but this does not work:
echo "Current .txt files "
ls -R |grep .txt
echo "--------------------------------------------------------------------------------"
echo -n "Please select a file to search in: "
read fileName
file=$fileName.txt
i thought in this case since i am appending an extension on to the end of the variable name but this has not worked.
Put quotes around it:
file="$filename.txt"
EDIT: As it happens, this answer is incorrect. See the comments below and the other answer.
Your code should work. See below-
Contents of test.sh:
#!/bin/bash
echo "Current .txt files "
ls -R |grep .txt
echo "--------------------------------------------------------------------------------"
echo -n "Please select a file to search in: "
read fileName
file=$fileName.txt
echo "You are searching for $file"
ls -l "$file"
Test run:
$ ./test.sh
Current .txt files
p1.txt
t1.txt
t2.txt
--------------------------------------------------------------------------------
Please select a file to search in: t2
You are searching for t2.txt
-rw-r----- 1 d1rld1f1 d1rld1f1 4 Apr 20 12:41 t2.txt
$
BTW, it is generally advisable to enclose variable names in double quotes. This prevents reinterpretation of all special characters within the quoted string.
I've got a Bash script (Cygwin) that uses some Windows paths with spaces in them. Consequently, I have escaped the space with a \ in my variable definition.
Everything within the script works fine. However, I need to pass this variable as an argument to a command-line executable. When I do that, my escaping gets messed up.
Sample non-functional script:
#!/bin/sh
# File paths
destinationPath="/cygdrive/c/Documents and Settings/Eric/My Documents/"
attachments="\n2013-12-12.pdf"
body="Test Body"
recipient="asdf#asdf.com"
# Prepare attachments
args=""
for file in $attachments ; do
file=${file//[ \\n]/}
touch $file
mv $file "$destinationPath/$file"
args="$args -a $destinationPath/$file"
done
# Send email
echo -e $body | email --from-addr nosender#mydomain.com --from-name "Automated CSS Downloader" --subject "Downloaded new file(s) \
from CSS" $args eric#mydomain.com
Output from the script:
$ bash -x test.sh
+ destinationPath='/cygdrive/c/Documents and Settings/Eric/My Documents/'
+ attachments='\n2013-12-12.pdf'
+ body='Test Body'
+ recipient=asdf#asdf.com
+ args=
+ for file in '$attachments'
+ file=2013-12-12.pdf
+ touch 2013-12-12.pdf
+ mv 2013-12-12.pdf '/cygdrive/c/Documents and Settings/Eric/My Documents//2013-12-12.pdf'
mv: listing attributes of `2013-12-12.pdf': Invalid argument
+ args=' -a /cygdrive/c/Documents and Settings/Eric/My Documents//2013-12-12.pdf'
+ echo -e Test Body
+ email --from-addr nosender#mydomain.com --from-name 'Automated CSS Downloader' --subject 'Downloaded new file(s) from CSS' -a /cygdrive/c/Documents and Settings/Eric/My Documents//2013-12-12.pdf eric#mydomain.com
email: WARNING: Email address 'and' is invalid. Skipping...
email: FATAL: Could not open attachment: /cygdrive/c/Documents: No such file or directory
So, as you can see, the escaped space in the path is not being exported in the $args variable. I am assuming the error comes on the line "args=...". But I am not sure how to escape $destinationPath to ensure that the escaped characters are retained.
I've tried using double quotes (with no escaped space) in destinationPath, but to no avail. If I try to double quote $destinationPath in the args= line, then the output also gets all screwed up with a bunch of extra quoting.
How can I get this to work? I've tried playing around with the $IFS variable, but I don't really know what I'm doing with it and can't seem to get it working with that either, although I suspect the solution has something to do with $IFS.
There's no good way of doing this without an array. (There's a not good way of doing it, using eval, but the array is simpler.)
The problem is that you want each file to end up as one argument to email, but you want to accumulate all those arguments into a Bash variable. There's just no way to do that: you can cause the Bash variable to be inserted as a single argument ("$arg") or you can cause it to be inserted as however many words it gets split into ($arg), but you can't get it to be split according to whether or not spaces were escaped when the variable was created, because Bash only remembers the string assigned to a variable, not the escape marks.
However, you can do it with an array, because you can make every filename exactly one array element, and you can get Bash to insert an array as one argument per element.
Here's how:
# File paths
destinationPath="/cygdrive/c/Documents and Settings/Eric/My Documents/"
attachments="\n2013-12-12.pdf"
body="Test Body"
recipient="asdf#asdf.com"
# Prepare attachments
args=()
for file in $attachments ; do
file=${file//[ \\n]/}
touch $file
mv $file "$destinationPath/$file"
args+=(-a "$destinationPath/$file")
done
# Send email
echo -e $body |
email --from-addr nosender#mydomain.com \
--from-name "Automated CSS Downloader" \
--subject "Downloaded new file(s) from CSS" \
"${args[#]}" eric#mydomain.com
You might also want to make the attachments variable into an array; from the presence of the newline character, I'm assuming that you're actually setting it in some more complicated way, so I didn't change the code. But your current code won't work if the attachment name has a space in it (and I'm pretty sure that the newline will be eliminated by the parameter expansion in the for form, unless you've altered the value of $IFS, so file=${file//[ \\n]/} shouldn't be necessary.)
You need escaped double quoted marks when providing destinationPath in a string.
This should work:
#!/bin/sh
# file paths
destinationPath="/cygdrive/c/Documents and Settings/Eric/My Documents/"
attachments="\n2013-12-12.pdf"
body="Test Body"
recipient="asdf#asdf.com"
# prepare attachments
args=""
for file in $attachments ; do
file=${file//[ \\n]/}
touch $file
mv $file "$destinationPath/$file"
#HERE YOU NEED ESCAPED DOUBLEQUOTED
args="$args -a \"$destinationPath/$file\""
done
# send email
echo -e $body | email --from-addr nosender#mydomain.com --from-name "Automated CSS Downloader" --subject "Downloaded new file(s) \
from CSS" $args eric#mydomain.com
I want to make a script that takes a file path for argument, and cds into its folder.
Here is what I made :
#!/bin/bash
#remove the file name, and change every space into \space
shorter=`echo "$1" | sed 's/\/[^\/]*$//' | sed 's/\ /\\\ /g'`
echo $shorter
cd $shorter
I actually have 2 questions (I am a relative newbie to shell scripts) :
How could I make the cd "persistent" ? I want to put this script into /usr/bin, and then call it from wherever in the filesystem. Upon return of the script, I want to stay in the $shorter folder. Basically, if pwd was /usr/bin, I could make it by typing . script /my/path instead of ./script /my/path, but what if I am in an other folder ?
The second question is trickier. My script fails whenever there is a space in the given argument. Although $shorter is exactly what I want (for instance /home/jack/my\ folder/subfolder), cd fails whith the error /usr/bin/script : line 4 : cd: /home/jack/my\: no file or folder of this type. I think I have tried everything, using things like cd '$shorter' or cd "'"$shorter"'" doesn't help. What am I missing ??
Thanks a lot for your answers
in your .bashrc add the following line:
function shorter() { cd "${1%/*}"; }
% means remove the smaller pattern from the end
/* is the patern
Then in your terminal:
$ . ~/.bashrc # to refresh your bash configuration
$ type shorter # to check if your new function is available
shorter is a function
shorter ()
{
cd "${1%/*}"
}
$ shorter ./your/directory/filename # this will move to ./your/directory
The first part:
The change of directory won't be “persistent” beyond the lifetime of your script, because your script runs in a new shell process. You could, however, use a shell alias or a shell function. For example, you could embed the code in a shell function and define it in your .bash_profile or other source location.
mycdfunction () {
cd /blah/foo/"$1"
}
As for the “spaces in names” bit:
The general syntax for referring to a variable in Bourne shells is: "$var" — the "double quotes" tell the shell to expand any variables inside of them, but to group the outcome as a single parameter.
Omitting the double quotes around $var tells the shell to expand the variable, but then split the results into parameters (“words”) on whitespace. This is how the shell splits up parameters, normally.
Using 'single quotes' causes the shell to not expand any contents, but group the parameters togethers.
You can use \ (backslash-blank) to escape a space when you're typing (or in a script), but that's usually harder to read than using 'single quotes' or "double quotes"…
Note that the expansion phase includes: $variables wild?cards* {grouping,names}with-braces $(echo command substitution) and other effects.
| expansion | no expansion
-------------------------------------------------------
grouping | " " | ' '
splitting | (no punc.) | (not easily done)
For the first part, there is no need for the shorter variable at all. You can just do:
#!/bin/bash
cd "${1%/*}"
Explanation
Most shells, including bash, have what is called Parameter Expansion and they are very powerful and efficient as they allow you to manipulate variables nativly within the shell that would normally require a call to an external binary.
Two common examples of where you can use Parameter Expansion over an external call would be:
${var%/*} # replaces dirname
${var##*/} # replaces basename
See this FAQ on Parameter Expansion to learn more. In fact, while you're there might as well go over the whole FAQ
When you put your script inside /usr/bin you can call it anywhere. And to deal with whitespace in the shell just put the target between "" (but this doesn't matter !!).
Well here is a demo:
#!/bin/bash
#you can use dirname but that's not apropriate
#shorter=$(dirname $1)
#Use parameter expansion (too much better)
shorter=${1%/*}
echo $shorter
An alternate way to do it, since you have dirname on your Mac:
#!/bin/sh
cd "$(dirname "$1")"
Since you mentioned in the comments that you wanted to be able to drag files into a window and cd to them, you might want to make your script allow file or directory paths as arguments:
#!/bin/sh
[ -f "$1" ] && set "$(dirname "$1")" # convert a file to a directory
cd "$1"