This question already has answers here:
"ls: not found" after running "read PATH"
(2 answers)
Closed 6 years ago.
I'd like for this Bash script to:
read and assign the content of a file to URLS
prompt the user to enter a path to download content to, assigning it to PATH
iterate through URLS
use wget to download the content to PATH.
Here's what I've got so far:
#!/bin/bash
URLS=$(<urls.txt)
read -e -p "Please enter the path of the directory you'd like to save your files to: " PATH
for link in $URLS
do
wget --background --tries=45 $URLS --directory-prefix $PATH
done
exit 0
My hunch is that I am not using $PATH correctly with wget, as the script will run correctly if I comment out --directory-prefix $PATH.
Thanks in advance for the assistance.
PATH is an environment variable with a reserved meaning: It's used to find the location of binaries to execute.
Use a non-reserved name to avoid overwriting it, such as path; this convention is explicitly given in the POSIX specification to avoid overwrite of environment variables with meaning to the system. Quoting with emphasis added:
Environment variable names used by the utilities in the Shell and Utilities volume of IEEE Std 1003.1-2001 consist solely of uppercase letters, digits, and the '_' (underscore) from the characters defined in Portable Character Set and do not begin with a digit. Other characters may be permitted by an implementation; applications shall tolerate the presence of such names. Uppercase and lowercase letters shall retain their unique identities and shall not be folded together. The name space of environment variable names containing lowercase letters is reserved for applications. Applications can define any environment variables with names from this name space without modifying the behavior of the standard utilities.
...which is pertinent even to shell variables which aren't explicitly exported to the environment because assignments to shell variables with names that overlap with an environment variable (such as PATH) implicitly overwrite the latter.
Thus, your code might look like:
#!/bin/bash
read -e -p "Please enter the path of the directory you'd like to save your files to: " path
while IFS= read -r link; do
wget --background --tries=45 --directory-prefix "$path" "$link"
done <urls.txt
See also:
BashFAQ #001, describing best practices around reading a file from bash.
DontReadLinesWithFor, explicitly describing caveats around using for to iterate over what's intended to be interpreted as line-oriented data.
Also, note that zsh is not a POSIX-compliant shell; one of the respects in which it diverges from the spirit and intent of the standard is that it gives path special treatment as well, making path not safe as an alternative to PATH if zsh compatibility is needed.
Related
In the bash shell, I'm using git-bash.exe, how do I access the Windows 10 ProgramFiles(x86) environment variable?
If I execute printenv I see it in the output with the casing noted but attempts to access it using echo $ProgramFiles(x86), echo $ProgramFiles\(x86\) and echo $"ProgramFiles(x86)" do not work.
I am able to access the non-x86 version of that environment variable without any issue using echo $PROGRAMFILES and do relevant colon removal and backslash to forward replacements necessary to use it in PATH environment variable updates, e.g. PATH=$PATH:"/${PROGRAMFILES//:\\//} (x86)/Some App Path With Spaces/" followed by echo $PATH and printenv PATH that confirms the desired result. The issue is that I'd rather not have to compose the ProgramFiles(x86) environment variable versus being able to use it directly in updates to the PATH environment variable.
Along these same lines when trying to use the Windows APPDATA [ = C:\Users<username>\AppData\Roaming ] environment variable in updates to PATH environment variable I need to be able to replace not only the initial colon & backslash but also the subsequent backslashes with forward slashes. Using echo ${APPDATA//:\\//} produces C/Users\<username>\AppData\Roaming and I'm not aware of how to get the bash environment variable character matching and substitution syntax to cover both cases in order to produce the required C/Users/<username>/AppData/Roaming necessary for use in updates to PATH environment variable.
Note: there's a flaw in the process described below. In particular, if some environment variable is set to a multi-line value where one of the value lines matches the sed expression, you'll capture that line as well. To avoid this, if you have a Python available, you could use:
python -c 'import os; print(os.getenv("FOO(BAR)"))'
for instance. This will print None if the variable is not set, so you might want to make it fancier: e.g., supply a default value, or use sys.exit(1) if the variable is not set, for instance. (But if you have a Python interpreter available, you might consider writing in Python rather than directly in bash.)
Unix shell (sh, bash, etc) variable names—including environment variables—are constrained to character sets that exclude parentheses. In particular, "$FOO(BAR)" always parses as a reference to variable $FOO, followed by (BAR) as a separate word. This holds even with braceed expansion forms, where the separate word (BAR) is syntactically invalid:
bash$ echo "${FOO(BAR)}"
bash: ${FOO(BAR)}: bad substitution
Nonetheless, it is possible to set such variables, and access them, using other programs. For instance, using Python I set FOO(BAR) to hello:
>>> import os
>>> os.environ["FOO(BAR)"] = "hello"
>>> import subprocess
>>> subprocess.call("bash")
bash$
This bash instance cannot directly access the variable, but env prints all the variables:
bash$ env | grep FOO
FOO(BAR)=hello
If you have env (you probably do) and sed, you can combine them to extract arbitrary variables:
bash$ setting="$(env | sed -n 's/^FOO(BAR)=//p')"
bash$ echo "$setting"
hello
So assuming that Windows Bash doesn't have any special case to work around this particular clumsiness better, this same trick should work for "ProgramFiles(x86)".
Substituting multiple backslashes with forward slashes
You're mostly there: the problem is that your pattern looks specifically for :\ but the strings have multiple \s without colons. Your best bet is probably to have a program or function that actually understands Windows paths, as they don't necessarily have drive letters at the front (see https://learn.microsoft.com/en-us/dotnet/standard/io/file-path-formats). But this pattern works for all-backslash:
bash$ v='a\b\c'
bash$ echo ${v//\\/\/}
a/b/c
The double slash means "substitute all occurrences". The pattern is then \\, which matches one backslash. The next slash introduces the replacement string, which is \/, which means one forward slash. (This can also be written as just / but I find that harder to read, oddly enough.)
Of course this does not replace the colon in C:, so we need one more substitution. You can't do that in one ${...} expansion, so the trick is to add another one:
bash$ v='C:\a\b\c'
bash$ echo ${v//\\/\/}
C:/a/b/c
bash$ v1="${v//\\//}"; echo ${v1/:/}
C/a/b/c
Put this inside a shell function, which you can eventually make smart enough to handle all valid paths, and that way you can use local to keep the variable name v1 from leaking.
Regarding APPDATA: The cygpath program can convert pathnames between Windows, Unix and "Mixed" conventions. Both Cygwin and Git for Windows come with this tool. Example:
$ echo "$APPDATA"
C:\Users\me\AppDataRoaming\
$ cygpath -u "$APPDATA"
/c/Users/me/AppData/Roaming
$ cygpath -m "$APPDATA"
C:/Users/me/AppData/Roaming
$ cygpath -w "$APPDATA"
C:\Users\me\AppData\Roaming
The "mixed" format is quite usefull because even most windows programs and Git for Windows can handle that format directly.
Assigning the output of cygpath to a variable works like this (note the quotes!):
$ XAPP=$(cygpath "$APPDATA")
$ echo "$XAPP"
$ cd "$XAPP"
I have spent several hours trying to make the following piece of code work
PATH="C:\Ben\MyPictures"
echo $PATH
MY=`expr 2 + 2`
but this will not work because "expr: command not found". The only thing I've dug on StackOverflow are pathing issue (I.E. set my environment variable), but if that's the problem, why would other functions like echo, let, and declare already work fine?
For more context, this is on a near-fresh installation of window's cygwin. My question is why can't I find expr?
You have modified your PATH to have only 1 directory(therefore it cant find expr). You must append your new path to PATH and not replace existing PATH values, like this:
export PATH="$PATH:C:\Ben\MyPictures"
Also instead of calling an external process expr for calculation you can use the bash's builtin arithmetic evaluation:
$ echo $((2+2))
4
Edit:
Yes those would work because they are not executable files found from directories listed in $PATH.
Instead they(echo, type etc) are functionality provided by the bash shell itself called shell built-ins.
Type out type echo and type expr to know what type of command is it(alias/shell builtin/executable file etc.)
Shell built-ins help can be usually found out by help shellBuiltin where as we use man pages for executable files.
PS: type itself is a shell built-in(see type type)
The path variable PATH is used to find the location of standard binaries that ship with your environment. In fact system environment variables are usually capitalized( for example HOME ).
In your command :
PATH="C:\Ben\MyPictures"
You have accidentally(or intentionally?) replaced the standard paths to the standard binaries by C:\Ben\MyPictures so that command interpreter, bash here, can no longer find expr. When you use capitalized variables,say PATH, you could do:
if [ -z "$PATH" ] #check if a variable is empty
then
PATH="C:\Ben\MyPictures"
else
PATH="$PATH:C:\Ben\MyPictures"
fi
The better option indeed is to use small letters for user defined variables :
path="C:\Ben\MyPictures"
#in this case you should not get an error for expr but
# I am not sure how a Linux-Like environment handle character case.
# check that out.
I want to put my ~/Library/Application Support/ directory to a variable in my ~/.bash_profile` to make it easier to reference from within Terminal. I first attempted to define it as follows:
export L=~/Library/Application\ Support
However, when I tried to source ~/.bash_profile and then called ls $L, I got an error: /Users/username/Library/Application: Not a directory.
However, no matter how I define it I cannot define it properly, as far as I came up with the way to define it. Here's the list that I tried, but none of them worked properly.
~/Library/Application Support
"~/Library/Application Support"
"~/Library/Application\ Support"
So is it feasible to store a string which includes a whitespace to a variable in bash to begin with?
Your export statement is fine; the space is properly escaped. You just need to quote the expansion of the parameter, so that bash gives a single argument to the ls command:
ls "$L"
This question already has answers here:
Getting "command not found" error in bash script
(6 answers)
Closed 2 years ago.
I've created a simple script to check if a folder exists and if not to create it. The script that follow
#!/bin/bash
PATH=~/Dropbox/Web_Development/
FOLDER=Test
if [ ! -d $PATH$FOLDER ]
then
echo $PATH$FOLDER 'not exists'
/bin/mkdir $PATH$FOLDER
echo $PATH$FOLDER 'has been created'
fi
works only if the mkdir command is preceded by /bin/. Failing in that, bash env output the error message "command cannot be found".
I though this could have been related to the system $PATH variable, but it looks regular (to me) and the output is as following:
/Library/Frameworks/Python.framework/Versions/2.7/bin:/bin:/usr/local/bin:/usr/bin:/sbin:/usr/local/sbin:/usr/sbin
I'm not sure whether the order with the different bin folders have been listed make any difference, but the /bin one (where the mkdir on my OSX Maverick) seems to reside is there hence I would expect bash to being able to execute this.
In fact, if I call the bash command from terminal, by typing just mkdir bash output the help string to suggest me how the mkdir command should be used. This suggests me that at a first instance bash is able to recognise the $PATH variable.
So what could be the cause? Is there any relation between the opening statement at the top of my .sh - #!/bin/bash - file and the "default" folder?
Thanks
Yeah, sometimes it is a bad idea to use capital letters for constant variables, because there are some default ones using the same convention. You can see some of the default variables here (Scroll to Special Parameters and Variables section). So it is better to use long names if you don't want to get any clashes.
Another thing to note is that you're trying to replicate mkdir -p functionality, which creates a folder if it does not exist (also it does create all of the parents, which is what you need in most cases)
One more thing - you always have to quote variables, otherwise they get expanded. This may lead to some serious problems. Imagine that
fileToRemove='*'
rm $fileToRemove
This code will remove all files in the current folder, not a file named * as you might expect.
One more thing, you should separate path from a folder with /. Like this "$MY_PATH/$MY_FOLDER". That should be done in case you forget to include / character in your path variable. It does not hurt to have two slashes, that means that /home/////////user/// folder is exactly the same /home/user/ folder.
Sometimes it is tricky to get ~ working, so using $HOME is a bit safer and more readable anyway.
So here is your modified script:
#!/bin/bash
MY_PATH="$HOME/Dropbox/Web_Development/"
MY_FOLDER='Test'
mkdir -p "$MY_PATH/$MY_FOLDER"
The problem is that your script sets PATH to a single directory, and that single directory does not contain a program called mkdir.
Do not use PATH as the name of a variable (use it to list the directories to be searched for commands).
Do learn the list of standard environment variable names and those specific to the shell you use (e.g. bash shell variables). Or use a simple heuristic: reserved names are in upper-case, so use lower-case names for variables local to a script. (Most environment variables are in upper-case — standard or not standard.)
And you can simply ensure that the directory exists by using:
mkdir -p ~/Dropbox/Web_Development
If it already exists, no harm is done. If it does not exist, it is created, and any other directories needed on the path to the directory (eg ~/Dropbox) is also created if that is missing.
I am trying to configure my windows portable git bash shell. When I execute env I get:
ANT_HOME=C:\Program Files\WinAnt
PORTABLEAPPS.COMVIDEOS:FORWARDSLASH=H:/Documents/Videos
VBOX_INSTALL_PATH=C:\Program Files\Oracle\VirtualBox\
PORTABLEAPPS.COMLOCALEWINNAME=LANG_ENGLISH
PAL:LASTPORTABLEAPPSBASEDIR:DOUBLEBACKSLASH=H:
PAL:DRIVELETTER=H **** this is the variable I am after ******
PAL:APPDIR=H:\PortableApps\GitPortable\App
TEMP=/tmp
The variable I am trying to reference is PAL:DRIVELETTER=H. I want to use this set my path in my .bash_profile script. This is all on a USB stick and the drive letter will of course change from time to time.
I have tried echoing:
$PAL:DRIVELETTER
${PAL:DRIVELETTER}
and numerous other things.
The bash "Definitions" does mention explicitly:
name
A word consisting solely of letters, numbers, and underscores, and beginning with a letter or underscore.
Names are used as shell variable and function names.
Also referred to as an identifier.
So your variable name PAL:DRIVELETTER is actually invalid.
You need to extract it from the 'env' output, as proposed in this answer:
pal_driveletter=$(env |grep "^PAL:DRIVELETTER=" | cut -d= -f2-)