Brand new to KornShell (ksh). I have an absolutely painful and hopefully simple problem.
I need to run a script with a string argument that includes single quotations mark and a variable. In the shell, it looks like this:
run_script -x '*082512*'
And that works fine.
But once I try to run it in a script with a variable for the date, I can not produce those single quotes. I have made the "today's date" variable without problem:
today=$(date "+%m%d%y")
But the problem occurs when I try to run the script. I have tried every possible combination of quotes within quotes and escaping the single quotes etc. etc. to no avail.
Any insight regarding this issue would be appreciated.
commenting here because the formatting is better.
Did you try
today=$(date "+%m%d%y")
run_script -x '*'"$today"'*'
As Glenn Jackman points out, quote are only 'seen' at the first level of shell invocation, where you typed in your command.
If the above doesn't help you, consider editing your question to show exact use case, with exact error messages or other non--expected behavior.
Good luck.
Related
I'm using Paramiko's exec_command() to run wp-cli commands on a remote server. Any command argument with spaces in it (no matter how I quote it) gives me a "Too many positional arguments" error. Here's sample code:
sshClient.exec_command("wp option update blogname \"Text with spaces\" 2>&1")
I've ssh'd into the remote machine from my local terminal and the command (wp option update blogname "Text with spaces") works fine there. I've also tried using combinations of double and single quotes.
Basically it's like the inner quotes are being ignored entirely and "Text with spaces" is being seen as 3 additional arguments. Is this about how Python parses strings with quotes, or about paramiko's exec_command()? What am I missing?
So this ended up being an issue with how the shell handles quotes in a command. Thanks to Martin for his comment pointing me in the right direction. Here is what finally worked for me:
sshClient.exec_command("wp option update blogname '\"Text with spaces\"' 2>&1")
After some Googling, I found this guide very helpful in understanding what was going on:
Shell Command Line Quoting Mechanisms
My understanding of the issue is that the shell on the remote machine where the command is being run strips out quotes, and always uses spaces as separators for arguments. So in order for "Text with spaces" to be interpreted by the remote shell as a single argument, you have to wrap it in single quotes, and escaped double quotes.
With some deeper understanding of how the shell interprets commands it makes more sense, but at face value it's not intuitive at all.
I am new to scripting in Linux and I think I'm getting confused with using variables inside command substitution the more I learn and read about it. Can someone explain to me the following scenario?
In my ksh script, I am trying to use a ksh variable inside an sqlplus script as follows:
temp_var="'a', 'b'"
randomVar=$(sqlplus -s $con_details <<EOF
update table ABC
Set field1='val'
Where field2 NOT IN ("${temp_var}");
EOF)
But the above syntax leads to an error in the query and it fails with code 1.
However when I unquote the variable and simply write
Where field2 NOT IN (${temp_var});
The query runs fine. I have seen a lot of examples on SO and Unix and Linux advising to always quote your variables used inside command substitution, but it seems the opposite works for me.
I don't seem to get why using quotes inside $() give an error as opposed to not using them.
Also, the query runs fine when I don't use the ksh variable in it (i.e. Without the WHERE clause).
This is a different situation than where the usual advice applies -- you're using the variable in a here-document, rather than as part of the command line. The difference is in how it gets parsed.
When you use a variable on a command line (something like ls $file), the variable gets replaced by its value partway through the process of parsing the command, with weird and generally undesirable results. The standard solution is to double-quote the variable (ls "$file") to prevent it from being parsed at all, just used directly. The standard mistake people make is putting quotes in the variable's value, which doesn't work because the variable gets replaced after quotes have already been parsed.
But you're using the variable in a here-document, and those work a lot differently. What happens is that the shell just does variable expansion (and some escape parsing) in the here-document, but doesn't do any more extensive parsing. In particular, it doesn't parse quotes in the here-document, just treats them like any other characters. The document then gets passed as input to the command (sqlplus in your case), and it parses the document according to whatever its syntax rules are. Since the parsing happens after variable replacement, it doesn't matter if the quotes are in the variable or around it; they work the same either way. But you can't do both, which is what was happening with double-quotes around the variable. Essentially, you were sending this document to sqlplus:
update table ABC
Set field1='val'
Where field2 NOT IN ("'a', 'b'");
... and sqlplus doesn't like that double-quotes around single-quotes thing, and complains.
I'm working on what should be a very simple BASH script. What I want to do is a pull an image from a webcamera using curl and write it to a file whose name is datestamped.
#! /bin/bash
DATE=$(date +%Y-%m-%d_%H-%M)
DIRECTORY1=home/manager/security_images/Studio_1/
TARGET1=${DIRECTORY1}${DATE}.jpg
curl http://web#192.168.180.211/snapshot.cgi > $TARGET1
When I try to run this I am told that there is no such file or directory. I believe this is due to an error in my escaping but I have tried seemingly every combination of quotation marks around the variables at each stage and still can't get it to work. I just don't understand what is going wrong and could really use some pointers towards what I'm doing wrong.
Many thanks
No, it’s just a typo.
DIRECTORY1=home/manager/security_images/Studio_1/
^^
Should be
DIRECTORY1=/home/manager/security_images/Studio_1/
of course.
As for escaping, even though only safe characters are used now, so quotes are technically superfluous, quoting out all $variables in every line by default is a good habit in shell scripting, there are very few cases when you do not want to use them.
Double quoting the redirection target should be enough:
curl http://web#192.168.180.211/snapshot.cgi > "$TARGET1"
Just make sure the path to it exists. You can run your script with
set -xv
to see how variables are interpolated.
I use ZSH for my terminal shell, and whilst I've written several functions to automate specific tasks, I've never really attempted anything that requires the functionality I'm after at the moment.
I've recently re-written a blog using Jekyll and I want to automate the production of blog posts and finally the uploading of the newly produced files to my server using something like scp.
I'm slightly confused about the variable bindings/usage in ZSH; for example:
DATE= date +'20%y-%m-%d'
echo $DATE
correctly outputs 2011-08-23 as I'd expect.
But when I try:
DATE= date +'20%y-%m-%d'
FILE= "~/path/to/_posts/$DATE-$1.markdown"
echo $FILE
It outputs:
2011-08-23
blog.sh: line 4: ~/path/to/_posts/-.markdown: No such file or directory
And when run with what I'd be wanting the blog title to be (ignoring the fact the string needs to be manipulated to make it more url friendly and that the route path/to doesn't exist)
i.e. blog "blog title", outputs:
2011-08-23
blog.sh: line 4: ~/path/to/_posts/-blog title.markdown: No such file or directory
Why is $DATE printing above the call to print $FILE rather than the string being included in $FILE?
Two things are going wrong here.
Firstly, your first snippet is not doing what I think you think it is. Try removing the second line, the echo. It still prints the date, right? Because this:
DATE= date +'20%y-%m-%d'
Is not a variable assignment - it's an invocation of date with an auxiliary environment variable (the general syntax is VAR_NAME=VAR_VALUE COMMAND). You mean this:
DATE=$(date +'20%y-%m-%d')
Your second snippet will still fail, but differently. Again, you're using the invoke-with-environment syntax instead of assignment. You mean:
# note the lack of a space after the equals sign
FILE="~/path/to/_posts/$DATE-$1.markdown"
I think that should do the trick.
Disclaimer
While I know bash very well, I only started using zsh recently; there may be zshisms at work here that I'm not aware of.
Learn about what a shell calls 'expansion'. There are several kinds, performed in a particular order:
The order of word expansion is as follows:
tilde expansion
parameter expansion
command substitution
arithmetic expansion
pathname expansion, unless set -f is in effect
quote removal, always performed last
Note that tilde expansion is only performed when the tilde is not quoted; viz.:
$ FILE="~/.zshrc"
$ echo $FILE
~/.zshrc
$ FILE=~./zshrc
$ echo $FILE
/home/user42/.zshrc
And there must be no spaces around the = in variable assignments.
Since you asked in a comment where to learn shell programming, there are several options:
Read the shell's manual page man zsh
Read the specification of the POSIX shell, http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html, especially if you want to run your scripts on different operating systems (and you will find yourself in that situation one fine day!)
Read books about shell programming.
Hang out in the usenet newsgroup comp.unix.shell where a lot of shell wizards answer questions
Alright, I have a script that takes a few arguments, runs data, and then rsyncs the data out to a different server. The problem is that to run the data, I have to take one of the arguments and then run a report using it, which is very bash unfriendly (Ex. [3023.2<>1], [5111.3$]="5", etc).
So if I'm going to run the command, I need to put the argument in single quotes, which would then make the argument not be pulled into it.
Thus if I were to run the script...
arg1 = [5111.3$]="5"
runjob specfile.spx '$arg1'
This wouldn't work, but if I were to run it with double quotes, then there is a good chance that the argument that gets passed in will have double quotes. Any ideas on how to get around this?
Use single quotes around the value when you set it, then use double-quotes around the variable when you expand it:
$ arg1='[5111.3$]="5"'
$ echo "$arg1"
[5111.3$]="5"
Quote escapes. Try
[5111.3$]=\"5\"
The Advanced Scripting Guide has a good section on quoting.