I have a script that takes text as an argument and encodes it. If the text contains only single quotes I surround it with double and vice versa, but it gets tricky if it contains both because it will be closed at first appearance.
script "I can't use "test" text"
// Double quote will be terminated at "I can't use "
So how to escape both cases without using a backslash, like in python using triple single, or double quotes.
script '''I can't use "test" text'''
// or
script """I can't use "test" text"""
Use a here-document, and use command subsitution to turn it into an argument.
script "$(cat <<'EOF'
I can't use "test" text
EOF)"
Or
Related
As per Change gnome-terminal title to reflect the current directory? I have tried to use:
PROMPT_COMMAND='echo -ne "\033]0;$(basename ${PWD})\007"'
... in my ~/.bashrc to indicate the name of the current directory in the gnome-terminal title bar.
For the most part it works fine; but if I try to open gnome-terminal from a directory like:
/run/user/1000/gvfs/mtp:host=%5Busb%3A001%2C008%5D/Internal shared storage/DCIM/Camera
... then it fails with:
basename: extra operand ‘storage/DCIM/Camera’
Try 'basename --help' for more information.
... every time the prompt is about to be shown in terminal.
Apparently the problem are the spaces in the directory name (even if the curly braces in ${PWD} should in principle obviate the need for quotes). So, I tried escaping with quotes, and it turns out it is not trivial (apparently just a single quote escape \"${PWD}\" does not work) - best I could get to is:
PROMPT_COMMAND='echo -ne "\033]0;$(basename \\\\"${PWD}\\\\")\007"'
... which works without an error - but it prints an extra backslash in the gnome-terminal title; for instance, in this case, the title is: Camera\.
What would be the correct construction of PROMPT_COMMAND, so it handles directories with spaces correctly, and it prints only the directory (e.g. Camera) without an appended backslash?
Why are you trying to escape your double quotes?
PROMPT_COMMAND='echo -ne "\033]0;$(basename "${PWD}")\007"'
would be correct. We can go through the levels of nesting:
top-layer single quotes. Inside single quotes, you do not need to escape double quotes. You would need to "escape" single quotes via '\'' (actually, this is string concatenation, but that's not relevant here).
second-layer double quotes. Inside double quotes, if you were trying to write literal ", you would need \". However, you have a third layer:
$() / command substitution. This layer, when executed, won't see the double quotes around it (single quotes would be a different story - you'd need '\''). Bash will correctly parse something like echo "$(echo "my variable is $foo")".
Also, the curly braces in ${PWD} have nothing to do with quotes. There is no difference between ${PWD} and $PWD from any point of view. Curly braces only come into play if your variable butts up against other "word" characters:
echo PWD_$PWD_USER_$USER // expands variables "USER" and "PWD_USER_"
echo PWD_${PWD}_USER_$USER // expands variables "USER" and "PWD"
I want to run a command from a bash script which has single quotes and some other commands inside the single quotes and a variable.
e.g. repo forall -c '....$variable'
In this format, $ is escaped and the variable is not expanded.
I tried the following variations but they were rejected:
repo forall -c '...."$variable" '
repo forall -c " '....$variable' "
" repo forall -c '....$variable' "
repo forall -c "'" ....$variable "'"
If I substitute the value in place of the variable the command is executed just fine.
Please tell me where am I going wrong.
Inside single quotes everything is preserved literally, without exception.
That means you have to close the quotes, insert something, and then re-enter again.
'before'"$variable"'after'
'before'"'"'after'
'before'\''after'
Word concatenation is simply done by juxtaposition. As you can verify, each of the above lines is a single word to the shell. Quotes (single or double quotes, depending on the situation) don't isolate words. They are only used to disable interpretation of various special characters, like whitespace, $, ;... For a good tutorial on quoting see Mark Reed's answer. Also relevant: Which characters need to be escaped in bash?
Do not concatenate strings interpreted by a shell
You should absolutely avoid building shell commands by concatenating variables. This is a bad idea similar to concatenation of SQL fragments (SQL injection!).
Usually it is possible to have placeholders in the command, and to supply the command together with variables so that the callee can receive them from the invocation arguments list.
For example, the following is very unsafe. DON'T DO THIS
script="echo \"Argument 1 is: $myvar\""
/bin/sh -c "$script"
If the contents of $myvar is untrusted, here is an exploit:
myvar='foo"; echo "you were hacked'
Instead of the above invocation, use positional arguments. The following invocation is better -- it's not exploitable:
script='echo "arg 1 is: $1"'
/bin/sh -c "$script" -- "$myvar"
Note the use of single ticks in the assignment to script, which means that it's taken literally, without variable expansion or any other form of interpretation.
The repo command can't care what kind of quotes it gets. If you need parameter expansion, use double quotes. If that means you wind up having to backslash a lot of stuff, use single quotes for most of it, and then break out of them and go into doubles for the part where you need the expansion to happen.
repo forall -c 'literal stuff goes here; '"stuff with $parameters here"' more literal stuff'
Explanation follows, if you're interested.
When you run a command from the shell, what that command receives as arguments is an array of null-terminated strings. Those strings may contain absolutely any non-null character.
But when the shell is building that array of strings from a command line, it interprets some characters specially; this is designed to make commands easier (indeed, possible) to type. For instance, spaces normally indicate the boundary between strings in the array; for that reason, the individual arguments are sometimes called "words". But an argument may nonetheless have spaces in it; you just need some way to tell the shell that's what you want.
You can use a backslash in front of any character (including space, or another backslash) to tell the shell to treat that character literally. But while you can do something like this:
reply=\”That\'ll\ be\ \$4.96,\ please,\"\ said\ the\ cashier
...it can get tiresome. So the shell offers an alternative: quotation marks. These come in two main varieties.
Double-quotation marks are called "grouping quotes". They prevent wildcards and aliases from being expanded, but mostly they're for including spaces in a word. Other things like parameter and command expansion (the sorts of thing signaled by a $) still happen. And of course if you want a literal double-quote inside double-quotes, you have to backslash it:
reply="\"That'll be \$4.96, please,\" said the cashier"
Single-quotation marks are more draconian. Everything between them is taken completely literally, including backslashes. There is absolutely no way to get a literal single quote inside single quotes.
Fortunately, quotation marks in the shell are not word delimiters; by themselves, they don't terminate a word. You can go in and out of quotes, including between different types of quotes, within the same word to get the desired result:
reply='"That'\''ll be $4.96, please," said the cashier'
So that's easier - a lot fewer backslashes, although the close-single-quote, backslashed-literal-single-quote, open-single-quote sequence takes some getting used to.
Modern shells have added another quoting style not specified by the POSIX standard, in which the leading single quotation mark is prefixed with a dollar sign. Strings so quoted follow similar conventions to string literals in the ANSI standard version of the C programming language, and are therefore sometimes called "ANSI strings" and the $'...' pair "ANSI quotes". Within such strings, the above advice about backslashes being taken literally no longer applies. Instead, they become special again - not only can you include a literal single quotation mark or backslash by prepending a backslash to it, but the shell also expands the ANSI C character escapes (like \n for a newline, \t for tab, and \xHH for the character with hexadecimal code HH). Otherwise, however, they behave as single-quoted strings: no parameter or command substitution takes place:
reply=$'"That\'ll be $4.96, please," said the cashier'
The important thing to note is that the single string that gets stored in the reply variable is exactly the same in all of these examples. Similarly, after the shell is done parsing a command line, there is no way for the command being run to tell exactly how each argument string was actually typed – or even if it was typed, rather than being created programmatically somehow.
Below is what worked for me -
QUOTE="'"
hive -e "alter table TBL_NAME set location $QUOTE$TBL_HDFS_DIR_PATH$QUOTE"
EDIT: (As per the comments in question:)
I've been looking into this since then. I was lucky enough that I had repo laying around. Still it's not clear to me whether you need to enclose your commands between single quotes by force. I looked into the repo syntax and I don't think you need to. You could used double quotes around your command, and then use whatever single and double quotes you need inside provided you escape double ones.
just use printf
instead of
repo forall -c '....$variable'
use printf to replace the variable token with the expanded variable.
For example:
template='.... %s'
repo forall -c $(printf "${template}" "${variable}")
Variables can contain single quotes.
myvar=\'....$variable\'
repo forall -c $myvar
I was wondering why I could never get my awk statement to print from an ssh session so I found this forum. Nothing here helped me directly but if anyone is having an issue similar to below, then give me an up vote. It seems any sort of single or double quotes were just not helping, but then I didn't try everything.
check_var="df -h / | awk 'FNR==2{print $3}'"
getckvar=$(ssh user#host "$check_var")
echo $getckvar
What do you get? A load of nothing.
Fix: escape \$3 in your print function.
Does this work for you?
eval repo forall -c '....$variable'
I run with the file with command line arguments:
samplebash.bsh fakeusername fakepassword&123
.bsh file:
echo "Beginning script..."
argUsername='$1'
argPassword='$2'
protractor indv.js --params.login.username=$argUsername --params.login.password=$argPassword
Output:
Beginning script...
123: command not found
The Issue: For some reason, it interprets what follows the & symbol from the password as a command, how do I avoid this?
The problem isn't happening in your script, it's happening in your original command line. & is a command terminator, which specifies that the command before it should be executed in the background. So your command was equivalent to:
samplebash.bsh fakeusername fakepassword &
123
You need to quote the argument to prevent special characters from being interpreted by the shell.
samplebash.bsh fakeusername 'fakepassword&123'
Also, you shouldn't put single quotes around a variable like you do in your assignments, that prevents the variable from being expanded. So it should be:
argUsername=$1
argPassword=$2
And you should put double quotes around the variables when you use them in the command, to prevent wildcards and whitespace from being interpreted.
protractor indv.js --params.login.username="$argUsername" --params.login.password="$argPassword"
As a general rule, you should always put double quotes around variables unless you know they're not needed.
I am trying to perform a cURL command within a bash script to POST to a URI. The command requires that one of the arguments be surrounded by double and single quotes i.e. '"jsimmons"' In my script however this argument is a variable so the command keeps failing which I believe is because the variable is doing some weird expansion and the command is losing the quotes necessary.
For my current attempt, which doesn't work, the argument looks like, '""$watcher""' as I am trying to expand the variable and place that string within the double and single quotes.
How can I expand my variable properly to fulfill the requirements of the command?
If you have double quotes around your whole command, you can insert single quotes without any trouble but need to escape double quotes.
For example:
$ watcher=jsimmons
$ echo "'\"$watcher\"'"
'"jsimmons"'
You can escape the surrounding 's and "s with \
\'\"$watcher\"\'
I have been searching the web for this but I can't understand what the unix shell terminal is doing when you just type a quotation mark "
$ "
and it gives you something like that
>
where you can enter text and probably commands.
Then if you enter again a single quotation mark character " it will quit the > prompt and go back to the regular $ prompt.
This is what the Bash manual states:
3.1.2.3 Double Quotes
Enclosing characters in double quotes (‘"’) preserves the literal
value of all characters within the quotes, with the exception of ‘$’,
‘’, ‘\’, and, when history expansion is enabled, ‘!’. The characters
‘$’ and ‘’ retain their special meaning within double quotes (see
Shell Expansions). The backslash retains its special meaning only when
followed by one of the following characters: ‘$’, ‘`’, ‘"’, ‘\’, or
newline. Within double quotes, backslashes that are followed by one of
these characters are removed. Backslashes preceding characters without
a special meaning are left unmodified. A double quote may be quoted
within double quotes by preceding it with a backslash. If enabled,
history expansion will be performed unless an ‘!’ appearing in double
quotes is escaped using a backslash. The backslash preceding the ‘!’
is not removed.
When you type a single double quote, Bash is just waiting for you to finish with the second double quote.
" on a bash command line starts (or finishes, if there was another one earlier) a double-quoted string (rather obvious naming convention). ' starts a single-quoted string. bash expects these to occur in pairs, and the entire string enclosed by a pair of quotes of either type is subject to different rules regarding how things are expanded and/or combined within them. Since you are typing only a single ", bash continues reading more input, waiting for you to put in the closing quote before it decides what it thinks you're asking it to do.
Both types of quotes are useful for naming files, commands, arguments, etc. that would otherwise be split at a space or other character. Single quotes also affect how variables are expanded within the string. Those are the most common uses, but there are others. Read man bash for more information.
It's showing you that it's still treating it as one command. This let's you format longer commands over a number of lines for legibility.
AFAIK, parentheses or backslashes will give you the same feature in most shells.
EDIT: +1 on Charles Duffy's comment; just to further expand on the differences between the three I mentioned above (I didn't realise, so might be useful to point them out). In a Bash shell (OSX):
Quotation marks: (Add a new line)
Using quotation marks, as Charles Duffy mentioned, allow you to put your argument over multiple lines. The new line character becomes part of your argument however, e.g.:
$ touch "hello
> world"
Will give you a filename that has a newline control character as part of it's name.
Backslash (continue same command)
Adding a backslash to the end of one line will enter multi-line mode, but will continue to add onto the same command, e.g.:
$ touch hello \
> world
Will give the same as touch hello world (i.e., passing two parameters to touch & so creating two files hello and world).
Brackets (execute multiple commands)
Parentheses will let you chain commands together, so:
$ (touch hello
> world)
Will execute touch hello and then world, i.e. it's the equivalent of doing touch hello; world on a single line. (probably giving -bash: world: command not found for the world command).
So quotation marks will enter multi-line input & also preserve any new line characters that you input as part of that.
like hexacyanide said, it is waiting for you to complete your command.
Double quote expands shell variables. So you could build your command with variables. I would add an example:
kent$ x=eq
kent$ "s$x" 3
1
2
3