Recently I have been having some trouble using GnuWin32 from PowerShell whenever double quotes are involved.
Upon further investigation, it appears PowerShell is stripping double quotes from command line arguments, even when properly escaped.
PS C:\Documents and Settings\Nick> echo '"hello"'
"hello"
PS C:\Documents and Settings\Nick> echo.exe '"hello"'
hello
PS C:\Documents and Settings\Nick> echo.exe '\"hello\"'
"hello"
Notice that the double quotes are there when passed to PowerShell's echo cmdlet, but when passed as an argument to echo.exe, the double quotes are stripped unless escaped with a backslash (even though PowerShell's escape character is a backtick, not a backslash).
This seems like a bug to me. If I am passing the correct escaped strings to PowerShell, then PowerShell should take care of whatever escaping may be necessary for however it invokes the command.
What is going on here?
For now, the fix is to escape command line arguments in accordance with these rules (which seem to be used (indirectly) by the CreateProcess API call which PowerShell uses to invoke .exe files):
To pass a double quote, escape with a backslash: \" -> "
To pass a one or more backslashes followed by a double quote, escape each backslash with another backslash and escape the quote: \\\\\" -> \\"
If not followed by a double quote, no escaping is necessary for backslashes: \\ -> \\
Note that further escaping of double quotes may be necessary to escape the double quotes in the Windows API escaped string to PowerShell.
Here are some examples, with echo.exe from GnuWin32:
PS C:\Documents and Settings\Nick> echo.exe "\`""
"
PS C:\Documents and Settings\Nick> echo.exe "\\\\\`""
\\"
PS C:\Documents and Settings\Nick> echo.exe "\\"
\\
I imagine that this can quickly become hell if you need to pass a complicated command line parameter. Of course, none of this documented in the CreateProcess() or PowerShell documentation.
Also note that this is not necessary to pass arguments with double quotes to .NET functions or PowerShell cmdlets. For that, you need only escape your double quotes to PowerShell.
Edit: As Martin pointed out in his excellent answer, this is documented in the CommandLineToArgv() function (which the CRT uses to parse the command line arguments) documentation.
It is a known thing:
It's FAR TOO HARD to pass parameters to applications which require quoted strings. I asked this question in IRC with a "roomful" of PowerShell experts, and it took hour for someone to figure out a way (I originally started to post here that it is simply not possible). This completely breaks PowerShell's ability to serve as a general purpose shell, because we can't do simple things like executing sqlcmd. The number one job of a command shell should be running command-line applications... As an example, trying to use SqlCmd from SQL Server 2008, there is a -v parameter which takes a series of name:value parameters. If the value has spaces in it, you must quote it...
...there is no single way to write a command line to invoke this application correctly, so even after you master all 4 or 5 different ways of quoting and escaping things, you're still guessing as to which will work when ... or, you can just shell out to cmd, and be done with it.
TL;DR
If you just want a solution for Powershell 5, see:
ConvertTo-ArgvQuoteForPoSh.ps: Powershell V5 (and C# Code) to allow escaping native command arguments
The Question I will try to answer
..., it appears PowerShell is stripping double quotes from command
line arguments, even when properly escaped.
PS C:\Documents and Settings\Nick> echo.exe '"hello"'
hello
PS C:\Documents and Settings\Nick> echo.exe '\"hello\"'
"hello"
Notice that the double quotes are there when passed to PowerShell's
echo cmdlet, but when passed as an argument to echo.exe, the double
quotes are stripped unless escaped with a backslash (even though
PowerShell's escape character is a backtick, not a backslash).
This seems like a bug to me. If I am passing the correct escaped
strings to PowerShell, then PowerShell should take care of whatever
escaping may be necessary for however it invokes the command.
What is going on here?
The Non-Powershell Background
The fact that you need to escape the quotes with backslashes \ has nothing to to with powershell, but with the CommandLineToArgvW function that is used by all msvcrt and C# programs to build the argv array from the single-string command line that the Windows process gets passed.
The details are explained at Everyone quotes command line arguments the wrong way and it basically boils down to the fact that this function historically has very uninutitive escaping rules:
2n backslashes followed by a quotation mark produce n backslashes followed by begin/end quote. This does not become part of the parsed
argument, but toggles the "in quotes" mode.
(2n) + 1 backslashes followed by a quotation mark again produce n backslashes followed by a quotation mark literal ("). This does not
toggle the "in quotes" mode.
n backslashes not followed by a quotation mark simply produce n backslashes.
leading to the described generic escaping function (shortquote of the logic here):
CommandLine.push_back (L'"');
for (auto It = Argument.begin () ; ; ++It) {
unsigned NumberBackslashes = 0;
while (It != Argument.end () && *It == L'\\') {
++It;
++NumberBackslashes;
}
if (It == Argument.end ()) {
// Escape all backslashes, but let the terminating
// double quotation mark we add below be interpreted
// as a metacharacter.
CommandLine.append (NumberBackslashes * 2, L'\\');
break;
} else if (*It == L'"') {
// Escape all backslashes and the following
// double quotation mark.
CommandLine.append (NumberBackslashes * 2 + 1, L'\\');
CommandLine.push_back (*It);
} else {
// Backslashes aren't special here.
CommandLine.append (NumberBackslashes, L'\\');
CommandLine.push_back (*It);
}
}
CommandLine.push_back (L'"');
The Powershell specifics
Now, up to Powershell 5 (including PoSh 5.1.18362.145 on Win10/1909) PoSh knows basically diddly about these rules, nor should it arguably, because these rules are not really general, because any executable you call could, in theory, use some other means to interpret the passed command line.
Which leads us to -
The Powershell Quoting Rules
What PoSh does do however is try to figure out whether the strings you pass it as arguments to the native commands need to be quoted because they contain whitespace.
PoSh - in contrast to cmd.exe - does a lot more parsing on the command you hand it, since it has to resolve variables and knows about multiple arguments.
So, given a command like
$firs = 'whaddyaknow'
$secnd = 'it may have spaces'
$third = 'it may also have "quotes" and other \" weird \\ stuff'
EchoArgs.exe $firs $secnd $third
Powershell has to take a stance on how to create the single string CommandLine for the Win32 CreateProcess (or rather the C# Process.Start) call it will evetually have to do.
The approach Powershell takes is weird and got more complicated in PoSh V7 , and as far as I can follow, it's got to do how powershell treats unbalanced quotes in unquoted string. The long stories short is this:
Powershell will auto-quote (enclose in <">) a single argument
string, if it contains spaces and the spaces don't mix with an
uneven number of (unsescaped) double quotes.
The specific quoting rules of PoSh V5 make it impossible to pass a certain category of string as single argument to a child process.
PoSh V7 fixed this, so that as long as all quotes are \" escaped -- which they need to be anyway to get them through CommandLineToArgvW -- we can pass any aribtrary string from PoSh to a child executable that uses CommandLineToArgvW.
Here's the rules as C# code as extracted from the PoSh github repo for a tool class of ours:
PoSh Quoting Rules V5
public static bool NeedQuotesPoshV5(string arg)
{
// bool needQuotes = false;
int quoteCount = 0;
for (int i = 0; i < arg.Length; i++)
{
if (arg[i] == '"')
{
quoteCount += 1;
}
else if (char.IsWhiteSpace(arg[i]) && (quoteCount % 2 == 0))
{
// needQuotes = true;
return true;
}
}
return false;
}
PoSh Quoting Rules V7
internal static bool NeedQuotesPoshV7(string arg)
{
bool followingBackslash = false;
// bool needQuotes = false;
int quoteCount = 0;
for (int i = 0; i < arg.Length; i++)
{
if (arg[i] == '"' && !followingBackslash)
{
quoteCount += 1;
}
else if (char.IsWhiteSpace(arg[i]) && (quoteCount % 2 == 0))
{
// needQuotes = true;
return true;
}
followingBackslash = arg[i] == '\\';
}
// return needQuotes;
return false;
}
Oh yeah, and they also added in a half baked attempt to correctly escape the and of the quoted string in V7:
if (NeedQuotes(arg))
{
_arguments.Append('"');
// need to escape all trailing backslashes so the native command receives it correctly
// according to http://www.daviddeley.com/autohotkey/parameters/parameters.htm#WINCRULESDOC
_arguments.Append(arg);
for (int i = arg.Length - 1; i >= 0 && arg[i] == '\\'; i--)
{
_arguments.Append('\\');
}
_arguments.Append('"');
The Powershell Situation
Input to EchoArgs | Output V5 (powershell.exe) | Output V7 (pwsh.exe)
===================================================================================
EchoArgs.exe 'abc def' | Arg 0 is <abc def> | Arg 0 is <abc def>
------------------------------|-----------------------------|---------------------------
EchoArgs.exe '\"nospace\"' | Arg 0 is <"nospace"> | Arg 0 is <"nospace">
------------------------------|-----------------------------|---------------------------
EchoArgs.exe '"\"nospace\""' | Arg 0 is <"nospace"> | Arg 0 is <"nospace">
------------------------------|-----------------------------|---------------------------
EchoArgs.exe 'a\"bc def' | Arg 0 is <a"bc> | Arg 0 is <a"bc def>
| Arg 1 is <def> |
------------------------------|-----------------------------|---------------------------
...
I'm snipping further examples here for time reasons. They shouldn't add overmuch to the answer anyways.
The Powershell Solution
To pass arbitrary Strings from Powershell to a native command using CommandLineToArgvW, we have to:
properly escape all quotes and Backslashes in the source argument
This means recognizing the special string-end handling for backslashes that V7 has. (This part is not implemented in the code below.)
and determine whether powershell will auto-quote our escaped string and if it won't auto-quote it, quote it ourselves.
and make sure that the string we quoted ourselves then doesn't get auto-quoted by powershell: This is what breaks V5.
Powershell V5 Source code for correctly escaping all arguments to any native command
I've put the full code on Gist, as it got too long to include here: ConvertTo-ArgvQuoteForPoSh.ps: Powershell V5 (and C# Code) to allow escaping native command arguments
Note that this code tries it's best, but for some strings with quotes in the payload and V5 you simply must add in leading space to the arguments you pass. (See code for logic details).
I personally avoid using '\' to escape things in PowerShell, because it's not technically a shell escape character. I've gotten unpredictable results with it. In double-quoted strings, you can use "" to get an embedded double-quote, or escape it with a back-tick:
PS C:\Users\Droj> "string ""with`" quotes"
string "with" quotes
The same goes for single quotes:
PS C:\Users\Droj> 'string ''with'' quotes'
string 'with' quotes
The weird thing about sending parameters to external programs is that there is additional level of quote evaluation. I don't know if this is a bug, but I'm guessing it won't be changed, because the behavior is the same when you use Start-Process and pass in arguments. Start-Process takes an array for the arguments, which makes things a bit clearer, in terms of how many arguments are actually being sent, but those arguments seem to be evaluated an extra time.
So, if I have an array, I can set the argument values to have embedded quotes:
PS C:\cygwin\home\Droj> $aa = 'arg="foo"', 'arg=""""bar""""'
PS C:\cygwin\home\Droj> echo $aa
arg="foo"
arg=""""bar""""
The 'bar' argument has enough to cover the extra hidden evaluation. It's as if I send that value to a cmdlet in double-quotes, then send that result again in double-quotes:
PS C:\cygwin\home\Droj> echo "arg=""""bar""""" # level one
arg=""bar""
PS C:\cygwin\home\Droj> echo "arg=""bar""" # hidden level
arg="bar"
One would expect these arguments to be passed to external commands as-is, as they are to cmdlets like 'echo'/'write-output', but they are not, because of that hidden level:
PS C:\cygwin\home\Droj> $aa = 'arg="foo"', 'arg=""""bar""""'
PS C:\cygwin\home\Droj> start c:\cygwin\bin\echo $aa -nonew -wait
arg=foo arg="bar"
I don't know the exact reason for it, but the behavior is as if there is another, undocumented step being taken under the covers that re-parses the strings. For example, I get the same result if I send the array to a cmdlet, but add a parsing level by doing it through invoke-expression:
PS C:\cygwin\home\Droj> $aa = 'arg="foo"', 'arg=""""bar""""'
PS C:\cygwin\home\Droj> iex "echo $aa"
arg=foo
arg="bar"
...which is exactly what I get when I send these arguments to my external Cygwin instance's 'echo.exe':
PS C:\cygwin\home\Droj> c:\cygwin\bin\echo 'arg="foo"' 'arg=""""bar""""'
arg=foo arg="bar"
With PowerShell 7.2.0, it is finally possible for arguments passed to native executables to behave as expected. This is currently an experimental feature and needs to be enabled manually.
Enable-ExperimentalFeature PSNativeCommandArgumentPassing
After that edit your PSProfile, for example, using notepad:
notepad.exe $PROFILE
Add $PSNativeCommandArgumentPassing = 'Standard' to the top of the file. You may instead also use $PSNativeCommandArgumentPassing = 'Windows' which uses the Legacy behaviour for some native executables. The differences are documented in this pull request.
Finally, restart PowerShell. Command arguments will no longer have quotes removed.
The new behaviour can be verified with this little C program:
#include <stdio.h>
int main(int argc, char** argv) {
for (int i = 1; i < argc; i++) {
puts(argv[i]);
}
return 0;
}
Compile it with gcc and pass in some arguments with quotes, like a JSON string.
> gcc echo-test.c
> ./a.exe '{"foo": "bar"}'
With the Legacy behaviour, the output is {foo: bar}. However, with the Standard option, the output becomes {"foo": "bar"}.
Relying on the CMD to shell out the issue as indicated in the accepted answer didn't work for me as double quotes were still stripped out when calling the CMD executable.
The good solution for me was to structure my command line as an array of strings instead of a single full string containing all the arguments. Then simply pass that array as the arguments for the binary invocation:
$args = New-Object System.Collections.ArrayList
$args.Add("-U") | Out-Null
$args.Add($cred.UserName) | Out-Null
$args.Add("-P") | Out-Null
$args.Add("""$($cred.Password)""")
$args.Add("-i") | Out-Null
$args.Add("""$SqlScriptPath""") | Out-Null
& SQLCMD $args
In that case, double quotes surrounding arguments are properly passed to the invoked command.
If you need, you can test and debug it with EchoArgs from the PowerShell Community Extensions.
This seems to be fixed in recent versions of PowerShell at the time of this writing, so it is no longer something to worry about.
If you still think you see this issue, remember it may be related to something else, such as the program that invokes PowerShell, so if you cannot reproduce it when invoking PowerShell directly from a command prompt or the ISE, you should debug elsewhere.
For example, I found this question when investigating a problem of disappearing quotes when running a PowerShell script from C# code using Process.Start. The issue was actually C# Process Start needs Arguments with double quotes - they disappear.
Oh dear. Clearly trying to escape double quotes to get them into PowerShell from the command line, or worse, some other language you are using to generate such a command line, or execution environments which might chain PowerShell scripts, can be a colossal waste of time.
As an attempt at a practical solution, what can we do instead? Silly-looking workarounds can sometimes be effective:
powershell Write-Host "'say ___hi___'.Replace('___', [String][Char]34)"
But it depends a lot on how this is being executed. Note that if you want that command to have the same results when pasted in PowerShell instead of run from command prompt, you need those outer double quotes! Because the hosting Powershell turns the expression into a string object which becomes just one more parameter to 'powershell.exe'
PS> powershell Write-Host 'say ___hi___'.Replace('___', [String][Char]34)
Which then, I guess, parses its arguments as Write-Host say "hi"
So the quotes you are trying so hard to reintroduce with string.Replace() will just disappear!
Related
On linux I can rappresent the / char in a different way:
${HOME:0:1}
So, for example, cat ${HOME:0:1}etc${HOME:0:1}passwd would be treated like cat /etc/passwd
Is there any way I can do the same thing on windows via powershell and cmd.exe for the backslash?
PowerShell has no equivalent to the parameter expansions available in POSIX-compatible shells such as Bash, of which your substring extraction (${HOME:0:1} to get the substring of length 1 at character position 0, i.e the first char. of the value of variable $HOME) is an example (link is to the Bash manual).
However, PowerShell makes it easy:
to embed the results of arbitrary expressions and even whole statements inside expandable (double-quoted) string ("..."), using $(...), the subexpression operator.
to pass the results of any expression or command (pipeline) as an argument to a command, by enclosing it in (...), the grouping operator.
The following command variations are equivalent, and dynamically use the platform-appropriate path (directory) separator, i.e. / on Unix-like platforms, and \ on Windows:
# -> '/etc/passwd' on Unix
# -> '\etc\passwd' on Windows
Write-Output "$([System.IO.Path]::DirectorySeparatorChar)etc$([System.IO.Path]::DirectorySeparatorChar)passwd"
# Ditto.
Write-Output ('{0}etc{0}passwd' -f [System.IO.Path]::DirectorySeparatorChar)
See also:
[System.IO.Path]::DirectorySeparatorChar
-f, the string format operator
How to set preference $ErrorView = "CategoryView" before start powershell.exe ?
powershell.exe -command "$ErrorView = "CategoryView" ; dir wrong.txt" doesnt work.
your code has a serious error in it. you used 4 double quotes instead of two on the outside and a pair of single quotes on the inside. [grin]
this works ...
powershell.exe -command "$ErrorView = 'CategoryView' ; dir wrong.txt; pause"
remove the pause when you are certain things are working as needed. [grin]
To complement Lee Dailey's helpful answer: As Lee points out, your primary problem is that you neglected to escape the " chars. embedded in your overall "..." command.
Assuming that you're calling your command from outside of PowerShell, such as from cmd.exe (Command Prompt):
Using embedded single-quoting ('...') in lieu of the embedded "..." is an option in this case, as shown in Lee's answer, because CategoryView is to be treated as a literal string.
Using ' for the embedded quoting conveniently obviates the need for escaping.
However, in cases where the embedded string contains variable references (e.g., $var) or expressions (e.g, $(Get-Date)), use of a double-quoted string ("...") is a must, because only double-quoted strings are expandable (interpolated). Escaping the embedded " as \" is then a must.
Note that, by contrast, inside PowerShell " chars. must be escaped as `".
# From cmd.exe, for instance.
C:\>powershell.exe -command "$ErrorView = \"CategoryView\"; dir wrong.txt"
If, for some reason, you must invoke another PowerShell instance from within PowerShell, use a script block ({ ... }), which also obviates the need for escaping (and better integrates with the calling session by returning objects from the invocation, not just strings).
# From Powershell.
PS> powershell.exe -command { $ErrorView = "CategoryView" ; dir wrong.txt }
I'm trying to get ACLs and parse into the array reg_perms, the code works fine without the Where-Object{($_.IdentityReference -eq "BUILTIN\Users")
command ='powershell "(Get-Acl \'HKLM:\SOFTWARE\\Microsoft\Windows NT\CurrentVersion\Winlogon\').Access | Where-Object{($_.IdentityReference -eq "BUILTIN\Users")} | Format-List RegistryRights,AccessControlType,IdentityReference"'
data = ::Mixlib::ShellOut.new(command).run_command.stdout.strip.gsub(/\r\n?/, "\n")
reg_perms = data.split("\n\n").each_with_object([]) do |set, arr|
arr << set.split("\n").map do |f|
f.split(':').collect(&:strip)
end.to_h
end
You are using single quotes for your entire string: '. Then when your string is evaled with double quotes the double quotes around the BUILTIN\Users string are not escaped, this mean you need to escape the double quotes around the ""BUILTIN\Users"" the powershell way or use single quotes \'BUILTIN\Users\' and escape them the ruby way.
This should work:
command ='powershell "(Get-Acl \'HKLM:\SOFTWARE\\Microsoft\Windows NT\CurrentVersion\Winlogon\').Access | Where-Object{
($_.IdentityReference -eq \'BUILTIN\Users\')
} | Format-List RegistryRights,AccessControlType,IdentityReference"'
You're trying to embed "BUILTIN\Users" - a double-quoted string - within the overall double-quoted command string you're passing to the PowerShell executable (powershell "..."), which cannot work, because embedding the same type of quotes in a quoted string requires escaping.
PowerShell, when it is called from outside via its CLI (powershell.exe), requires that embedded " chars. be \"-escaped (even though PowerShell-internally, `" or "" are used).[1]
Since you're using single-quoting on the Ruby (Chef) side (command = '...'), escaping the inner embedded " chars. as \" for the sake of PowerShell is sufficient.
Therefore, replace -eq "BUILTIN\Users" with -eq \"BUILTIN\Users\"; i.e.:
command ='powershell "(Get-Acl \'HKLM:\SOFTWARE\\Microsoft\Windows NT\CurrentVersion\Winlogon\').Access | Where-Object{($_.IdentityReference -eq \"BUILTIN\Users\")} | Format-List RegistryRights,AccessControlType,IdentityReference"'
Alternatively - given that the content of your quoted string is a literal, you can use single quotes around BUILTIN\Users in the PowerShell command; in that case, however, because on the Ruby side you're using single quotes for the overall command, you need to escape the embedded ' instances as \' for Ruby's benefit:
Therefore, replace -eq "BUILTIN\Users" with -eq \'BUILTIN\Users\'; i.e.:
command ='powershell "(Get-Acl \'HKLM:\SOFTWARE\\Microsoft\Windows NT\CurrentVersion\Winlogon\').Access | Where-Object{($_.IdentityReference -eq \'BUILTIN\Users\')} | Format-List RegistryRights,AccessControlType,IdentityReference"'
[1] When calling the PowerShell CLI from cmd.exe - be it from the command prompt, a batch file, or via Chef/Ruby - escaping a literal " as "" sometimes , but not always works (try
powershell -Command "'Nat ""King"" Cole'" directly from a cmd.exe command prompt).
Instead, \"-escaping is the safe choice.
`"-escaping, which is the typical PowerShell-internal way to escape " inside "...", never works in this scenario.
A Perl system call must send the following string to the UnixShell:
'"XYZ"'
In my Perl script I have used the following command:
system("cleartool mkattr -replace ATTRIBUTE '"$attribute"' lbtype:$label");
Everything is well passed to the Shell Unix, except both uses of the quote character:
'
Indeed,
cleartool mkattr -replace ATTRIBUTE
The above command is passed as it is exactly what I want.
The Perl variables $attribute and $label are well interpreted.
But I don't know what to do to obtain exactly:
'"XYZ"'
Here XYZ is the value of the Perl variable $attribute
OS is AIX (Unix) and Shell is ksh. cleartool is the command line interface of Clearcase but no Clearcase skill is necessary to fix my problem.
If you want to execute a system command and don't have to use any shell syntax like redirects, it's usually better and safer to use the list form of system:
system(
'cleartool', 'mkattr', '-replace', 'ATTRIBUTE',
qq{"$attribute"}, qq{lbtype:$label}
);
# or, if you really want to pass both types of quotes:
system(
'cleartool', 'mkattr', '-replace', 'ATTRIBUTE',
qq{'"$attribute"'}, qq{lbtype:$label}
);
See perldoc -f system
It's not clear from your question if you want to pass '"XYZ"' or "XYZ".
See "Quote and Quote like Operators" and use qq{...}:
system(qq{cleartool mkattr -replace ATTRIBUTE '"$attribute"' lbtype:$label});
qq{...} is exactly like "..." except you can then use double quotes " in your string without escaping them.
You can use any character directly after the qq and must then use the same character to denote the end-of-string, i.e. qqX...X would work the same way. You would run into problems if your string contains Xes, so don't do that.
You can also use paired characters as delimiter ({}, (), <>) which is what you usually will see.
In Unix I could run myscript '"test"' and I would get "test".
In Windows cmd I get 'test'.
How can I pass double-quotes as a parameter? I would like to know how to do this manually from a cmd window so I don't have to write a program to test my program.
Another way to escape quotes (though probably not preferable), which I've found used in certain places is to use multiple double-quotes. For the purpose of making other people's code legible, I'll explain.
Here's a set of basic rules:
When not wrapped in double-quoted groups, spaces separate parameters:program param1 param2 param 3 will pass four parameters to program.exe: param1, param2, param, and 3.
A double-quoted group ignores spaces as value separators when passing parameters to programs:program one two "three and more" will pass three parameters to program.exe: one, two, and three and more.
Now to explain some of the confusion:
Double-quoted groups that appear directly adjacent to text not wrapped with double-quotes join into one parameter:hello"to the entire"world acts as one parameter: helloto the entireworld.
Note: The previous rule does NOT imply that two double-quoted groups can appear directly adjacent to one another.
Any double-quote directly following a closing quote is treated as (or as part of) plain unwrapped text that is adjacent to the double-quoted group, but only one double-quote:"Tim says, ""Hi!""" will act as one parameter: Tim says, "Hi!"
Thus there are three different types of double-quotes: quotes that open, quotes that close, and quotes that act as plain-text.
Here's the breakdown of that last confusing line:
" open double-quote group
T inside ""s
i inside ""s
m inside ""s
inside ""s - space doesn't separate
s inside ""s
a inside ""s
y inside ""s
s inside ""s
, inside ""s
inside ""s - space doesn't separate
" close double-quoted group
" quote directly follows closer - acts as plain unwrapped text: "
H outside ""s - gets joined to previous adjacent group
i outside ""s - ...
! outside ""s - ...
" open double-quote group
" close double-quote group
" quote directly follows closer - acts as plain unwrapped text: "
Thus, the text effectively joins four groups of characters (one with nothing, however):
Tim says, is the first, wrapped to escape the spaces
"Hi! is the second, not wrapped (there are no spaces)
is the third, a double-quote group wrapping nothing
" is the fourth, the unwrapped close quote.
As you can see, the double-quote group wrapping nothing is still necessary since, without it, the following double-quote would open up a double-quoted group instead of acting as plain-text.
From this, it should be recognizable that therefore, inside and outside quotes, three double-quotes act as a plain-text unescaped double-quote:
"Tim said to him, """What's been happening lately?""""
will print Tim said to him, "What's been happening lately?" as expected. Therefore, three quotes can always be reliably used as an escape.However, in understanding it, you may note that the four quotes at the end can be reduced to a mere two since it technically is adding another unnecessary empty double-quoted group.
Here are a few examples to close it off:
program a b REM sends (a) and (b)
program """a""" REM sends ("a")
program """a b""" REM sends ("a) and (b")
program """"Hello,""" Mike said." REM sends ("Hello," Mike said.)
program ""a""b""c""d"" REM sends (abcd) since the "" groups wrap nothing
program "hello to """quotes"" REM sends (hello to "quotes")
program """"hello world"" REM sends ("hello world")
program """hello" world"" REM sends ("hello world")
program """hello "world"" REM sends ("hello) and (world")
program "hello ""world""" REM sends (hello "world")
program "hello """world"" REM sends (hello "world")
Final note: I did not read any of this from any tutorial - I came up with all of it by experimenting. Therefore, my explanation may not be true internally. Nonetheless all the examples above evaluate as given, thus validating (but not proving) my theory.
I tested this on Windows 7, 64bit using only *.exe calls with parameter passing (not *.bat, but I would suppose it works the same).
I cannot quickly reproduce the symptoms: if I try myscript '"test"' with a batch file myscript.bat containing just #echo.%1 or even #echo.%~1, I get all quotes: '"test"'
Perhaps you can try the escape character ^ like this: myscript '^"test^"'?
Try this:
myscript """test"""
"" escape to a single " in the parameter.
The 2nd document quoted by Peter Mortensen in his comment on the answer of Codesmith made things much clearer for me. That document was written by windowsinspired.com. The link repeated: A Better Way To Understand Quoting and Escaping of Windows Command Line Arguments.
Some further trial and error leads to the following guideline:
Escape every double quote " with a caret ^. If you want other characters with special meaning to the Windows command shell (e.g., <, >, |, &) to be interpreted as regular characters instead, then escape them with a caret, too.
If you want your program foo to receive the command line text "a\"b c" > d and redirect its output to file out.txt, then start your program as follows from the Windows command shell:
foo ^"a\^"b c^" ^> d > out.txt
If foo interprets \" as a literal double quote and expects unescaped double quotes to delimit arguments that include whitespace, then foo interprets the command as specifying one argument a"b c, one argument >, and one argument d.
If instead foo interprets a doubled double quote "" as a literal double quote, then start your program as
foo ^"a^"^"b c^" ^> d > out.txt
The key insight from the quoted document is that, to the Windows command shell, an unescaped double quote triggers switching between two possible states.
Some further trial and error implies that in the initial state, redirection (to a file or pipe) is recognized and a caret ^ escapes a double quote and the caret is removed from the input. In the other state, redirection is not recognized and a caret does not escape a double quote and isn't removed. Let's refer to these states as 'outside' and 'inside', respectively.
If you want to redirect the output of your command, then the command shell must be in the outside state when it reaches the redirection, so there must be an even number of unescaped (by caret) double quotes preceding the redirection. foo "a\"b " > out.txt won't work -- the command shell passes the entire "a\"b " > out.txt to foo as its combined command line arguments, instead of passing only "a\"b " and redirecting the output to out.txt.
foo "a\^"b " > out.txt won't work, either, because the caret ^ is encountered in the inside state where it is an ordinary character and not an escape character, so "a\^"b " > out.txt gets passed to foo.
The only way that (hopefully) always works is to keep the command shell always in the outside state, because then redirection works.
If you don't need redirection (or other characters with special meaning to the command shell), then you can do without the carets. If foo interprets \" as a literal double quote, then you can call it as
foo "a\"b c"
Then foo receives "a\"b c" as its combined arguments text and can interpret it as a single argument equal to a"b c.
Now -- finally -- to the original question. myscript '"test"' called from the Windows command shell passes '"test"' to myscript. Apparently myscript interprets the single and double quotes as argument delimiters and removes them. You need to figure out what myscript accepts as a literal double quote and then specify that in your command, using ^ to escape any characters that have special meaning to the Windows command shell. Given that myscript is also available on Unix, perhaps \" does the trick. Try
myscript \^"test\^"
or, if you don't need redirection,
myscript \"test\"
I'm calling powershell from cmd, and passing quotes and neither escapes here worked. The grave accent worked to escape double quotes on this Win 10 surface pro.
>powershell.exe "echo la`"" >> test
>type test
la"
Below are outputs I got for other characters to escape a double quote:
la\
la^
la
la~
Using another quote to escape a quote resulted in no quotes.
As you can see, the characters themselves got typed, but didn't escape the double quotes.
Maybe you came here, because you wonder how to escape quotes that you need in the command that you pass to /c on cmd.exe? Well you don't:
CMD /c "MKDIR "foo bar""
will execute
MKDIR "foo bar"
which is really a behavior that I did not expect in the first glance.