I am trying to write a Windows batch file which will replace occurrences of angled brackets (><) with a newline in between.
I am new to PowerShell, but in searching though possible solutions, I have found the following works from PowerShell:
(get-content input.txt) -replace "><", ">`n<" | set-content output.txt
To use this within a windows batch, I need to wrap it inside
powershell -command "arguments"
So the final command is something like:
powershell -command "(gc input.txt) -replace '><', '>`n<' | sc output.txt"
However, this of course does not work because the single quotes around the replace text causes the grave quote escape character to be treated literally.
I have searched far and wide on the correct combination of escape characters to use to allow the PowerShell escape character to be recognised and have found a similar answer in here, but when I try this suggestion, I get a "< was unexpected at this time" error. I think what I need is more complicated because my search string also contains the angled brackets.
Look at the powershell.exe command line options. You can use a script block:
powershell -command {(gc input.txt) -replace "><", ">`n<" | sc output.txt}
Avoid using the escape character and double quotes?
powershell -command "(gc d:\t\input.txt) -replace '><', ('>'+[char]10+'<') | sc d:\t\output.txt"
I have solved the problem.
I also used delayed expansion, so the final command is:
powershell -Command "(gc !inputfile!) -replace (\"^>^<\", \"^>`n^<\") | sc !outputfile!"
So it actually uses three different types of escape characters! Combination of \ and ^ and `.
I wish I could say I worked it out logically, but in the end it was just a random attempt using different escapes on the ><.
But this is now a good reference on how to use PowerShell inside Windows batch without using single quotes which turn escape characters into literals.
Related
I asked for a first explanation "here" and "here" but going to try a more complex situation I was unable (after two hours of trying) to understand how to solve. I read how the regular expression works but nothing, I went into the ball.
The modified code is this:
(Fsutil Dirty Query %SystemDrive%>Nul)||(powershell.exe -c "[Environment]::CommandLine; Start -Verb RunAs cmd /k, ("^""%~f0"^"" -replace '[;,()= &^]', '^$&')" & echo exit)
and the folder with the poison characters is this:
C:\Users\fposc\Desktop\Pie & tea % # ' $^
I have tried to escape the ^ in the regular expression with \^ but don't work. I have escaped also ( and ) with \( and \). But nothing work:
(Fsutil Dirty Query %SystemDrive%>Nul)||(powershell.exe -c "[Environment]::CommandLine; Start -Verb RunAs cmd /c, ("^""%~f0"^"" -replace '[;,\(\)= &\^]', '^$&')" & exit)
I added the round brackets because I wanted to put all possible characters to make the code as generic as possible.
I don't know if I was right to open another question. Maybe I should change the original question? Since other combinations are possible and not having understood the mechanism I could open many other similar questions. What do you advise me to do?
The problem is the presence of $ in your folder name, which causes the PowerShell command to interpret it as the start of a variable reference.
The workaround is to use an aux. environment variable to store the batch file's full path and let PowerShell perform its escaping based on this variable's value:
:: Unless already elevated, re-invoke this batch file with elevation,
:: via PowerShell.
set "__THISFILE=%~f0"
Fsutil Dirty Query %SystemDrive% >Nul || (powershell.exe -c "Start-Process -Verb RunAs cmd /k, ($env:__THISFILE -replace '[ &%%^]', '^$&')" & exit)
I have updated the answer to your original question to incorporate this approach, which now shows a - hopefully - robust approach to on-demand re-invocation of a batch file with elevation, including support for arguments.
Update 1:
After looking carefully at the output again, I sort of figured it out.
By adding a trailing space between the closing parenthesis and the quotes it works:
powershell -ExecutionPolicy ByPass -File "$(SolutionDir)\BuildScripts\InjectGitVersion.ps1" "$(ProjectDir) " "$(TargetDir) "
I am suspecting that PowerShell somehow interprets the )".
Is there a more elegant way around this issue?
Update 2:
This is weird. I have another script that does the clean up and works with out the space between ) and " like this:
powershell -ExecutionPolicy ByPass -File "$(SolutionDir)\BuildScripts\InjectGitVersionCleanup.ps1" "$(ProjectDir)"
However adding trailing spaces will cause it to fail, because internally it appends a file-name to the path, so that the path will be incorrect.
If any one understand this, I would be happy to accept the explanation as the correct answer!
Original:
I have the following prebuild-command in VisualStudio, which I want to use to inject the version from a Git-tag:
powershell -ExecutionPolicy ByPass -File "$(SolutionDir)\BuildScripts\InjectGitVersion.ps1" $(ProjectDir) $(TargetDir)
This works fine, if the path $(SolutionDir) does not contain spaces (which will then also present in $(ProjectDir) or $(TargetDir)).
When the path $(SolutionDir) does contain spaces, it appears the script starts as expected, but the arguments are not passed correctly and I am unable to figure out how to escape them in the arguments to the PS-script.
I have tried adding sing ", triple """ and also ', which gives the following (each PS-command tries a different method for escaping the spaces):
powershell -ExecutionPolicy ByPass -File "$(SolutionDir)\BuildScripts\InjectGitVersion.ps1" $(ProjectDir) $(TargetDir)
args[0]:
D:\VisualStudio
args[1]:
Projects\software_git_repo\ProgramEditor\
powershell -ExecutionPolicy ByPass -File "$(SolutionDir)\BuildScripts\InjectGitVersion.ps1" "$(ProjectDir)" "$(TargetDir)"
BS: args[0]:
D:\VisualStudio Projects\software_git_repo\ProgramEditor" D:\VisualStudio
BS: args[1]:
Projects\software_git_repo\ProgramEditor\bin\Debug"
powershell -ExecutionPolicy ByPass -File "$(SolutionDir)\BuildScripts\InjectGitVersion.ps1" """$(ProjectDir)""" """$(TargetDir)"""
BS: args[0]:
"D:\VisualStudio
BS: args[1]:
Projects\software_git_repo\ProgramEditor"
powershell -ExecutionPolicy ByPass -File "$(SolutionDir)\BuildScripts\InjectGitVersion.ps1" '$(ProjectDir)' '$(TargetDir)'
BS: args[0]:
'D:\VisualStudio
BS: args[1]:
Projects\software_git_repo\ProgramEditor\'
Doing:
echo ProjectDir:
echo $(ProjectDir)
echo TargetDir:
echo $(TargetDir)
I get:
ProjectDir:
D:\VisualStudio Projects\software_git_repo\ProgramEditor\
TargetDir:
D:\VisualStudio Projects\software_git_repo\ProgramEditor\bin\Debug\
Beware unexpected escaped quotes when a VS directory macro is expanded in a quoted parameter.
A script parameter containing a path should be wrapped in quotes to avoid being interpreted as multiple parameters if the path includes spaces. The following would therefore seem reasonable in Visual Studio's pre/post build event command line:-
powershell.exe -file "$(SolutionDir)script.ps1" -projectDirectory "$(ProjectDir)"
But the macro values that specify directories always include a trailing backslash; so when they're expanded the line becomes, for example:-
powershell.exe -file "C:\sln path\script.ps1" -projectDirectory "C:\sln path\project\"
...where the trailing \" is an escaped character that is interpreted as part of the parameter value, which is passed to the script as:-
C:\sln path\project"
Solution
The fix in my case was to insert an additional backslash after the directory macro to be escaped instead of the quotes, which restores the expected parameter value and prevents the mangling of any subsequent params:-
powershell.exe -file "$(SolutionDir)script.ps1" -projectDirectory "$(ProjectDir)\"
Not very intuitive though. Let me know if I'm missing something more obvious...
One thing I noticed is an extra backslash after $(SolutionDir). I'm using VS2017 and I think consistently $(SolutionDir) has always had a backslash. Not sure though.
I have a .bat in my solution folder that creates a json config file. My Post-Build event looks like this.
call "$(SolutionDir)CreateConfigurationJson.bat" "$(TargetDir)mySettings.json"
{"ProjectName":"$(ProjectName)"}
"$(TargetDir)mySettings.json" is the first parameter of the .bat file
{"ProjectName":"$(ProjectName)"} is the second parameter.
No escape characters necessary. Hope this helps.
The code for the bat file is:
echo off
set filePath=%1
break>%filePath%
set json=%2
echo %json% >>%filePath%
thank you #tranquil_tarn for your great answer!
I struggled with the BATCH + Powershell escaping hell for multiple hours.
Posting final code here to hopefully help someone out.
Solution
#ECHO OFF
cls
echo .SYNOPSIS
echo Allows prebuild to be Powershell, instead of BATCH. (You'll thank me later)
echo Specifically this example shows passing in VisualStudio (ProjectDir) variable AND correctly handles spaces in the path.
echo .DESCRIPTION
echo Call prebuild.ps1 and pass in built-in VS Variables: https://learn.microsoft.com/en-us/visualstudio/ide/reference/pre-build-event-post-build-event-command-line-dialog-box?view=vs-2022
echo Use powershell best-practices named parameters rather than positional to avoid positional-unmatched errors.
echo If Username '$(User)' is blank DO NOT include the Switch at all to avoid powershell errors
REM TASK1: Check a variable for blank/empty and only include the Param if not empty
SET "user=$(User)"
IF [%user%]==[] (
echo "userParam is blank"
) ELSE (
SET "userParam=-User:%user%"
)
echo "userParam: %userParam%"
REM TASK2: Warning: (ProjectDir) in VisualStudio ends in a slash, which when expanded, results in escaped quote \' at end of line
REM if you DO NOT include the quotes, then spaces in path will break it. if you DO include quotes then it gets escaped.
REM solution is to include an extra slash at the end, resulting in double backslash \\ which escapes down to single backslash '\'... whew!
REM 'see https://stackoverflow.com/questions/52205117/how-to-escape-spaces-the-parameters-to-a-ps-script-running-in-vs-post-build-even/64146880#64146880'
cd "$(ProjectDir)\"
SET cmd1=powershell -NoProfile -ExecutionPolicy Unrestricted -Command ^
".\prebuild.ps1 -ProjectDir:'$(ProjectDir)\' -Conf:'$(Configuration)' -ProjectName:'$(ProjectName)' %userParam% "
REM For debugging, view the command (including escaped strings) before calling it.
echo %cmd1%
call %cmd1%
From my understanding, the invoke operator (&) and the Invoke-Expression cmdlet should behave similar. However, as can be seen below, this is not the case:
PS C:\Users\admin> powershell -Command "& {""([Text.Encoding]::UTF8.GetString([Convert]::FromBase64String('ZWNobyAnaGVsb
G93b3JsZCc=')))""}"
echo 'helloworld'
PS C:\Users\admin> powershell -Command "IEX ""([Text.Encoding]::UTF8.GetString([Convert]::FromBase64String('ZWNobyAnaGVs
bG93b3JsZCc=')))"""
helloworld
Here, 'ZWNobyAnaGVsbG93b3JsZCc=' is the Base64 encoded string "echo helloworld".
Can someone clarify?
Invoke-Expression (whose built-in alias is iex) and &, the call operator, serve different purposes:
Invoke-Expression evaluates a given string as PowerShell source code, as if you had executed the string's content directly as a command.
As such, it is similar to eval in bash and therefore only to be used with input that is fully under the caller's control or input that the caller trusts.
There are often better solutions available, so Invoke-Expression should generally be avoided
& is used to invoke a command (& <nameOrPath> [...]) or a script block (& { ... } [...]):
Neither case involves evaluating a string as source code.
In the case at hand:
The core of your command is the following expression, which returns the string
"echo 'helloworld'" (its content doesn't include the enclosing " - this is simply the representation of the resulting string as a PowerShell string literal):
[Text.Encoding]::UTF8.GetString([Convert]::FromBase64String('ZWNobyAnaGVsbG93b3JsZCc='))
Also note that, due to how the command line is parsed, the ""..."" surrounding the core expression in your original commands are effectively ignored, which explains why the expression is executed rather than being treated as the content of a string.[1]
Therefore, your two commands amount to:
& { "echo 'helloworld'" }
& executes the statement inside the script block, which happens to be a string, and a string by itself - if it isn't assigned to a variable or redirected elsewhere - is simply output as-is.
In this case, the command is effectively the same as just executing "echo 'helloworld'" by itself (including the enclosing ", which you can think of as
echo "echo 'helloworld'"), so echo 'helloworld' prints to the console.
Note that echo is a built-in alias for the Write-Output cmdlet, whose explicit use is rarely necessary: Return values from commands or expressions are implicitly output, if they are not captured in some form, as in this case, where executing a string by itself as a statement simply outputs the string. (You can try this by submitting just 'hi' at the prompt, for instance).
iex "echo 'helloworld'"
This makes iex (Invoke-Expression) evaluate the string's content as source code, which therefore executes echo 'helloworld', which prints helloworld to the console.
[1] Optional reading: PowerShell quoting woes when calling external programs
Note:
Handling of quoting with respect to external programs or when calling from an external programs is not part of the official documentation, as far as I can tell (as of this writing, neither about_Parsing nor about_Quoting_Rules nor about_Special_Characters mentions it - I've opened this issue on GitHub to address that).
There are flaws in the existing handling, but they cannot be fixed without breaking backward compatibility.
When calling from PowerShell, the best approach is to use a script block, which bypasses the quoting problems - see below.
Even though you correctly embedded " by escaping them as "" inside the overall "..." string from a PowerShell-internal perspective, additional escaping of " with \ is needed in order to pass them through to an external program, even if that external program is another instance of PowerShell called via powershell.exe.
Take this simplified example:
powershell.exe -command " ""hi"" " # !! BROKEN
powershell.exe -command ' "hi" ' # !! BROKEN
PowerShell-internally, " ""hi"" " and ' "hi" ' evaluate to a string with literal contents "hi" , which, when executed, prints hi.
Regrettably, PowerShell passes this string to powershell.exe as " "hi" " - note how the "" turned into plain " and the enclosing single quotes were replaced with double quotes - which effectively results in hi after parsing by the new instance (because " "hi" " is parsed as the concatenation of substrings " ", hi, and " "), so PowerShell ends up trying to execute a (presumably nonexistent) command named hi.
By contrast, if you manage to pass the embedded as " as \" (sic) - after meeting PowerShell's own escaping needs - the command works as intended.
Therefore, as stated, you need to combine PowerShell-internal escaping with for-the-CLI escaping in order to pass an embedded ", so that:
inside overall "...", each embedded " must be escaped as \"" (sic) or \`" (sic)
inside overall '...', \" can be used as-is.
powershell.exe -command " \""hi\"" " # OK
powershell.exe -command " \`"hi\`" " # OK
powershell.exe -command ' \"hi\" ' # OK
Alternatively, use a script block instead of a command string, which bypasses the quoting headaches:
powershell.exe -command { "hi" } # OK, but only works when calling from PS
Note that the script-block technique only works when calling from PowerShell, not from cmd.exe.
cmd.exe has its own quoting requirements:
Notably, cmd.exe only supports "" for embedding double quotes (not also `"); thus, among the solutions above, only
powershell.exe -command " \""hi\"" " works from cmd.exe (a batch file) without additional escaping.
The down-side of \"", however, is that runs of interior whitespace between \""...\"" are collapsed to a single space each. To avoid that, use \"...\", but cmd.exe then sees substrings between the \" instances as unquoted, which would cause the command to break if that substring contained metacharacters such as | or &; e.g., powershell.exe -command " \"a|b\" "; to fix that you must individually ^-escape the following characters: & | < > ^
powershell.exe -command ' "hi" ' is similarly brittle, because cmd.exe doesn't recognize ' as a string delimiter, so any metacharacters outside embedded "..." are again interpreted by cmd.exe itself; e.g., powershell.exe -command ' "hi" | Measure-Object '
Finally, using just "" from cmd.exe for embedding " sometimes works, but not reliably; e.g., powershell.exe -command " 'Nat ""King"" Cole' " prints Nat "King Cole (the closing " is missing).
This appears to have been fixed in PowerShell Core.
I am trying to replace a string including a line break in a file. I am using the command line for this.
I am trying to use the same command in a CMD shell and in PowerShell, however I can only seem to get it to work in the latter.
Here is the command:
powershell -Command "(Get-Content client.properties -Raw).Replace('#test`r`n','test`r`n') | Set-Content client2.properties"
Why is this not working in a CMD shell, and how do I make it work?
The `r`n escape sequence won't work inside single-quotes.
Use the -replace operator instead and use regex escapes:
powershell -Command "(Get-Content client.properties -Raw)-replace('#test\r?\n','test'+$([Environment]::NewLine)) | Set-Content client2.properties"
Im making a bat-file for windows that will run a simulation.
It will then look at the result in the console and search for the string "win%".
Ftm I can find anything including win but that gives me alot of unnessesary data.
This is what I got now:
command | findstr win >> file.txt
This gives me alot of unnessesary data.
I want to find:
command | findstr win% >> file.txt
But this dosent work at all....
How can I find the strings including "%"?
Br
Based on your question tags, your command appears within a batch script.
Percent literals must be escaped as %% within a batch script.
command | findstr win%% >> file.txt
The above will always work because your search string contains only one percent.
Note that each side of the pipe is executed in its own cmd.exe process using command line context (not batch). This could lead to a problem, depending on your search string and the current defined variables.
Suppose you wanted to search for win%lose%. The following might work:
command | findstr win%%lose%% >>file.txt
It works as long as there is no variable named lose in the environment. Since the FINDSTR command executes in a command line context, the %lose% string is preserved if lose is not defined.
But if lose is defined, then %lose% is expanded into the value and you get the wrong search. This could be solved by introducing a disappearing caret into the expression. A string like %^lose% will not expand a variable named lose. The variable expansion will include the caret as part of the name, not find anything, and leave the string intact. Afterwards, the normal escape phase will "escape" the l into itself, and the caret disappears. Now this would fail if a variable named ^lose is defined, but that is highly unlikely.
But the command is within a batch script, so the caret must be escaped.
command | findstr win%%^^lose%% >> file.txt
It is easier to simply enclose the string in quotes so you don't need to escape the caret.
command | findstr "win%%^lose%%" >> file.txt