On linux I can rappresent the / char in a different way:
${HOME:0:1}
So, for example, cat ${HOME:0:1}etc${HOME:0:1}passwd would be treated like cat /etc/passwd
Is there any way I can do the same thing on windows via powershell and cmd.exe for the backslash?
PowerShell has no equivalent to the parameter expansions available in POSIX-compatible shells such as Bash, of which your substring extraction (${HOME:0:1} to get the substring of length 1 at character position 0, i.e the first char. of the value of variable $HOME) is an example (link is to the Bash manual).
However, PowerShell makes it easy:
to embed the results of arbitrary expressions and even whole statements inside expandable (double-quoted) string ("..."), using $(...), the subexpression operator.
to pass the results of any expression or command (pipeline) as an argument to a command, by enclosing it in (...), the grouping operator.
The following command variations are equivalent, and dynamically use the platform-appropriate path (directory) separator, i.e. / on Unix-like platforms, and \ on Windows:
# -> '/etc/passwd' on Unix
# -> '\etc\passwd' on Windows
Write-Output "$([System.IO.Path]::DirectorySeparatorChar)etc$([System.IO.Path]::DirectorySeparatorChar)passwd"
# Ditto.
Write-Output ('{0}etc{0}passwd' -f [System.IO.Path]::DirectorySeparatorChar)
See also:
[System.IO.Path]::DirectorySeparatorChar
-f, the string format operator
Related
In GNU/Linux I would do:
PROGPATH=/long/and/complicated/path/to/some/bin
$PROGPATH/program args...
but in Powershell if I try this:
$PROGPATH=\long\and\complicated\path\to\some\bin
$PROGPATH\program args...
I get:
At script.ps1:2 char:...
+ $PROGPATH\program args ...
+ ~~~~~~~~
Unexpected token '\program' in expression or statement.
+ CategoryInfo : ParserError: (:) [], ParseException
+ FullyQualifiedErrorId : UnexpectedToken
So how do I do this simple thing I know how to do in bash, in Powershell?
js2010's helpful answer shows the correct solution:
Because your command name/path contains a variable reference ($PROGPATH/...), you must invoke it with &.
The same applies if a grouping expression, (...) is used, or a subexpression, $(...) is involved.
Additionally, the same applies if a command name/path is quoted ('...' or "...")[1], as is required if the path contains spaces, for instance.
To put it differently: Direct invocation is only supported if the command name/path is a verbatim, unquoted string[1]; in all other cases, & must be used.
As for why:
&, the call operator is necessary to force interpretation of a statement as a command, i.e. to have it parsed in argument mode (see below), so as to result in command execution rather than expression evaluation.
PowerShell has two fundamental parsing modes:
argument mode, which works like a traditional shell, where the first token is a command name/path, such as a cmdlet or an external program, with subsequent tokens representing the arguments, which only require quoting if they contain shell metacharacters (chars. with special meaning to PowerShell, such as spaces to separate tokens).
expression mode, which works like expressions in programming languages.
PowerShell decides based on a statement's first token what parsing mode to apply:
If, among other things, the first token starts with a variable reference or is a quoted string, PowerShell parses in expression mode.
In expression mode, \ starts a new token, and unrecognized token \program results in the syntax error you saw.
(If you had used /, it would have been interpreted as the division operator, and program wouldn't be a valid divisor operand.)
[1] Note that if your executable path is a literal string (doesn't contain variable references of expressions) you may alternatively `-escape individual characters (spaces) in lieu of enclosing entire string in '...' or "...", in which case & is then not necessary; e.g.:
C:\Program` Files\Notepad++\notepad++.exe
With a literal string you can even employ partial single- or double-quoting as long as the first token is unquoted; e.g.:
C:\"Program Files"\Notepad++\notepad++.exe
Use the call operator "&". https://ss64.com/ps/call.html
Related: Executing a command stored in a variable from PowerShell
$progpath = 'c:\windows\system32'
& $progpath\notepad somefile.txt
Something with a space:
& 'C:\Program Files\internet explorer\iexplore' yahoo.com
Other options, adding to the path:
$env:path += ';C:\Program Files\internet explorer'
iexplore yahoo.com
And backquoting the spaces:
C:\Program` Files\internet` explorer\iexplore yahoo.com
In the first part of my question I will provide some background info as a
service to the community. The second part contains the actual question.
Part I
Assume I've created the following alias:
alias ls='ls -r'
I know how to temporarily unalias (i.e., override this alias) in the following
ways, using:
1) the full pathname of the command: /bin/ls
2) command substitution: $(which ls)
3) the command builtin: command ls
4) double quotation marks: "ls"
5) single quotation marks: 'ls'
6) a backslash character: \ls
Case 1 is obvious and case 2 is simply a variation. The command builtin in case 3 was designed to ignore shell functions, but apparently it also works for circumventing aliases. Finally, cases 4 and 5 are consistent with both the POSIX standard (2.3.1):
"a resulting word that is identified
to be the command name word of a
simple command shall be examined to
determine whether it is an unquoted,
valid alias name."
and the Bash Reference Manual (6.6):
"The first word of each simple
command, if unquoted, is checked to
see if it has an alias."
Part II
Here's the question: why is case 6 (overriding the alias by saying \ls)
considered quoting the word? In keeping with the style of this question, I am looking for references to the "official" documentation.
The documentation says that a backslash only escapes the following
character, as opposed to single and double quotation marks, which quote a
sequence of characters. POSIX standard (2.2.1):
"A backslash that is not quoted shall
preserve the literal value of the
following character, with the
exception of a < newline >"
Bash Reference Manual (3.1.2.1):
"A non-quoted backslash ‘\’ is the
Bash escape character. It preserves
the literal value of the next
character that follows, with the
exception of newline."
(BTW, isn't "the next character that follows" a bit of overkill?)
A possible answer might be that this situation isn't that special: it is
similar to a few cases in ANSI-C quoting, e.g. \nnn. However, that is still
escaping a single character (the eight-bit character whose value is the octal
value nnn), not a sequence of characters.
Historically, and maintained by POSIX, quoting any part of the word causes the entire word to be considered quoted for the purposes of functions and alias expansion. It also applies to quoting the end token for a here document:
cat << \EOF
this $text is fully quoted
EOF
Just for completion, here's yet another way to suppress alias & function lookups (by clearing the entire shell environment for a single command):
# cf. http://bashcurescancer.com/temporarily-clearing-environment-variables.html
env -i ls
\ls only quotes the first character rather than the whole word. It's equivalent to writing 'l's.
You can verify it like this:
$ touch \?l
$ \??
bash: ?l: command not found
If \?? quoted the whole word it would say ?? not found rather than ?l not found.
I.e. it has the same effect as:
$ '?'?
bash: ?l: command not found
rather than:
$ '??'
bash: ??: command not found
A Perl system call must send the following string to the UnixShell:
'"XYZ"'
In my Perl script I have used the following command:
system("cleartool mkattr -replace ATTRIBUTE '"$attribute"' lbtype:$label");
Everything is well passed to the Shell Unix, except both uses of the quote character:
'
Indeed,
cleartool mkattr -replace ATTRIBUTE
The above command is passed as it is exactly what I want.
The Perl variables $attribute and $label are well interpreted.
But I don't know what to do to obtain exactly:
'"XYZ"'
Here XYZ is the value of the Perl variable $attribute
OS is AIX (Unix) and Shell is ksh. cleartool is the command line interface of Clearcase but no Clearcase skill is necessary to fix my problem.
If you want to execute a system command and don't have to use any shell syntax like redirects, it's usually better and safer to use the list form of system:
system(
'cleartool', 'mkattr', '-replace', 'ATTRIBUTE',
qq{"$attribute"}, qq{lbtype:$label}
);
# or, if you really want to pass both types of quotes:
system(
'cleartool', 'mkattr', '-replace', 'ATTRIBUTE',
qq{'"$attribute"'}, qq{lbtype:$label}
);
See perldoc -f system
It's not clear from your question if you want to pass '"XYZ"' or "XYZ".
See "Quote and Quote like Operators" and use qq{...}:
system(qq{cleartool mkattr -replace ATTRIBUTE '"$attribute"' lbtype:$label});
qq{...} is exactly like "..." except you can then use double quotes " in your string without escaping them.
You can use any character directly after the qq and must then use the same character to denote the end-of-string, i.e. qqX...X would work the same way. You would run into problems if your string contains Xes, so don't do that.
You can also use paired characters as delimiter ({}, (), <>) which is what you usually will see.
I am trying to execute script with commands:
sed -i "USER/c\export USER=${signumid}" .bashrc
sed -i "DEVENVHOME=$/c\export DEVENVHOME=${DEVENVHOME:-/home/${signumid}/CPM_WORKAREA/devenv.x}" .bashrc
I want to replace the line with string "USER" in .bashrc with export USER=${signumid} where $signumid variable is being provided through Cygwin prompt. Similarly I want to replace line with string DEVENVHOME=$ with export DEVENVHOME=${DEVENVHOME:-/home/${signumid}/CPM_WORKAREA/devenv.x} in .bashrc file, where $signumid variable is provided through Cygwin prompt.
But I am getting following errors on Cygwin termminal.:
sed: -e expression #1, char 1: unknown command: `U'
sed: -e expression #1, char 3: extra characters after command
The general syntax of a sed script is a sequence of address command arguments statements (separated by newline or semicolon). The most common command is the s substitution command, with an empty address, so we can perhaps assume that that is what you want to use here. You seem to be attempting to interpolate a shell variable $signumid which adds a bit of a complication to this exposition.
If your strings were simply static text, it would make sense to use single quotes; then, the shell does not change the text within the quotes at all. The general syntax of the s command is s/regex/replacement/ where the slash as the argument separator is just a placeholder, as we shall soon see.
sed -i 's/.*USER.*/export USER=you/
s% DEVENVHOME=\$%export DEVENVHOME=${DEVENVHOME:-/home/you/CPM_WORKAREA/devenv.x}%' .bashrc
This will find any line with USER and substitute the entire line with export USER=you; then find any line which contains DEVENVHOME=$ (with a space before, and a literal dollar character) and replace the matched expression with the long string. Because the substitution string uses slashes internally, we use a different regex separator % -- alternatively, we could backslash-escape the slashes which are not separators, but as we shall see, that quickly becomes untenable when we add the following twist. Because the dollar sign has significance as the "end of line" metacharacter in regular expressions, we backslash-escape it.
I have ignored the c\ in your attempt on the assumption that it is simply a misunderstanding of sed syntax. If it is significant, what do you hope to accomplish with it? c\export is not a valid Bash command, so you probably mean something else, but I cannot guess what.
Now, to interpolate the value of the shell variable signumid into the replacement, we cannot use single quotes, because those inhibit interpolation. You have correctly attempted to use double quotes instead (in your edited question), but that means we have to make some additional changes. Inside double quotes, backslashes are processed by the shell, so we need to double all backslashes, or find alternative constructs. Fortunately for us, the only backslash is in \$ which can equivalently be expressed as [$], so let's switch to that notation instead. Also, where a literal dollar sign is wanted in the replacement string, we backslash-escape it in order to prevent the shell from processing it.
sed -i "s/.*USER.*/export USER=$signumid/
s% DEVENVHOME=[$]%export DEVENVHOME=\${DEVENVHOME:-/home/$signumid/CPM_WORKAREA/devenv.x}%" .bashrc
Equivalenty, you could use single quotes around the parts of the script which are meant to be untouched by the shell, and then put an adjacent double-quoted string around the parts which need interpolation, like
'un$touched*by$(the!shell)'"$signumid"'more$[complex]!stuff'
This final script still rests on a number of lucky or perhaps rather unlucky guesses about what you actually want. On the first line, I have changed just USER to a regular expression which matches the entire line -- maybe that's not what you want? On the other hand, the second line makes the opposite assumption, just so you can see the variations -- it only replaces the actual text we matched. Probably one or the other needs to be changed.
Finally, notice how the two separate sed commands have been conflated into a single script. Many newcomers do not realize that sed is a scripting language which accepts an arbitrary number of commands in a script, and simply treat it as a "replace" program with a funny syntax.
Another common source of confusion is the evaluation order. The shell processes the double-quoted string even before sed starts to execute, so if you have mistakes in the quoting, you can easily produce syntax errors in the sed script which lead to rather uninformative error messages (because what sed tells you in the error message is based on what the script looks like after the shell's substutions). For example, if signumid contains slashes, it will produce syntax errors, because sed will see those as terminating separators for the s/// command. An easy workaround is to switch to a separator which does not occur in the value of signumid.
I have program whose textual output I want to directly execute in a shell. How shall I format the output of this program such that the paths with spaces are accepted by the shell ?
$(echo ls /folderA/folder\ with\ spaces/)
Some more info: the program that generates the output is coded in Haskell (source). It's a simple program that keeps a list of my favorite commands. It prints the commands with 'cmdl -l'. I can then choose one command to execute with 'cmdl -g12' for command number 12. Thanks for pointing out that instead of $( ) use 'cmdl -g12 | bash', I wasn't aware of that...
How shall I format the output of this program such that the paths with
spaces are accepted by the shell ?
The shell cannot distinguish between spaces that are part of a path and spaces that are separator between arguments, unless those are properly quoted. Moreover, you actually need proper quoting using single quotes ('...') in order to "shield" all those characters combinations that might otherwise have special meaning for the shell (\, &, |, ||, ...).
Depending the language used for your external tool, their might be a library available for that purpose. As as example, Python has pipes.quote (shlex.quote on Python 3) and Perl has String::ShellQuote::shell_quote.
I'm not quite sure I understand, but don't you just want to pipe through the shell?
For a program called foo
$ foo | sh
To format output from your program so Bash won't try to space-separate them into arguments either update, probably easiest just to double-quote them with any normal quoting method around each argument, e.g.
mkdir "/tmp/Joey \"The Lips\" Fagan"
As you saw, you can backslash the spaces alternatively, but I find that less readable ususally.
EDIT:
If you may have special shell characters (&|``()[]$ etc), you'll have to do it the hard/proper way (with a specific escaper for your language and target - as others have mentioned.
It's not just spaces you need to worry about, but other characters such as [ and ] (glob a.k.a pathname-expansion characters) and metacharacters such as ;, &, (, ...
You can use the following approach:
Enclose the string in single quotes.
Replace existing single quotes in the string with '\'' (which effectively breaks the string into multiple parts with spliced in \-escaped single quotes; the shell then reassembles the parts into a single string).
Example:
I'm good (& well[1];) would encode to 'I'\''m good (& well[1]);'
Note how single-quoting allows literal use of the glob characters and metacharacters.
Since single quotes themselves can never be used within single-quoted strings (there's not even an escape), the splicing-in approach described above is needed.
As described by #mklement0, a safe algorithm is to wrap every argument in a pair of single quotes, and quote single quotes inside arguments as '\''. Here is a shell function that does it:
function quote {
typeset cmd="" escaped
for arg; do
escaped=${arg//\'/\'\\\'\'}
cmd="$cmd '$escaped'"
done
printf %s "$cmd"
}
$ quote foo "bar baz" "don't do it"
'foo' 'bar baz' 'don'\''t do it'