I was playing around with some bash features and as I tried echo-ing some output, I noticed, that
echo what about in some more complex example ?
results in
what about in some more complex example \
I know that escaping the question mark or the whole line with quotes resolves the problem, but I am curious of why is it happening.
So my 2 questions are:
What is the meaning of a simple question mark in bash (I know for example about the '$?' special parameter and regex usage) ?
I suppose that is is a bash environment variable or some king of variable. How can I inspect a variable ? For example a command can be inspected with type keyword, i.e.
type cd
In that context it functions as a glob pattern. If there are files with one-character names in the current working directory, the shell expands an unquoted question mark to their names.
$ echo ? \? '?' "?"
? ? ? ?
$ touch a b c
$ echo ? \? '?' "?"
a b c ? ? ?
Similarly, ?? is expanded to two-character filenames, ??* to filenames longer than one character, and ??[ab] to three-character filenames ending with an a or a b, etc.
See Filename Expansion for further information.
Related
Essentially what I am trying to do is take a string with a bunch of text and if it has a substring of "$$" to replace it with a substring of "$$$"
ex:
string="abcde\$\$fghi"
# Modify string
echo $string
# ^ should give "abcde$$$fghi"
I have been at this for like 2 hours now and it seems like a very simple thing, so if anyone could provide some help then I would greatly appreciate it. Thanks!
EDIT: Changed original string in the question from "abcde$$fghi" to "abcde\$\$fghi"
$$ is a special variable in the shell, it contains the ID of the current process. The variables are expanded in double quotes, therefore string does not contain $$ but a number (the PID of shell) instead.
Enclose the string in apostrophes (single quotes) to get $$ inside it.
The replacement you need can be done in multiple ways. The simplest way (probably) and also the fastest way (for sure) is to use / in the parameter expansion of $string:
echo "${string/'$$'/'$$$'}"
To make it work you have to use the same trick as before: wrap $$ and $$$ in single quotes to prevent the shell replace them with something else. The quotes around the entire expression are needed to preserve the space characters contained by $string, otherwise the line is split to words by whitspaces and and echo outputs these words separated by one space character.
Check it online.
If you quote the string with single quote marks (i.e. string='abcde$$fghi') you can do the replacement with echo "${string/'$$'/'$$$'}"
Edit: this is basically what #axiac said in their comment
To rephrase - I want to use Bash command substitution and string substitution in the same line.
My actual commands are longer, but the ridiculous use of echo here is just a "substitution" for shortness and acts the same - with same errors ;)
I know we can use a Bash command to produce it's output string as a parameter for another command like this:
echo "$(echo "aahahah</ddd>")"
aahahah</ddd>
I also know we can remove last known part of a string like this:
var="aahahah</ddd>"; echo "${var%</ddd>}"
aahahah
I am trying to write a command where one command gives a string output, where I want to remove last part, which is known.
echo "${$(echo "aahahah</ddd>")%</ddd>}"
-bash: ${$(echo "aahahah</ddd>")%</ddd>}: bad substitution
It might be the order of things happening or substitution only works on variables or hardcoded strings. But I suspect just me missing something and it is possible.
How do I make it work?
Why doesn't it work?
When a dollar sign as in $word or equivalently ${word} is used, it asks for word's content. This is called parameter expansion, as per man bash.
You may write var="aahahah</ddd>"; echo "${var%</ddd>}": That expands var and performs a special suffix operation before returning the value.
However, you may not write echo "${$(echo "aahahah</ddd>")%</ddd>}" because there is nothing to expand once $(echo "aahahah</ddd>") is evaluated.
From man bash (my emphasis):
${parameter%word}
Remove matching suffix pattern. The word is expanded to produce a
pattern just as in pathname expansion. If the pattern
matches a trailing portion of the expanded value of parameter, then
the result of the expansion is the expanded value of parameter
with the shortest matching pattern (the ''%'' case) or the longest matching pattern (the ''%%'' case) deleted.
Combine your commands like this
var=$(echo "aahahah</ddd>")
echo ${var/'</ddd>'}
I'm having to code a subversion hook script, and I found a few examples online, mostly python and perl. I found one or two shell scripts (bash) as well. I am confused by a line and am sorry this is so basic a question.
FILTER=".(sh|SH|exe|EXE|bat|BAT)$"
The script later uses this to perform a test, such as (assume EXT=ex):
if [[ "$FILTER" == *"$EXT"* ]]; then blah
My problem is the above test is true. However, I'm not asking you to assist in writing the script, just explaining the initial assignment of FILTER. I don't understand that line.
Editing in a closer example FILTER line. Of course the script, as written does not work, because 'ex' returns true, and not just 'exe'. My problem here is only, however, that I don't understant the layout of the variable assignment itself.
Why is there a period at the beginning? ".(sh..."
Why is there a dollar sign at the end? "...BAT)$"
Why are there pipes between each pattern? "sh|SH|exe"
You probably looking for something as next:
FILTER="\.(sh|SH|exe|EXE|bat|BAT)$"
for EXT
do
if [[ "$EXT" =~ $FILTER ]];
then
echo $EXT extension disallowed
else
echo $EXT is allowed
fi
done
save it to myscript.sh and run it as
myscript.sh bash ba.sh
and will get
bash is allowed
ba.sh extension disallowed
If you don't escape the "dot", e.g. with the FILTER=".(sh|SH|exe|EXE|bat|BAT)$" you will get
bash extension disallowed
ba.sh extension disallowed
What is (of course) wrong.
For the questions:
Why is there a period at the beginning? ".(sh..."
Because you want match .sh (as extension) and not for example bash (without the dot). And therefore the . must be escaped, like \. because the . in regex mean "any character.
Why is there a dollar sign at the end? "...BAT)$"
The $ mean = end of string. You want match file.sh and not file.sh.jpg. The .sh should be at the end of string.
Why are there pipes between each pattern? "sh|SH|exe"
In the rexex, the (...|...|...) construction delimites the "alternatives". As you sure quessed.
You really need read some "regex tutorial" - it is more complicated - and can't be explained in one answer.
Ps: NEVER use UPPERCASE variable names, they can collide with environment variables.
This just assigns a string to FILTER; the contents of that string have no special meaning. When you try to match it against the pattern *ex*, the result is true assuming that the value of $FILTER consists the string ex surrounded by anything on either side. This is true; ex is a substring of exe.
FILTER=".(sh|SH|exe|EXE|bat|BAT)$"
^^
|
+---- here is the "ex" from the pattern.
As I can this is similar to regular expression pattern:
In regular expressions the string start with can be show with ^, similarly in this case . represent seems doing that.
In the bracket you have exact string, which represents what the exact file extensions would be matched, they are 'Or' by using the '|'.
And at the end the expression should only pick the string will '$' or end point and not more than.
I would say that way original author might have looked at it and implemented it.
I have program whose textual output I want to directly execute in a shell. How shall I format the output of this program such that the paths with spaces are accepted by the shell ?
$(echo ls /folderA/folder\ with\ spaces/)
Some more info: the program that generates the output is coded in Haskell (source). It's a simple program that keeps a list of my favorite commands. It prints the commands with 'cmdl -l'. I can then choose one command to execute with 'cmdl -g12' for command number 12. Thanks for pointing out that instead of $( ) use 'cmdl -g12 | bash', I wasn't aware of that...
How shall I format the output of this program such that the paths with
spaces are accepted by the shell ?
The shell cannot distinguish between spaces that are part of a path and spaces that are separator between arguments, unless those are properly quoted. Moreover, you actually need proper quoting using single quotes ('...') in order to "shield" all those characters combinations that might otherwise have special meaning for the shell (\, &, |, ||, ...).
Depending the language used for your external tool, their might be a library available for that purpose. As as example, Python has pipes.quote (shlex.quote on Python 3) and Perl has String::ShellQuote::shell_quote.
I'm not quite sure I understand, but don't you just want to pipe through the shell?
For a program called foo
$ foo | sh
To format output from your program so Bash won't try to space-separate them into arguments either update, probably easiest just to double-quote them with any normal quoting method around each argument, e.g.
mkdir "/tmp/Joey \"The Lips\" Fagan"
As you saw, you can backslash the spaces alternatively, but I find that less readable ususally.
EDIT:
If you may have special shell characters (&|``()[]$ etc), you'll have to do it the hard/proper way (with a specific escaper for your language and target - as others have mentioned.
It's not just spaces you need to worry about, but other characters such as [ and ] (glob a.k.a pathname-expansion characters) and metacharacters such as ;, &, (, ...
You can use the following approach:
Enclose the string in single quotes.
Replace existing single quotes in the string with '\'' (which effectively breaks the string into multiple parts with spliced in \-escaped single quotes; the shell then reassembles the parts into a single string).
Example:
I'm good (& well[1];) would encode to 'I'\''m good (& well[1]);'
Note how single-quoting allows literal use of the glob characters and metacharacters.
Since single quotes themselves can never be used within single-quoted strings (there's not even an escape), the splicing-in approach described above is needed.
As described by #mklement0, a safe algorithm is to wrap every argument in a pair of single quotes, and quote single quotes inside arguments as '\''. Here is a shell function that does it:
function quote {
typeset cmd="" escaped
for arg; do
escaped=${arg//\'/\'\\\'\'}
cmd="$cmd '$escaped'"
done
printf %s "$cmd"
}
$ quote foo "bar baz" "don't do it"
'foo' 'bar baz' 'don'\''t do it'
Why does this line output "z" instead of "?"
$ ruby -e 'puts %x[ echo #{"?"} ]'
Suppose the expression inside the #{...} is a variable that may have the value of "?". How should I modify this script so that the question mark is outputted instead of "z"?
(Please forgive the title of this question -- I don't yet understand what is going on here well enough to provide a more descriptive title.)
It's not ruby, it's your shell.
Many shells expand the ? character to match a single character in command line arguments.
It's useful, if you have a bunch of files name tempA,temp1,tempB,...,temp9 that you want to delete, but you don't want to delete 'temple'
% rm temp?
So I'm guessing you have a file or directory in your working directory named 'z', and the ? matches that, so it gets replaced by the shell.
Normally, when inside single quotes (like your ruby script) it wouldn't get expanded, but since you're passing the question mark to a shell command, it gets expanded there.
% ruby -e 'puts %x[ echo ? ]'
z
%
Should show you the same behaviour.
Also, if you touch a couple other single character filenames like a b c d, those should show up too:
% touch a b c
% ruby -e 'puts %x[ echo ? ]'
a b c z
%
If you want to avoid this when calling exterior shell commands from within ruby, you'll have to escape any strings you pass out. For most purposes String#inspect should give a good enough escaping.
It doesn't?
irb(main):001:0> puts %x[echo #{"?"}]
?
=> nil
Using #{} will give you the value of any variables inside - I'm not sure why you're using it instead of
puts %x[echo "?"]
or just
puts '?'