Maybe this is a dumb question, but I'm writing a very small Gedit external tool for compiling .qrc (Qt resource files) on a key press. This is my code:
echo $GEDIT_CURRENT_DOCUMENT_PATH
pyrcc $GEDIT_CURRENT_DOCUMENT_PATH -o ${GEDIT_CURRENT_DOCUMENT_PATH/.qrc/.py}
Running the above, I get
/home/user/.local/lib/python3/qrecartivi/resources.qrc
/home/user/.config/gedit/tools/new-tool: 12: /home/user/.config/gedit/tools/new-tool: Bad substitution
where line 12 actually is the 2n line of my sh code (This offset is due to gedit adding some comments in).
I just cannot figure out why this substition should be wrong. Thanks in advance.
As stated in comment by #chepner, the syntax ${var/str1/str2} is not supported by your shell interpreter (Dash) which is strictly POSIX-compliant.
You have two solutions:
Use Bash which supports this kind of variable substitution
Use a POSIX-compliant way to substitute an extension with another. Like this for example:
pyrcc "$GEDIT_CURRENT_DOCUMENT_PATH" -o "$(basename "$GEDIT_CURRENT_DOCUMENT_PATH" .qrc).py"
Related
I've got a python script that wraps a bash command line tool, that gets it's variables from external source (environment variables). is there any way to perform some soft of escaping to prevent malicious user from executing bad code in one of those parameters.
for example if the script looks like this
/bin/sh
/usr/bin/tool ${VAR1} ${VAR2}
and someone set VAR2 as follows
export VAR2=123 && \rm -rf /
so it may not treat VAR2 as pure input, and perform the rm command.
Is there any way to make the variable non-executable and take the string as-is to the command line tool as input ?
The correct and safe way to pass the values of variables VAR1 and VAR2 as arguments to /usr/bin/tool is:
/usr/bin/tool -- "$VAR1" "$VAR2"
The quotes prevent any special treatment of separator or pattern matching characters in the strings.
The -- should prevent the variable values being treated as options if they begin with - characters. You might have to do something else if tool is badly written and doesn't accept -- to terminate command line options.
See Quotes - Greg's Wiki for excellent information about quoting in shell programming.
Shellcheck can detect many cases where quotes are missing. It's available as either an online tool or an installable program. Always use it if you want to eliminate many common bugs from your shell code.
The curly braces in the line of code in the question are completely redundant, as they usually are. Some people mistakenly think that they act as quotes. To understand their use, see When do we need curly braces around shell variables?.
I'm guessing that the /bin/sh in the question was intended to be a #! /bin/sh shebang. Since the question was tagged bash, note that #! /bin/sh should not be used with code that includes Bashisms. /bin/sh may not be Bash, and even if it is Bash it behaves differently when invoked as /bin/sh rather than /bin/bash.
Note that even if you forget the quotes the line of code in the question will not cause commands (like rm -rf /) embedded in the variable values to be run at that point. The danger is that badly-written code that uses the variables will create and run commands that include the variable values in unsafe ways. See should I avoid bash -c, sh -c, and other shells' equivalents in my shell scripts? for an explanation of (only) some of the dangers.
To avoid injections at best, consider switching to [T]csh.
Unlike Bourne Shells, the C Shell is "limited", thus instructing one to take different, safer paths to write scripts. The "limitations" imposed by the C Shell make it one of the most reliable Shells to work with.
(E.g: Nesting is minimal to impossible, thus preventing injections at all costs; there are better ways to achieve what one want.)
CMake's execute_process command seems to only let you, well, execute a process - not an arbitrary line you could feed a command shell. The thing is, I want to use pipes, file descriptor redirection, etc. - and that does not seem to be possible. The alternative would be very painful for me (I think)...
What should I do?
PS - CMake 2.8 and 3.x answer(s) are interesting.
You can execute any shell script, using your shell's support for taking in a script within a string argument.
Example:
execute_process(
COMMAND bash "-c" "echo -n hello | sed 's/hello/world/;'"
OUTPUT_VARIABLE FOO
)
will result in FOO containing world.
Of course, you would need to escape quotes and backslashes with care. Also remember that running bash would only work on platforms which have bash - i.e. it won't work on Windows.
execute_process command seems to only let you, well, execute a process - not an arbitrary line you could feed a command shell.
Yes, exactly this is written in documentation for that command:
All arguments are passed VERBATIM to the child process. No intermediate shell is used, so shell operators such as > are treated as normal arguments.
I want to use pipes
Different COMMAND within same execute_process invocation are actually piped:
Runs the given sequence of one or more commands with the standard output of each process piped to the standard input of the next.
file descriptor redirection, etc. - and that does not seem to be possible.
For complex things just prepare separate shell script and run it using execute_process. You can pass variables from CMake to this script using its parameters, or with prelimiary configure_file.
I needed to pipe two commands one after the other and actually learned that each COMMAND of the execute_process is piped already. So at least that much is resolved by simply adding commands one after the other:
execute_process(
COMMAND echo "Hello"
COMMAND sed -e 's/H/h/'
OUTPUT_VARIABLE GREETINGS
OUTPUT_STRIP_TRAILING_WHITESPACE)
Now the variable GREETINGS is set to hello.
If you indeed need a lot of file redirection (as you stated), you probably want to write an external script and then execute that script from CMakeLists.txt. It's really difficult to get all the escaping right in CMake.
If you can simplify your scripts to one command generating a file, then another handling that file, etc. then you can always use the INPUT_FILE and OUTPUT_FILE options. Or pass a filename to your command for the input.
It's often much cleaner to handle one file at a time. Although I understand that some commands may need multiple sources and destinations.
I was writing a script when I decided to move the functions to a lib file, but when I open the lib file all the $( and the consecutive ) are red highlighted, here are some examples of the script
TAB="$(printf '\t')"
percent=$(echo "scale=2; $number/$total*100" | bc | sed -e 's/\.[[:digit:]]*//g')
if [[ -z $(grep $site/post $max_lim) ]];then
The filetype is conf but I've set it as sh syntax in .vimrc
Any idea of what is happenning?
Thank you
Edit: Thanks for the quick answers, I found that this line makes vim match the files with the extension specified behind the * with the syntax sh
au BufReadPost * set syntax=sh
I've also thought that using shebang in the libraries was not allowed, but is a nice solution
Anyway using g:is_bash in .vimrc returns an error of pattern not found
So what I would like to do is as I only write in bash, to vim recognize any file without extension as bash
The syntax file for sh actually handles several different kinds of shell syntax: bash, ksh, and plain old sh. Since your conf file isn't recognized as bash or ksh, it falls back to sh. $(...) isn't a valid construct in sh, so it is highlighted as an error.
To fix this, you can make sure "g:is_bash" is set for the file, so that the sh syntax script will know your file should be highlighted as bash code. Please edit your question to include what you added to your .vimrc to make the file use sh syntax highlighting. This will make it easier to suggest the correct way of setting "g:is_bash".
UPDATE: As Alok commented, you should be able to add the following to the file
#!/bin/bash
to let vim know the correct syntax highlighting to use as well.
In my case, I wanted to preserve #!/bin/sh as the shebang line because not every system has /bin/bash available.
Whereas the original Bourne shell may have not supported the $(...) syntax, most sh shells nowadays are POSIX-compliant, and the POSIX spec supports this syntax. For example,
On Ubuntu, /bin/sh is /bin/dash.
On MacOS, /bin/sh is /bin/bash.
On Alpine, /bin/sh is /bin/ash.
All of which satisfy the POSIX spec. Traditionally, if we'd like to write portable Shell, we should leave the shebang line as #!/bin/sh. We shouldn't change it to #!/bin/bash just for syntax highlighting if we're not going to use any Bashisms.
Okay, but what about the erroneous red highlighting? The problem is with Vim interpreting #!/bin/sh as a reference to the original Bourne shell from 1979 with no support for $(...). Maybe this is a testament to Vim's backwards compatibility, or maybe not enough people care. Here's a related GitHub issue describing the same behavior.
In any case, the best solution for me was to set let g:is_posix = 1 in my config. Interestingly, if you look through Vim's runtime files, it's equivalent to setting let g:is_kornshell = 1.
A brief interesting history on how the Bourne shell was bourne, bourne again as bash as a substitute for /bin/sh on Ubuntu, and eventually replaced in favor of dash can be found at https://askubuntu.com/a/976504.
I need to extract part of the string before the last opt in the string:
NDS=/opt/novell/opt/eDirectory/opt/abc
I want just the /opt/novell/opt/eDirectory/ part
I used NDSHOME=${NDS%opt*}
but I am getting bad substitution on Solaris 10, that is using the Bourne shell,
although this script works fine on Linux and AIX.
Can somebody find a solution to this??
Solaris' /bin/sh is notoriously not Posix compatible.
You could try the following workaround with sed
NDSHOME=`echo "$NDS" | sed 's/^\(.*\)\<opt\>.*$/\1/'`
Note: Normally I would use $() instead of backtics for command substitution, but I don't think those work on Solaris either.
Edit
Changed it so that opt will not match intra-word boundaries
$ echo $NDS
/opt/novell/opt/eDirectory/opt/helicopter
$ echo "$NDS" | sed 's/^\(.*\)\<opt\>.*$/\1/'
/opt/novell/opt/eDirectory/
The dirname command prints all but the last level of the path name given as an argument.
$ NDS=/opt/novell/opt/eDirectory/opt/abc
$ NDSHOME=`dirname $NDS`
$ echo $NDSHOME
/opt/novell/opt/eDirectory/opt
(Add export as needed.)
EDIT :
The above doesn't work in this case. The OP wants to remove all components of the path following the last opt, not just the last component. I'll leave it here in case it's useful to someone else.
You could use dirname in a loop, stopping when the last component (as determined by basename) is `opt':
NDSHOME="$NDS"
while [ "`basename $NDSHOME`" != opt ] ; do
NDSHOME="`dirname $NDSHOME`"
done
but that's more complex than it needs to be; sed is a better solution.
SiegeX's answer is good, but it treats opt as a string, not just as a path component. For example, it will turn
/opt/novell/opt/eDirectory/opt/helicopter
into
/opt/novell/opt/eDirectory/opt/helic
Try this:
NDSHOME=`echo "$NDS" | sed 's,\(.*\)/opt/.*$,\1,'`
(Note the use of , rather than / as a delimiter, so the / characters in the pattern don't have to be escaped.)
Solaris is definitely POSIX (and several related standards like xpg4 and susv3) compliant but it also takes compatibility with older versions quite seriously.
The side effect is that when both these features conflict in some way, the affected commands are, by default, not POSIX compliant not to break legacy scripts.
Should you want POSIX compatibility and have no legacy script issues, just set your PATH like this:
PATH=/usr/xpg6/bin:/usr/xpg4/bin:/usr/ccs/bin:$PATH
and set your login shell to /usr/xpg4/bin/sh
Alternatively, if you do not need strict compliance, you might just use /bin/ksh as your default shell.
With Solaris 10 and older, you shouldn't really use /bin/sh at all in new scripts.
For reference, here is the Solaris 10 standards manual page and the one for Solaris 11 is here.
Use the Bourne-Again shell instead.
I use ZSH for my terminal shell, and whilst I've written several functions to automate specific tasks, I've never really attempted anything that requires the functionality I'm after at the moment.
I've recently re-written a blog using Jekyll and I want to automate the production of blog posts and finally the uploading of the newly produced files to my server using something like scp.
I'm slightly confused about the variable bindings/usage in ZSH; for example:
DATE= date +'20%y-%m-%d'
echo $DATE
correctly outputs 2011-08-23 as I'd expect.
But when I try:
DATE= date +'20%y-%m-%d'
FILE= "~/path/to/_posts/$DATE-$1.markdown"
echo $FILE
It outputs:
2011-08-23
blog.sh: line 4: ~/path/to/_posts/-.markdown: No such file or directory
And when run with what I'd be wanting the blog title to be (ignoring the fact the string needs to be manipulated to make it more url friendly and that the route path/to doesn't exist)
i.e. blog "blog title", outputs:
2011-08-23
blog.sh: line 4: ~/path/to/_posts/-blog title.markdown: No such file or directory
Why is $DATE printing above the call to print $FILE rather than the string being included in $FILE?
Two things are going wrong here.
Firstly, your first snippet is not doing what I think you think it is. Try removing the second line, the echo. It still prints the date, right? Because this:
DATE= date +'20%y-%m-%d'
Is not a variable assignment - it's an invocation of date with an auxiliary environment variable (the general syntax is VAR_NAME=VAR_VALUE COMMAND). You mean this:
DATE=$(date +'20%y-%m-%d')
Your second snippet will still fail, but differently. Again, you're using the invoke-with-environment syntax instead of assignment. You mean:
# note the lack of a space after the equals sign
FILE="~/path/to/_posts/$DATE-$1.markdown"
I think that should do the trick.
Disclaimer
While I know bash very well, I only started using zsh recently; there may be zshisms at work here that I'm not aware of.
Learn about what a shell calls 'expansion'. There are several kinds, performed in a particular order:
The order of word expansion is as follows:
tilde expansion
parameter expansion
command substitution
arithmetic expansion
pathname expansion, unless set -f is in effect
quote removal, always performed last
Note that tilde expansion is only performed when the tilde is not quoted; viz.:
$ FILE="~/.zshrc"
$ echo $FILE
~/.zshrc
$ FILE=~./zshrc
$ echo $FILE
/home/user42/.zshrc
And there must be no spaces around the = in variable assignments.
Since you asked in a comment where to learn shell programming, there are several options:
Read the shell's manual page man zsh
Read the specification of the POSIX shell, http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html, especially if you want to run your scripts on different operating systems (and you will find yourself in that situation one fine day!)
Read books about shell programming.
Hang out in the usenet newsgroup comp.unix.shell where a lot of shell wizards answer questions