We have an application that keeps some info in an encrypted file. To edit the file we have to put the text editor name in an environment variable in bash, for example, EDITOR=vi. Then we run the application and it opens the decrypted file in vi. I am trying to come up with a bash script that updates the encrypted file. The only solution that I can think of is passing sed command instead of vi to the EDITOR variable. It works perfectly for something like EDITOR='sed -i s#aaaa#bbbb#'.
Problem starts when I need space and regular expression. For example: EDITOR='sed -i -r "s#^(\s*masterkey: )(.*)#\1xxxxx#"' which return error. I tried running the EDITOR in bash with $EDITOR test.txt and I can see the problem. It doesn't like double quotes and space between them so I added a backslash before the double quotes and \s instead of space. Now it says unterminated address regex. For several hours I googled and couldn't find any solution. I tried replacing single quotes with double quotes and vice versa and everything that I could find on the internet but no luck.
How can I escape and which characters should I escape here?
Update:
Maybe if I explain the whole situation somebody could suggest an alternative solution. There is an application written by Ruby and it is inside a container. Ruby application has a secret_key_base for production and we supposed to change the key with EDITOR=vi rails credentials:edit --environment=production. I don't know Ruby and google did not return any ruby solution for automation so I could only think about sending sed instead of vi to Ruby.
How can I escape and which characters should I escape here?
That is not possible. Word splitting on the result of expansion cannot be escaped from inside the result of that expansion, it will always run. Note that filename expansion is also running over the result of the expansion.
Create an executable file with the script content and set EDITOR to it.
You could export a bash shell function, after some tries I got to:
myeditor() {
sed -i -E 's#^(\s*masterkey: )(.*)#\1xxxxx#' "$#"
}
export -f myeditor
EDITOR='bash -c "$#" _ myeditor'
Related
I have a script that outputs a list of env vars like:
THING=one
ANOTHER_THING=two
A_PATH="path/to a/directory"
When I try to export these env vars as export `./script`, the A_PATH env var exports as path/to.
If I export the list as plain text like:
export THING=one ANOTHER_THING=two A_PATH="path/to a/directory", it works just fine.
I'm stumped as to why bash treats the white space in the A_PATH differently in these two cases. I've tried various attempts at escaping the whitespace and I've even tried exporting line by line, but in every case it sees the whitespace as a delimiter rather than as a part of the path string.
why bash treats the white space in the A_PATH differently in these two cases
The result of command substitution `...` undergoes word splitting. Do not use ` backticks - use $(....) instead. Check your scripts with shellcheck.net .
Spaces inside string around double quotes are literally preserved. See quoting.
If the file has proper correct shell syntax, and it's meant to be sourced and support shell-ish execution, see https://unix.stackexchange.com/questions/614568/is-is-possible-to-export-all-variables-obtained-from-sourcing-a-file . If the file contains = separated variable name and optionally quoted string with custom syntax, write a parser for the file for that syntax - see ex. Parsing variables from config file in Bash for a stub to get started.
Without the actual code, it is difficult to understand what is going on. But one workaround that might solve your problem is to output the export command and evaluate it.
Example (script.sh):
#!/bin/bash
echo "export THING=one"
echo "export ANOTHER_THING=two"
echo "export A_PATH='path/to a/directory'"
eval $(./script.sh)
I am trying to write an alias for the following command that takes your current working directory, surrounded by quotes, and copies it to the clipboard.
echo \"$(pwd)\" | xclip
However, writing an alias for this in .bashrc is not working quite right. This is the alias I have written:
alias cpwd="echo \"\$(pwd)\" | xclip"
But when that command is ran the double quotes are omitted. I have checked answers to similar problems, such as bash alias command with both single and double quotes, but I think I am escaping all the required characters, so I do not know why this command isn't working.
The current result of the command would be something like: home/user/My Folder rather than "home/user/My Folder" like I am wanting.
You need more escaping. You've correctly escaped the $; you need to do the same with the existing backslashes.
alias cpwd="echo \\"\$(pwd)\\" | xclip"
Alternatively, you can avoid all the escaping by using single quotes.
alias cpwd='echo \"$(pwd)\" | xclip'
Best of all, use a function instead of an alias. A function lets you write the command exactly as you would normally without any extra quotes or escaping.
cpwd() {
echo \"$(pwd)\" | xclip
}
I'm trying to write a regex string in the terminal but zsh is interpreting this regex instead of just printing it. My shell code:
echo "((https?:\/\/(?:www\.|(?!www)))?[a-zA-Z0-9][a-zA-Z0-9-]+[a-zA-Z0-9]\.[^\s]{2,}|www\.[a-zA-Z0-9][a-zA-Z0-9-]+[a-zA-Z0-9]\.[^\s]{2,}|https?:\/\/(?:www\.|(?!www))[a-zA-Z0-9]+\.[^\s]{2,}|www\.[a-zA-Z0-9]+\.[^\s]{2,})"
Current output:
zsh: event not found: www)))?[a
I already tried to use simple quotes, double quotes and no quotes.
If you type this command in a file and run as a script, it should be fine, unless you have explicitly enabled history expansion in your script. But then, you know what you are doing.
If you really literally hack such a huge command manually into an interactive shell, either turn off history expansion (by setopt nobanghist), or prefix your ! by a \ (unless the ! is already between single-quotes).
Example: Typing echo !www won't work, but typing echo \!www will.
If you never use history expansion, turning it off permanently would probably be the best choice.
I am trying to give an argument to my python program through the terminal.
For this I am using the lines:
import sys
something = sys.argv[1]
I now try to put in a string like this through the bash terminal:
python my_script.py 2m+{N7HiwH3[>!"4y?t9*y#;/$Ar3wF9+k$[3hK/WA=aMzF°L0PaZTM]t*P|I_AKAqIb0O4# cm=sl)WWYwEg10DDv%k/"c{LrS)oVd§4>8bs:;9u$ *W_SGk3CXe7hZMm$nXyhAuHDi-q+ug5+%ioou.,IhC]-_O§V]^,2q:VBVyTTD6'aNw9:oan(s2SzV
This returns a bash error because some of the characters in the string are bash special characters.
How can I use the string exactly as it is?
You can put the raw string into a file, for example like this, with cat and a here document.
cat <<'EOF' > file.txt
2m+{N7HiwH3[>!"4y?t9*y#;/$Ar3wF9+k$[3hK/WA=aMzF°L0PaZTM]t*P|I_AKAqIb0O4# cm=sl)WWYwEg10DDv%k/"c{LrS)oVd§4>8bs:;9u$ *W_SGk3CXe7hZMm$nXyhAuHDi-q+ug5+%ioou.,IhC]-_O§V]^,2q:VBVyTTD6'aNw9:oan(s2SzV
EOF
and then run
python my_script.py "$(< file.txt)"
You can also use the text editor of your choice for the first step if you prefer that.
If this is a reoccurring task, which you have to perform from time to time, you can make your life easier with a little alias in your shell:
alias escape='read -r string ; printf "Copy this:\n%q\n" "${string}"'
It is using printf "%q" to escape your input string.
Run it like this:
escape
2m+{N7HiwH3[>!"4y?t9*y#;/$Ar3wF9+k$[3hK/WA=aMzF°L0PaZTM]t*P|I_AKAqIb0O4# cm=sl)WWYwEg10DDv%k/"c{LrS)oVd§4>8bs:;9u$ *W_SGk3CXe7hZMm$nXyhAuHDi-q+ug5+%ioou.,IhC]-_O§V]^,2q:VBVyTTD6'aNw9:oan(s2SzV
Copy this:
2m+\{N7HiwH3\[\>\!\"4y\?t9\*y#\;/\$Ar3wF9+k\$\[3hK/WA=aMzF°L0PaZTM\]t\*P\|I_AKAqIb0O4#\ cm=sl\)WWYwEg10DDv%k/\"c\{LrS\)oVd§4\>8bs:\;9u\$\ \*W_SGk3CXe7hZMm\$nXyhAuHDi-q+ug5+%ioou.\,IhC\]-_O§V\]\^\,2q:VBVyTTD6\'aNw9:oan\(s2SzV
You can use the escaped string directly in your shell, without additional quotes, like this:
python my_script.py 2m+\{N7HiwH3\[\>\!\"4y\?t9\*y#\;/\$Ar3wF9+k\$\[3hK/WA=aMzF°L0PaZTM\]t\*P\|I_AKAqIb0O4#\ cm=sl\)WWYwEg10DDv%k/\"c\{LrS\)oVd§4\>8bs:\;9u\$\ \*W_SGk3CXe7hZMm\$nXyhAuHDi-q+ug5+%ioou.\,IhC\]-_O§V\]\^\,2q:VBVyTTD6\'aNw9:oan\(s2SzV
In order to make life easier, shells like bash do a little bit of extra work to help users pass the correct arguments to the programs they instruct it to execute. This extra work usually results in predictable argument arrays getting passed to programs.
Oftentimes, though, this extra help results in unexpected arguments getting passed to programs; and sometimes results in the execution of undesired additional commands. In this case, though, it ended up causing Bash to emit an error.
In order to turn off this extra work, Bash allows users to indicate where arguments should begin and end by surrounding them by quotation marks. Bash supports both single quotes (') and double quotes (") to delimit arguments. As a last resort, if a string may contain single and double quotes (or double quotes are required but aren't aggressive enough), Bash allows you to indicate that a special- or whitespace-character should be part of the adjacent argument by preceding it with a backslash (\\).
If this method of escaping arguments is too cumbersome, it may be worth simplifying your program's interface by having it consume this data from a file instead of a command line argument. Another option is to create a program that loads the arguments from a more controlled location (like a file) and directly execs the target program with the desired argument array.
I have a simple script as follows:
#!/bin/ksh
JAVA_SRC="/home/myuser/Setup\ Files/jre/bin/java"
eval $JAVA_SRC
When I run this on HP-UX, I get this error:
$ spacetest.sh
/home/myuser/Setup^BFiles/jre/bin/java: not found
It seems like the shell cannot properly handle the escaped whitespace and is putting garbage instead of the "\ ". This works perfectly fine on other *NIX platforms. If I echo $JAVA_SRC right before the eval, the path shows fine.
Is there any way to properly handle this on HP-UX?
The solution is to simply, as William suggested above, use quotes around the variable name ("$JAVA_SRC").