I'm using Capistrano for deployment and I need to run a command for setting the correct environment in the .htaccess file on the server.
I do that with sed like this: run "sed -i -r 's/APPLICATION_ENV \w+/APPLICATION_ENV #{stage}/' #{current_release}/public/.htaccess"
My problem is that the \w+ is escaped by Ruby/Cap and the regexp ends up trying to match only w+. If I use \\w+ I end up with a regexp that attempts to match \\w+.
How can I have an interpolated double quoted string and successfully escape the \w? Do I really need to change to concatenation of single quoted string and variables?
On my sysem your sample works but you could try it with one of these two (comment the ones out you don't use)
begin
stage = "stage"
current_release = "current_release"
s = "sed -i -r 's/APPLICATION_ENV \\w+/APPLICATION_ENV #{stage}/' #{current_release}/public/.htaccess"
s = "sed -i -r 's/APPLICATION_ENV #{"\\w+"}/APPLICATION_ENV #{stage}/' #{current_release}/public/.htaccess"
s = "sed -i -r 's/APPLICATION_ENV #{92.chr}w+/APPLICATION_ENV #{stage}/' #{current_release}/public/.htaccess"
puts s
system s
rescue
puts $!
system('pause')
end
Related
I have a bash script where i'm trying to call a curl which is having a variable value as input. When trying to execute the bash script the variable value is not getting expanded in double quotes.
Expected curl in script after variable expansion should be as following:
/usr/bin/curl -s -vvvv http://hmvddrsvr:8044/query/service -u iamusr:pssd -d 'statement=DELETE FROM `test_bucket` WHERE type = "Metadata" AND market = "ES" AND status = "active" AND meta(test_bucket).id="fgsd34sff334" '
Getting executed as follows when observed in debug mode:
/usr/bin/curl -s -vvvv http://hmvddrsvr:8044/query/service -u iamusr:pssd -d 'statement=DELETE FROM `test_bucket` WHERE type = "Metadata" AND market = "ES" AND status = "active" AND meta(test_bucket).id=\""$idp_sub"\" '
My bash script is as follows:
#!/bin/bash
idp_sub=""
for idp_sub in $(cat /opt/SP/jboss/home/mayur/es_idp_sub.txt)
do
/usr/bin/curl -s -vvvv http://hmvddrsvr:8044/query/service -u iamusr:pssd -d 'statement=DELETE FROM `test_bucket` WHERE type = "Metadata" AND market = "ES" AND status = "active" AND meta(test_bucket).id=\""$idp_sub"\" ' -o /opt/SP/jboss/home/mayur/es_delete_response.txt
done
How does do i expand the variable value within double quotes as shown above in expected output ?
Your double-quoted string is inside single quotes, where it won't be expanded.
Compare:
foo=bar
echo 'foo=\""$foo\"'
echo 'foo="'"$foo"'"'
In the second example, we end the single quotes, and double-quote $foo, then start new single quotes for the final '.
It's probably easier to read if we expand using printf instead:
printf 'foo=%s\n' "$foo"
That's something you might want to run as a process substitution.
BUT...
This is a wrong and dangerous way to construct an SQL query (and the web server is also poor, if it forwards arbitrary queries - I hope it has no write permissions to the data). Read about "SQL command injection" and come back to this code when you understand the issues.
Nothing inside single quotes will be expanded by bash, including any double-quotes, and variable names. The good news is you can end your single-quoted section and immediately start a double-quoted section to introduce the variable, and it will all be concatenated into a single argument for the application (curl). Try:
/usr/bin/curl -s -vvvv http://hmvddrsvr:8044/query/service -u iamusr:pssd -d 'statement=DELETE FROM `test_bucket` WHERE type = "Metadata" AND market = "ES" AND status = "active" AND meta(test_bucket).id=\"'"$idp_sub"'\" ' -o /opt/SP/jboss/home/mayur/es_delete_response.txt
You can make your code strongly injection-proof by rejecting any string containing a double-quote, but you might reject some strings that have been legitimately escaped.
If you can use the q syntax to quote the string, you can make it more injection-proof, but I guess the attacker just has to inject ]":
/usr/bin/curl -s -vvvv http://hmvddrsvr:8044/query/service -u iamusr:pssd -d 'statement=DELETE FROM `test_bucket` WHERE type = "Metadata" AND market = "ES" AND status = "active" AND meta(test_bucket).id=q\"['"$idp_sub"]'\" ' -o /opt/SP/jboss/home/mayur/es_delete_response.txt
You could then search for and reject the pattern string ]" as your anti-injection, which will allow a much wider class of legitimate strings. You would have to tell the users that you have applied q[] quoting to their input, so they don't have to.
I am trying to access an env variable in Jenkins pipeline and want to use it in a Shell Script executing in the same pipeline but a differnt step,
pipeline {
agent any
tools {
maven 'M2'
}
environment {
stable_revision = sh(script: 'curl -H "Authorization: Basic $base64encoded" "https://xyz.sad" | jq -r "name"', returnStdout: true)
}
stages {
stage('Initial-Checks') {
steps {
echo "Stable Revision: ${env.stable_revision}" //displays the value
bat "sh && sh undeploy.sh"
}}
...
}}
This is a sample Shell script, it has many lines, but I have an issue in only accessing the above stable_revision variable,
#!/bin/bash
echo xyz = ${stable_revision} #### this gives the right value xyz = 22
echo xyz2 = ${stable_revision}/d ### here after the value the /d is written in new line
For example, let's say the stable_revision value is 22, then in the SH script echo I am getting the value as,
xyz2 = 22
/d
I want the value to be xyz2 = 22/d
You can use .trim() to strip off a trailing newline.
environment {
stable_revision = sh(script: 'curl -H "Authorization: Basic $base64encoded" "https://xyz.sad" | jq -r "name"', returnStdout: true).trim()
}
returnStdout (optional):
If checked, standard output from the task is returned as the step value as a String, rather than being printed
to the build log. (Standard error, if any, will still be printed to
the log.) You will often want to call .trim() on the result to strip
off a trailing newline.
https://jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#sh-shell-script
If you use bash instead of sh for your commands, you can benefit from Bash's built-in string transformations
Here it trims all trailing characters from the [:space:] class witch includes actual spaces and newlines.
echo "xyz2 = ${stable_revision%%[[:space:]]}/d"
If $stable_revision is always an integer, you can force the shell to use it like an integer with:
echo "xyz2 = $((stable_revision))/d"
If you are sure that $stable_revision contains no space, you can force the shell to trim all spaces by using it like a table element:
sr=($stable_revision); echo "xyz2 = ${sr[0]}/d"
You can also use the automatic trimming of a sub-shell returned value, that would trim any leading, trailing and duplicate spaces in-between:
echo "xyz2 = $(echo ${stable_revision})/d"`
I have the following snippet that is not working. Compiles but does not do what it is supposed to. Executing the same command on bash works. Why?
hash:="4ab32de"
cmd = "sed -i -e 's/clt_[0-9a-z]*/clt_"+hash+"/g' /tmp/test.env"
parts = strings.Fields(cmd)
for _, part :=range parts {
fmt.Printf("\n%s",part)
}
head = parts[0]
out, err = exec.Command(head,parts[1:]...).Output()
fmt.Printf("\nnew cmd is %s\n",cmd)
fmt.Printf("out:%s",string(out))
The output of parts is perfect, like this
sed
-i
-e
's/clt_[0-9a-z]*/clt_4ab32de/g'
/tmp/test.env
The exec package doesn't use the shell, so take out all quotes (and escapes). In your case, remove the single quotes.
Need help in fixing this bash script to set a variable with a value including double quotes. Somehow I am defining this incorrectly as my values foo and bar are not enclosed in double quotes as needed.
My script thus far:
#!/usr/local/bin/bash
set -e
set -x
host='127.0.0.1'
db='mydev'
_account="foo"
_profile="bar"
_version=$1
_mongo=$(which mongo);
exp="db.profile_versions_20170420.find({account:${_account}, profile:${_profile}, version:${_version}}).pretty();";
${_mongo} ${host}/${db} --eval "$exp"
set +x
Output shows:
+ host=127.0.0.1
+ db=mydev
+ _account=foo
+ _profile=bar
+ _version=201704112004
++ which mongo
+ _mongo=/usr/local/bin/mongo
+ exp='db.profile_versions_20170420.find({account:foo, profile:bar, version:201704112004}).pretty();'
+ /usr/local/bin/mongo 127.0.0.1/mydev --eval 'db.profile_versions_20170420.find({account:foo, profile:bar, version:201704112004}).pretty();'
MongoDB shell version: 3.2.4
connecting to: 127.0.0.1/mydev
2017-04-22T15:32:55.012-0700 E QUERY [thread1] ReferenceError: foo is not defined :
#(shell eval):1:36
What i need is account:"foo", profile:"bar" to be enclosed in double quotes.
In bash (and other POSIX shells), the following 2 states are equivalent:
_account=foo
_account="foo"
What you want to do is to preserve the quotations, therefore you can do the following:
_account='"foo"'
Since part of what you're doing here is forming JSON, consider using jq -- which will guarantee that it's well-formed, no matter what the values are.
host='127.0.0.1'
db='mydev'
_account="foo"
_profile="bar"
_version=$1
json=$(jq -n --arg account "$_account" --arg profile "$_profile" --arg version "$_version" \
'{$account, $profile, version: $version | tonumber}')
exp="db.profile_versions_20170420.find($json).pretty();"
mongo "${host}/${db}" --eval "$exp"
This makes jq responsible for adding literal quotes where appropriate, and will avoid attempted injection attacks (for instance, via a version passed in $1 containing something like 1, "other_argument": "malicious_value"), by replacing any literal " in a string with \"; a literal newline with \n, etc -- or, with the | tonumber conversion, failing outright with any non-numeric value.
Note that some of the syntax above requires jq 1.5 -- if you have 1.4 or prior, you'll want to write {account: $account, profile: $profile} instead of being able to write {$account, $profile} with the key names inferred from the variable names.
When you need to use double quotes inside a double quoted string, escape them with backslashes:
$ foo="acount:\"foo\"" sh -c 'echo $foo'
acount:"foo"
I needed to enquote something already in a variable and stick that in a variable. Expanding on Robert Seaman's answer, I found this worked:
VAR='"'$1'"'
(single quote, double quote, single quote, variable,single quote, double quote, single quote)
This is my code. I want to grab the p's value and insert it into the file changed.txt when matched 1. But it doesn't do what I want to, it seems it doesn't know what #{p} is
Net::SSH.start( "192.168.2.1", "root", :password => "password") do |ssh|
p = ssh.exec! "java -cp /var/lib/sonar/dev-tools.jar au.com.Tool test"
# puts #{p}
ssh.exec! "sed 's/1/#{p}/g' changed.txt"
end
The passing of the p value the way you have it should work fine. However, the sed command doesn't change the file. If you want it to change the file in place, use the -i option like so:
ssh.exec! "sed -i 's/1/#{p}/g' changed.txt"
Or if you want the changes in a different file, then use:
ssh.exec! "sed 's/1/#{p}/g' changed.txt > newfile.txt"
An alternative option would be:
ssh.exec! "sed -i 's/1/" + p + "/g' changed.txt"