I am trying to create a bash function where I can switch environments, below is what I tried. I installed the npm json package globally to edit the relevant file inline, but that may not be needed.
devUrl () { 'https://some-url.com'; }
testUrl () { 'https://some-test-url.com'; }
switchEnv () { 'json -I -f config.json -e "this.url = (this.url == "$1" ? "$2" : "$1")"'; }
alias switch='switchEnv devUrl testUrl';
what am I missing/doing wrong?
I also tried to template the strings devUrl and testUrl inside the double quotes in the switchEnv function, but that's where I got stuck.
Update:
I tried this:
devUrl='https://some-url.com'
testUrl='https://some-test-url.com'
switchEnv() { json -I -f config.json -e "this.url = (this.url == "$devUrl" ? "$testUrl" : "$devUrl")"; }
but got the following error:
this.url = (this.url == https://some-url.com ? https://some-test-url.com : https://some-url.com)
^
SyntaxError: Unexpected token :
at new Function (<anonymous>)
at main (/usr/local/lib/node_modules/json/lib/json.js:1289:27)
it doesn't like the : after https for some reason.
The below is a sample implementation that does what you're looking for; see the notes below for some details on why it was implemented as it was.
# String assignments
devUrl='https://some-url.com'
testUrl='https://some-test-url.com'
configFile="$PWD/config.json"
# Functions
switchEnv() {
local tempfile
tempfile=$(mktemp "$configFile.XXXXXX")
if jq --arg a "$1" \
--arg b "$2" \
'if .url == $a then .url=$b else .url=$a end' <"$configFile" >"$tempfile"; then
mv -- "$tempfile" "$configFile"
else
rm -f -- "$tempfile"
return 1
fi
}
switch() { switchEnv "$devUrl" "$testUrl"; }
Notes:
Unlike aliases, function bodies should be actual code, not strings containing code.
Storing data (as opposed to code) should be done using variables (be they strings or arrays, as appropriate).
Passing data out-of-band from code allows a malicious value of devUrl or testUrl to escape its quoting and run arbitrary json or jq commands. This is wise, in no small part because these languages become more powerful over time: Old versions of jq had no operations that wouldn't run in constant-time, whereas modern versions of the language allow code to be expressed that can be used for denial-of-service attacks; future versions might also add I/O support, allowing malicious code to have a wider array of surprising behaviors.
Now, let's say you were going to ignore the warning (above) about the importance of separating data and code. How would we modify your current code to behave "correctly" (when the strings being handled are non-malicious)?
switchEnv() {
json -I -f config.json -e 'this.url = (this.url == "'"$devUrl"'" ? "'"$testUrl"'" : "'"$devUrl"'")'; }
}
Related
I'm trying to include a bash script in an AWS SSM Document, via the Terraform templatefile function. In the aws:runShellScript section of the SSM document, I have a Bash for loop with an # sign that seems to be creating an error during terraform validate.
Version of terraform: 0.13.5
Inside main.tf file:
resource "aws_ssm_document" "magical_document" {
name = "magical_ssm_doc"
document_type = "Command"
document_format = "YAML"
target_type = "/AWS::EC2::Instance"
content = templatefile(
"${path.module}/ssm-doc.yml",
{
Foo: var.foo
}
)
}
Inside my ssm-doc.yaml file, I loop through an array:
for i in "$\{arr[#]\}"; do
if test -f "$i" ; then
echo "[monitor://$i]" >> $f
echo "disabled=0" >> $f
echo "index=$INDEX" >> $f
fi
done
Error:
Error: Error in function call
Call to function "templatefile" failed:
./ssm-doc.yml:1,18-19: Invalid character;
This character is not used within the language., and 1 other diagnostic(s).
I tried escaping the # symbol, like \#, but it didn't help. How do I
Although the error is pointing to the # symbol as being the cause of the error, it's the ${ } that's causing the problem, because this is Terraform interpolation syntax, and it applies to templatefiles too. As the docs say:
The template syntax is the same as for string templates in the main Terraform language, including interpolation sequences delimited with ${ ... }.
And the way to escape interpolation syntax in Terraform is with a double dollar sign.
for i in "$${arr[#]}"; do
if test -f "$i" ; then
echo "[monitor://$i]" >> $f
echo "disabled=0" >> $f
echo "index=$INDEX" >> $f
fi
done
The interpolation syntax is useful with templatefile if you're trying to pass in an argument, such as, in the question Foo. This argument could be accessed within the yaml file as ${Foo}.
By the way, although this article didn't give the answer to this exact issue, it helped me get a deeper appreciation for all the work Terraform is doing to handle different languages via the templatefile function. It had some cool tricks for doing replacements to escape for different scenarios.
I have two jsons :
env.json
{
"environment":"INT"
}
roles.json
{
"run_list":[
"recipe[splunk-dj]",
"recipe[tideway]",
"recipe[AlertsSearch::newrelic]",
"recipe[AlertsSearch]"
]
}
expected output should be some thing like this :
{
"environment":"INT",
"run_list":[
"recipe[splunk-dj]",
"recipe[tideway]",
"recipe[AlertsSearch::newrelic]",
"recipe[AlertsSearch]"
]
}
I need to merge these two json (and other like these two) into one single json using only available inbuilt bash commands.
only have sed, cat, echo, tail, wc at my disposal.
Tell whoever put the constraint "bash only" on the project that bash is not sufficient for processing JSON, and get jq.
$ jq --slurp 'add' env.json roles.json
I couldn't use jq either as I was limited due to client's webhost jailing the user on the command line with limited binaries as most discount/reseller web hosting companies do. Luckily they usually have PHP available and you can do a oneliner command like this which something like what I would place in my install/setup bash script for example.
php -r '$json1 = "./env.json";$json2 = "./roles.json";$data = array_merge(json_decode(file_get_contents($json1), true),json_decode(file_get_contents($json2),true));echo json_encode($data, JSON_PRETTY_PRINT);'
For clarity php -r accepts line feeds as well so using this also works.
php -r '
$json1 = "./env.json";
$json2 = "./roles.json";
$data = array_merge(json_decode(file_get_contents($json1), true), json_decode(file_get_contents($json2), true));
echo json_encode($data, JSON_PRETTY_PRINT);'
Output
{
"environment": "INT",
"run_list": [
"recipe[splunk-dj]",
"recipe[tideway]",
"recipe[AlertsSearch::newrelic]",
"recipe[AlertsSearch]"
]
}
A little bit hacky, but hopefully will do.
env_lines=`wc -l < $1`
env_output=`head -n $(($env_lines - 1)) $1`
roles_lines=`wc -l < $2`
roles_output=`tail -n $(($roles_lines - 1)) $2`
echo "$env_output" "," "$roles_output"
Expressions like this are short, but not super-readable:
if [ -f .bash_profile ]; then
...
fi
There are also other possible flags for expressions, for instance:
Description
-d file
True if file is a directory.
-e file
True if file exists.
-f file
True if file exists and is a regular file.
-L file
True if file is a symbolic link.
-z string
True if string is empty. (most innatural IMO)
-n string
True if string is not empty.
... and others...
Are there longer self-explaining versions? Something like:
[ --file-exists .bash_profile ]
This is extremely well documented already. As you can see, there is no long-form version of those conditional expressions.
If you want to use these in a more readable way, you can always create your own functions:
function is_a_file() { test -f "$1"; }
function is_a_dir() { test -d "$1"; }
#etc.
if is_a_file /the/file/name
then
#do something
fi
test is the canonical name for the [ command that is typically used. Its return value becomes the return value of the function, so we can use it in exactly the same way in an if statement.
No, use comments or use something less cryptic like Python
A recent vulnerability, CVE-2014-6271, in how Bash interprets environment variables was disclosed. The exploit relies on Bash parsing some environment variable declarations as function definitions, but then continuing to execute code following the definition:
$ x='() { echo i do nothing; }; echo vulnerable' bash -c ':'
vulnerable
But I don't get it. There's nothing I've been able to find in the Bash manual about interpreting environment variables as functions at all (except for inheriting functions, which is different). Indeed, a proper named function definition is just treated as a value:
$ x='y() { :; }' bash -c 'echo $x'
y() { :; }
But a corrupt one prints nothing:
$ x='() { :; }' bash -c 'echo $x'
$ # Nothing but newline
The corrupt function is unnamed, and so I can't just call it. Is this vulnerability a pure implementation bug, or is there an intended feature here, that I just can't see?
Update
Per Barmar's comment, I hypothesized the name of the function was the parameter name:
$ n='() { echo wat; }' bash -c 'n'
wat
Which I could swear I tried before, but I guess I didn't try hard enough. It's repeatable now. Here's a little more testing:
$ env n='() { echo wat; }; echo vuln' bash -c 'n'
vuln
wat
$ env n='() { echo wat; }; echo $1' bash -c 'n 2' 3 -- 4
wat
…so apparently the args are not set at the time the exploit executes.
Anyway, the basic answer to my question is, yes, this is how Bash implements inherited functions.
This seems like an implementation bug.
Apparently, the way exported functions work in bash is that they use specially-formatted environment variables. If you export a function:
f() { ... }
it defines an environment variable like:
f='() { ... }'
What's probably happening is that when the new shell sees an environment variable whose value begins with (), it prepends the variable name and executes the resulting string. The bug is that this includes executing anything after the function definition as well.
The fix described is apparently to parse the result to see if it's a valid function definition. If not, it prints the warning about the invalid function definition attempt.
This article confirms my explanation of the cause of the bug. It also goes into a little more detail about how the fix resolves it: not only do they parse the values more carefully, but variables that are used to pass exported functions follow a special naming convention. This naming convention is different from that used for the environment variables created for CGI scripts, so an HTTP client should never be able to get its foot into this door.
The following:
x='() { echo I do nothing; }; echo vulnerable' bash -c 'typeset -f'
prints
vulnerable
x ()
{
echo I do nothing
}
declare -fx x
seems, than Bash, after having parsed the x=..., discovered it as a function, exported it, saw the declare -fx x and allowed the execution of the command after the declaration.
echo vulnerable
x='() { x; }; echo vulnerable' bash -c 'typeset -f'
prints:
vulnerable
x ()
{
echo I do nothing
}
and running the x
x='() { x; }; echo Vulnerable' bash -c 'x'
prints
Vulnerable
Segmentation fault: 11
segfaults - infinite recursive calls
It doesn't overrides already defined function
$ x() { echo Something; }
$ declare -fx x
$ x='() { x; }; echo Vulnerable' bash -c 'typeset -f'
prints:
x ()
{
echo Something
}
declare -fx x
e.g. the x remains the previously (correctly) defined function.
For the Bash 4.3.25(1)-release the vulnerability is closed, so
x='() { echo I do nothing; }; echo Vulnerable' bash -c ':'
prints
bash: warning: x: ignoring function definition attempt
bash: error importing function definition for `x'
but - what is strange (at least for me)
x='() { x; };' bash -c 'typeset -f'
STILL PRINTS
x ()
{
x
}
declare -fx x
and the
x='() { x; };' bash -c 'x'
segmentation faults too, so it STILL accept the strange function definition...
I think it's worth looking at the Bash code itself. The patch gives a bit of insight as to the problem. In particular,
*** ../bash-4.3-patched/variables.c 2014-05-15 08:26:50.000000000 -0400
--- variables.c 2014-09-14 14:23:35.000000000 -0400
***************
*** 359,369 ****
strcpy (temp_string + char_index + 1, string);
! if (posixly_correct == 0 || legal_identifier (name))
! parse_and_execute (temp_string, name, SEVAL_NONINT|SEVAL_NOHIST);
!
! /* Ancient backwards compatibility. Old versions of bash exported
! functions like name()=() {...} */
! if (name[char_index - 1] == ')' && name[char_index - 2] == '(')
! name[char_index - 2] = '\0';
if (temp_var = find_function (name))
--- 364,372 ----
strcpy (temp_string + char_index + 1, string);
! /* Don't import function names that are invalid identifiers from the
! environment, though we still allow them to be defined as shell
! variables. */
! if (legal_identifier (name))
! parse_and_execute (temp_string, name, SEVAL_NONINT|SEVAL_NOHIST|SEVAL_FUNCDEF|SEVAL_ONECMD);
if (temp_var = find_function (name))
When Bash exports a function, it shows up as an environment variable, for example:
$ foo() { echo 'hello world'; }
$ export -f foo
$ cat /proc/self/environ | tr '\0' '\n' | grep -A1 foo
foo=() { echo 'hello world'
}
When a new Bash process finds a function defined this way in its environment, it evalutes the code in the variable using parse_and_execute(). For normal, non-malicious code, executing it simply defines the function in Bash and moves on. However, because it's passed to a generic execution function, Bash will correctly parse and execute additional code defined in that variable after the function definition.
You can see that in the new code, a flag called SEVAL_ONECMD has been added that tells Bash to only evaluate the first command (that is, the function definition) and SEVAL_FUNCDEF to only allow functio0n definitions.
In regard to your question about documentation, notice here in the commandline documentation for the env command, that a study of the syntax shows that env is working as documented.
There are, optionally, 4 possible options
An optional hyphen as a synonym for -i (for backward compatibility I assume)
Zero or more NAME=VALUE pairs. These are the variable assignment(s) which could include function definitions.
Note that no semicolon (;) is required between or following the assignments.
The last argument(s) can be a single command followed by its argument(s). It will run with whatever permissions have been granted to the login being used. Security is controlled by restricting permissions on the login user and setting permissions on user-accessible executables such that users other than the executable's owner can only read and execute the program, not alter it.
[ spot#LX03:~ ] env --help
Usage: env [OPTION]... [-] [NAME=VALUE]... [COMMAND [ARG]...]
Set each NAME to VALUE in the environment and run COMMAND.
-i, --ignore-environment start with an empty environment
-u, --unset=NAME remove variable from the environment
--help display this help and exit
--version output version information and exit
A mere - implies -i. If no COMMAND, print the resulting environment.
Report env bugs to bug-coreutils#gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
Report env translation bugs to <http://translationproject.org/team/>
So I have an array like:
al_ap_version=('ap_version' '[[ $data -ne $version ]]')
And the condition gets evaluated inside a loop like:
for alert in alert_list; do
data=$(tail -1 somefile)
condition=$(eval echo \${$alert[1]})
if eval "$condition" ; then
echo SomeAlert
fi
done
Whilst this generally works with many scenarios, if $data returns something like "-/-" or "4.2.9", I get errors as it doesn't seem to like complex strings in the variable.
Obviously I can't enclose the variable in single quotes as it won't expand so I'm after any ideas to expand the $data variable (or indeed the $version var which suffers the same possible fate) in a way that the evaluation can handle?
Ignoring the fact that eval is probably super dangerous to use here (unless the data in somefile is controlled by you and only you), there are a few issues to fix in your example code.
In your for loop, alert_list needs to be $alert_list.
Also, as pointed out by #choroba, you should be using != instead of -ne since your input isn't always an integer.
Finally, while debugging, you can add set -x to the top of your script, or add -x to the end of your shebang line to enable verbose output (helps to determine how bash is expanding your variables).
This works for me:
#!/bin/bash -x
data=2.2
version=1
al_ap_version=('ap_version' '[[ $data != $version ]]')
alert_list='al_ap_version'
for alert in $alert_list; do
condition=$(eval echo \${$alert[1]})
if eval "$condition"; then
echo "alert"
fi
done
You could try a more functional approach, even though bash is only just barely capable of such things. On the whole, it is usually a lot easier to pack an action to be executed into a bash function and refer to it with the name of the function, than to try to maintain the action as a string to be evaluated.
But first, the use of an array of names of arrays is awkward. Let's get rid of it.
It's not clear to me the point of element 0, ap_version, in the array al_ap_version but I suppose it has something to do with error messages. If the order of alert processing isn't important, you could replace the list of names of arrays with a single associative array:
declare -A alert_list
alert_list[ap_version]=... # see below
alert_list[os_dsk]=...
and then process them with:
for alert_name in ${!alert_list[#]}; do
alert=${alert_list[$alert_name]}
...
done
Having done that, we can get rid of the eval, with its consequent ugly necessity for juggling quotes, by creating a bash function for each alert:
check_ap_version() {
(($version != $1))
}
Edit: It seems that $1 is not necessarily numeric, so it would be better to use a non-numeric comparison, although exact version match might not be what you're after either. So perhaps it would be better to use:
check_ap_version() {
[[ $version != $1 ]]
}
Note the convention that the first argument of the function is the data value.
Now we can insert the name of the function into the alert array, and call it indirectly in the loop:
declare -A alert_list
alert_list[ap_version]=check_ap_version
alert_list[os_dsk]=check_op_dsk
check_alerts() {
local alert_name alert
local data=$(tail -1 somefile)
for alert_name in ${!alert_list[#]}; do
alert=${alert_list[$alert_name]}
if $alert "$data"; then
signal_alert $alert_name
fi
done
}
If you're prepared to be more disciplined about the function names, you can avoid the associative array, and thereby process the alerts in order. Suppose, for example, that every function has the name check_<alert_name>. Then the above could be:
alert_list=(ap_version os_dsk)
check_alerts() {
local alert_name
local data=$(tail -1 somefile)
for alert_name in $alert_list[#]; do
if check_$alert_name "$data"; then
signal_alert $alert_name
fi
done
}