I'm trying to pass some values into chef recipe from the json file.
All I want is to set some values in my recipe from the remote file.
My json file my_conf.json looks like this:
{
"something": {
"listen_port": "81",
"listen_path": "/myapp"
},
"users":{
"user": "me"
}
}
I run my chef-client with this json in params:
chef-client -z run.rb -j my_conf.json
In the recipe I tried:
#test = node[:something][:listen_port]
#test = node['something']['listen_port']
#test = node.default[:something][:listen_port]
But nothing works.
Any ideas?
node['something']['listen_port'] is correct but node[:something][:listen_port] will work as well. More likely assigning to an instance variable (#test =) instead of a normal local variable (test =) is confusing things. Chef does a lot of magic with Ruby scoping internally.
Related
I am receiving JSON from a http terraform data source
data "http" "example" {
url = "${var.cloudwatch_endpoint}/api/v0/components"
# Optional request headers
request_headers {
"Accept" = "application/json"
"X-Api-Key" = "${var.api_key}"
}
}
It outputs the following.
http = [{"componentID":"k8QEbeuHdDnU","name":"Jenkins","description":"","status":"Partial Outage","order":1553796836},{"componentID":"ui","name":"ui","description":"","status":"Operational","order":1554483781},{"componentID":"auth","name":"auth","description":"","status":"Operational","order":1554483781},{"componentID":"elig","name":"elig","description":"","status":"Operational","order":1554483781},{"componentID":"kong","name":"kong","description":"","status":"Operational","order":1554483781}]
which is a string in terraform. In order to convert this string into JSON I pass it to an external data source which is a simple ruby function. Here is the terraform to pass it.
data "external" "component_ids" {
program = ["ruby", "./fetchComponent.rb",]
query = {
data = "${data.http.example.body}"
}
}
Here is the ruby function
#!/usr/bin/env ruby
require 'json'
data = JSON.parse(STDIN.read)
results = data.to_json
STDOUT.write results
All of this works. The external data outputs the following (It appears the same as the http output) but according to terraform docs this should be a map
external1 = {
data = [{"componentID":"k8QEbeuHdDnU","name":"Jenkins","description":"","status":"Partial Outage","order":1553796836},{"componentID":"ui","name":"ui","description":"","status":"Operational","order":1554483781},{"componentID":"auth","name":"auth","description":"","status":"Operational","order":1554483781},{"componentID":"elig","name":"elig","description":"","status":"Operational","order":1554483781},{"componentID":"kong","name":"kong","description":"","status":"Operational","order":1554483781}]
}
I was expecting that I could now access data inside of the external data source. I am unable.
Ultimately what I want to do is create a list of the componentID variables which are located within the external data source.
Some things I have tried
* output.external: key "0" does not exist in map data.external.component_ids.result in:
${data.external.component_ids.result[0]}
* output.external: At column 3, line 1: element: argument 1 should be type list, got type string in:
${element(data.external.component_ids.result["componentID"],0)}
* output.external: key "componentID" does not exist in map data.external.component_ids.result in:
${data.external.component_ids.result["componentID"]}
ternal: lookup: lookup failed to find 'componentID' in:
${lookup(data.external.component_ids.*.result[0], "componentID")}
I appreciate the help.
can't test with the variable cloudwatch_endpoint, so I have to think about the solution.
Terraform can't decode json directly before 0.11.x. But there is a workaround to work on nested lists.
Your ruby need be adjusted to make output as variable http below, then you should be fine to get what you need.
$ cat main.tf
variable "http" {
type = "list"
default = [{componentID = "k8QEbeuHdDnU", name = "Jenkins"}]
}
output "http" {
value = "${lookup(var.http[0], "componentID")}"
}
$ terraform apply
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
http = k8QEbeuHdDnU
The following JSON is a transaction what will be sent to the Ripple Network to query accounts that hold cryptographic assets at a Gateway (somewhat like a bank, more like a trust account between its clients). This script is to be used in conjunction with PHP to fetch a Gateway's issued balances and ignored it's hot-wallet or day-to-day operations wallet. My question is what is the proper way to:
a. Assign JSON within a Ruby variable?
b. What is the best way to escape double quotes and deal with newlines where brackets and square brackets occur within the JSON syntax?
The JSON follows:
ripple_path="/home/rippled/build/rippled"
conf = "--conf /etc/rippled/rippled.cfg"
puts "About to set the JSON lines "
gatewayStart = "\"method\": \"gateway_balances\","
paramsLine = "\"params\": [ {"
accountLine = "\"account\": \"rGgS5Hw3PhSp3VNT43PDTXze9YfdthHUH\","
hotwalletLine = "\"hotwallet\": \"rKYNhsT3aLymkGH7WL7ZUHkm6RE27iuM4C\","
liLine = "\"ledger_index\": \"validated\","
strictLine = "\"strict\": "
trueLine = true
endLine = " } ] }"
balancesLine = "#{gatewayStart} #{paramsLine} #{accountLine} #>{hotwalletLine} #{liLine} #{strictLine} #{trueLine} #{endLine}"
lineString = "#{balancesLine.to_s}"
linetoJSON = "#{lineString}"
puts "linetoJSON: #{linetoJSON} "
cmd2=`#{ripple_path} #{conf} json gateway_balances #{linetoJSON}`
cmder="#{ripple_path} #{conf} json gateway_balances #{linetoJSON}"
puts "Done."
The output is:
root#xagate:WorkingDirectory# ruby gatewaybal.rb
About to set the JSON lines
linetoJSON: "method": "gateway_balances", "params": [ { "account":
"rGgS5Hw3PhSp3VNT43PDTXze9YfdthHUH", "hotwallet": "rKYNhsT3aLymkGH7WL7ZUHkm6RE27iuM4C", "ledger_index": "validated", "strict":rue } ] }
Loading: "/etc/rippled/rippled.cfg"
rippled [options] <command> <params>
General Options:
-h [ --help ] Display this message.
.....
Done.
It is noteworthy that this command also returns a badSyntax error when executed manually via the command line. Please see here for the mirror of this issue raised on the ripple forums.
jsonLine = "'{ \"account\": \"rGgS5Hw3PhSp3VNT43PDTXze9YfdthHUH\", \"hotwallet\": \"rKYNhsT3aLymkGH7WL7ZUHkm6RE27iuM4C\", \"ledger_index\": \"validated\", \"strict\": true }'"
Is the proper way to assign this JSON within a single variable; this solution was provided by JoelKatz. The completed code is now available on GitHub.
I have the following two settings in my elasticsearch.yml file. They are the only ones that pull from environment variables.
cloud.aws.access_key: ${AWS_ACCESS_KEY_ID}
cloud.aws.secret_key: ${AWS_SECRET_KEY}
When I restart elasticsearch to load these from the environment, I get an error that it can't resolve them. I've tested it and it will not resolve either, so this error applies to both (it just fails on the bottom one first)
- IllegalArgumentException[Could not resolve placeholder 'AWS_SECRET_KEY']
java.lang.IllegalArgumentException: Could not resolve placeholder 'AWS_SECRET_KEY'
at org.elasticsearch.common.property.PropertyPlaceholder.parseStringValue(PropertyPlaceholder.java:124)
at org.elasticsearch.common.property.PropertyPlaceholder.replacePlaceholders(PropertyPlaceholder.java:81)
at org.elasticsearch.common.settings.ImmutableSettings$Builder.replacePropertyPlaceholders(ImmutableSettings.java:1060)
at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareSettings(InternalSettingsPreparer.java:101)
at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:106)
at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:177)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
I did some investigating through elasticsearch's code repository on github and discovered this bit of code that pulls from the environment variables.
ImmutableSettings.java#resolvePlaceholder from elasticsearch#github
Namely. the lines inside that function that should be pulling from the environment variables are these one:
Code from resolvePlaceholder that pulls out environment variables
However, after resolvePlaceholder is run from inside function PropertyPlaceholder#parseStringValue, the System.getenv call must be returning null as that is the only way for that error to be thrown.
I wrote a simple test program that is essentially a copy of ImmutableSettings.java#resolvePlaceholder to test that System.getenv was pulling out the environment variables correctly on my system. This in fact returns the values I expect.
public class Cool {
public static void main(String[] args) {
System.out.println(resolvePlaceholder(args[0]));
}
public static String resolvePlaceholder(String placeholderName) {
if (placeholderName.startsWith("env.")) {
// explicit env var prefix
System.out.println("1: placeholderName.startsWith(\"env.\")");
return System.getenv(placeholderName.substring("env.".length()));
}
String value = System.getProperty(placeholderName);
if (value != null) {
System.out.println("2: System.getProperty");
return value;
}
value = System.getenv(placeholderName);
if (value != null) {
System.out.println("3: System.getenv");
return value;
}
return "Map should've had it";
}
}
When run, this is the output, showing we are getting the set environment variables (keys hidden for obvious reasons):
[ec2-user#ip-172-31-34-195 ~]$ java Cool AWS_SECRET_KEY
3: System.getenv
XXXXXXXXXXXXXXXXXX
[ec2-user#ip-172-31-34-195 ~]$ java Cool AWS_ACCESS_KEY_ID
3: System.getenv
XXXXXXXXXXXXXXXXXX
What is it about elasticsearch that isn't able to parse my environment variables from elasticsearch.yml? I've done quite a bit of digging at this point but I'm sure there is a simple solution around the corner. Any help would be very much appreciated.
I figured out the issue.
As I am running elasticsearch as a linux service, rather than a shell application, it has access to no environment variables except for a very select few.
I added the following line to the end of /etc/sysconfig/elasticsearch to load the environment variables I wanted available to the program:
. /path/to/environment/variables
I seem to have run into a syntax error on a previously-working puppet manifest. This is running on a local vagrant box with Ubuntu 12.04, and Puppet version 3.4.2. The puppet stuff was all generated at puphpet.com.
The error I'm getting is:
Error: Could not parse for environment production: Syntax error at '|'
at /tmp/vagrant-puppet/manifests/default.pp:263:29 on node
vagrant.example.com
Line 263 of default.pp is the second line of this snippet:
if count($php_values['ini']) > 0 {
$php_values['ini'].each { |$key, $value|
puphpet::ini { $key:
entry => "CUSTOM/${key}",
value => $value,
php_version => $php_values['version'],
webserver => $php_webserver_service
}
}
}
It looks like you haven't set parser to future.
Run this command:
puppet config print parser
If it returns current, you don't have access to the .each function. To change this, edit /etc/puppet/puppet.conf, and put parser = future under the [main] block. The above command should then return future.
Reference: http://docs.puppetlabs.com/references/latest/function.html#each
I'm trying to pass attributes to my recipe at run-time of chef client.
following is my default.rb attribute file:
if node['os'] == 'windows'
if node.attribute?('temp')
default['param']['name'] = node['temp']['tempname']
else node.attribute?('base')
default['param']['name'] = node['base']['nodename']
end
end
I first ran the chef-client with only the base attribute defined in the node.json file. My node.json looked like :
{
"base": {
"nodename": "John"
}
}
Chef-client run was successfull and the attribute was set accordingly. i.e.
default['param']['name'] = "John"
Than I ran chef-client with both base and temp attribute defined in the node.json.
And here is my node.json file :
{
"base": {
"nodename": "John"
},
"temp": {
"tempname": "Mike"
}
}
Chef-client again ran successfully and the attribute was set accordingly.
default['param']['name'] = "Mike"
The problem is when I run the chef-client again and pass only base attribute from the node.json file(see below for the file), the code doesn't seem to enter the else loop. It just runs the `if' loop with old value for temp attribute(Mike).
{
"base": {
"nodename": "John-2"
}
}
Any ideas to where I'm going wrong.
The second time you ran chef-client, it saved the node[:temp][:nodename] attribute to the node object on the Chef server.
When you ran chef-client again, it loaded the node's attributes back from the server before reaching the attributes/default.rb file, so node[:temp][:nodename] was set when it reached the conditions.
I'm not sure what you're trying to achieve, but it would probably be best to explicitly tell which attribute you want to use through another attribute (and maybe assume a default for it), such as:
default[:name_attribute] = 'temp'
then,
if node['os'] == 'windows'
if node['name_attribute'] == 'temp'
default['param']['name'] = node['temp']['tempname']
else
default['param']['name'] = node['base']['nodename']
end
end