Chef delay attribute assignment via data bag - ruby

So i have a bit of a pickle.
I have an encrypted data bag to store LDAP passwords. In my node run list, one of my recipes installs the secret key onto my client machine.
In my problematic cookbook, i have a helper (in /libraries) that pulls data from AD (using LDAP). Problem is, i can't find a way to delay the assignment of my node attribute after initial compile phase.
Take this line of code as example :
node.override['yp_chefserver']['osAdminUser'] = node['yp_chefserver']['osAdminUser'] + get_sam("#{data_bag_item('yp_chefserver', 'ldap', IO.read('/etc/chef/secret/yp_chefserver'))['ldap_password']}")
Im trying to override an attribute by adding an array returned by my helper function "get_sam" which returns an array, but it needs to run AFTER the compile phase since the file "/etc/chef/secret/yp_chefserver" doesnt exist before the convergence of my runlist.
So my question : Is there a way to assign node attributes via data_bag_items during the execution phase?
Some things i've tried :
ruby_block 'attribution' do
only_if { File.exist?('/etc/chef/secret/yp_chefserver')}
block do
node.override['yp_chefserver']['osAdminUser'] = node['yp_chefserver']['osAdminUser'] + get_sam("#{data_bag_item('yp_chefserver', 'ldap', IO.read('/etc/chef/secret/yp_chefserver'))['ldap_password']}")
Chef::Log.warn("content of osAdminUser : #{node['yp_chefserver']['osAdminUser']}")
end
end
This doesn't work because the custom resource ruby_block doesn't have the method "data_bag_item". I've tried using lazy attributes in my "chef_server" custom resource, but same problem.
I also tried having the attribution done directly in my helper module, but since the helper module compiles before the exec phase, the file doesn't exist when it assigns the variable.
Here is the helper function in question should anyone wonder, it pulls the SamAccountName from LDAP to assign admin users to my chef server. :
module YpChefserver
module LDAP
require 'net-ldap'
#ldap
def get_ldap(ldap_password)
if #ldap.nil?
#ldap = Net::LDAP.new :host => "ADSERVER",
:port => 389,
:auth => {
:method => :simple,
:username => "CN=USERNAME,OU=East Service Accounts,OU=System Accounts,DC=ad,DC=ypg,DC=com",
:password => "#{ldap_password}"
}
end
#ldap
end
def get_ldap_users(ldap_password)
filter = Net::LDAP::Filter.eq("cn", "DevOps")
treebase = "dc=ad, dc=ypg, dc=com"
get_ldap(ldap_password).search(:base => treebase, :filter => filter) do |entry|
#puts "DN: #{entry.dn}"
entry.each do |attribute, values|
return values if attribute == :member
end
end
end
def get_sam(ldap_password)
samacc = Array.new
get_ldap_users(ldap_password).entries.each{ |elem|
y = elem.to_s.split(/[,=]/)
filter = Net::LDAP::Filter.eq("cn", y[1])
treebase = "OU=Support Users and Groups,OU=CGI Support,DC=ad,DC=ypg,DC=com"
get_ldap(ldap_password).search(:base => treebase, :filter => filter, :attributes => "SamAccountName") do |entry|
samacc << entry.samaccountname
end
}
return samacc
end
end
end

Turns out you can actually call it inside a ruby block, just by using the actual Chef call instead of the resource name, as follow :
ruby_block 'attributes' do
only_if {File.exist?('/etc/chef/secret/yp_chefserver')}
block do
dtbg = Chef::EncryptedDataBagItem.load('yp_chefserver','ldap',"IO.read('/etc/chef/secret/yp_chefserver')")
end
end
Leaving this here for those who might need it
EDIT :
Here is final function using the code mentionned above to pull accounts from AD, using encrypted data bags to provide the password and to then pass those results to my node attributes, all during the execution phase :
ruby_block 'attributes' do
extend YpChefserver::LDAP
only_if {File.exist?('/etc/chef/secret/yp_chefserver')}
block do
# Chef::Config[:encrypted_data_bag_secret] = '/etc/chef/secret/yp_chefserver'
dtbg = Chef::EncryptedDataBagItem.load('yp_chefserver','ldap')
node.override['yp_chefserver']['ldap_pw'] = dtbg['ldap_password']
userarray = Array.new
userarray.push("#{node['yp_chefserver']['osAdminUser']}")
get_sam("#{node['yp_chefserver']['ldap_pw']}").each { |i| userarray.push(i[0]) }
node.override['yp_chefserver']['authorized_users'] = userarray
node.override['yp_chefserver']['local_admin_pw'] = dtbg['local_admin_pw']
end
end

Related

How to create crontab job in Chef based on a variable values from ruby code

Hello I'm new to Chef and Ruby.
I'm trying to make a recipe in Chef
To create a cron job on a server
based on a value of a variable that I get inside my ruby code.
Gem.clear_paths
node.default["value"] = "nil"
require 'net/http'
ruby_block "do-http-request-with-cutom-header" do
block do
Net::HTTP.get('example.com', '/index.html') # => String
uri = URI('http://example.com/index.html')
params = { :limit => 10, :page => 3 }
uri.query = URI.encode_www_form(params)
res = Net::HTTP.get_response(uri)
puts res.body if res.is_a?(Net::HTTPSuccess)
value= res.code
node["value"] = value
end
end
if node["value"] == "nil" then
cron "cassandra repair job" do
action :delete
end
else
cron "cassandra repair job" do
hour "0"
minute "55"
weekday node["value"]
mailto "root#localhost"
user "root"
command "/opt/cassandra/bin/nodetool repair -par -inc -pr"
end
end
I know that chef has Lazy Evaluation variable method and ruby code is executing on a converge phase, but I can not figure out the way to modify my code.
How can I use lazy evaluation in my code ?
node['value'] = value will create an attribute at the normal level, caveat, it is saved on the node object and stay there forever.
As you're using a volatile attribute coming from external source, you should use node.run_state['value'] which purpose is to keep transient value during the run.
Now, as you said it, you need to use lazy evaluation in your later resources for the value and for the action as you wish a different action depending of the external service return.
Example with update use of run_state here, untested code:
node.run_state['value'] = nil
node.run_state['action'] = :delete
require 'net/http'
ruby_block "do-http-request-with-cutom-header" do
block do
Net::HTTP.get('example.com', '/index.html') # => String
uri = URI('http://example.com/index.html')
params = { :limit => 10, :page => 3 }
uri.query = URI.encode_www_form(params)
res = Net::HTTP.get_response(uri)
puts res.body if res.is_a?(Net::HTTPSuccess)
value= res.code
node.run_state['value'] = value
node.run_state['action'] = :create
end
end
cron 'cassandra repair job' do
hour '0'
minute '55'
weekday lazy { node.run_state['value'] }
mailto 'root#localhost'
user 'root'
command '/opt/cassandra/bin/nodetool repair -par -inc -pr'
action lazy { node.run_state['action'] }
end
Using lazy on the action parameter is possible since chef 12.4, if you're under this you'll have to craft the resource and run it within the ruby block.
Example crafted from answer here (still untested):
ruby_block "do-http-request-with-cutom-header" do
block do
Net::HTTP.get('example.com', '/index.html') # => String
uri = URI('http://example.com/index.html')
params = { :limit => 10, :page => 3 }
uri.query = URI.encode_www_form(params)
res = Net::HTTP.get_response(uri)
puts res.body if res.is_a?(Net::HTTPSuccess)
if res.code.nil?
r = Chef::Resource::Cron.new "cassandra repair job"
r.run_action :delete
else
r = Chef::Resource::Cron.new "cassandra repair job"
r.hour "0"
r.minute "55"
r.weekday res.code
r.mailto "root#localhost"
r.user "root"
r.command "/opt/cassandra/bin/nodetool repair -par -inc -pr"
r.run_action :create
end
end
end
But at this point, it seems you would better get this information from a custom ohai plugin and use your original code without lazy.

Why am I getting an unsupportedSchemeError

I am writing a program that uses Mechanize to scrape a student's grades and classes from edline.net using a student account and return the data I need. However, after logging in, from the homepage I have to access a link (called 'Private Reports') which will then dynamically return a page of links to each of the student's classes and respective grades.
When testing I create a new object my_account that has several instance variables including the homepage. I pointed new variables to the instance variables for this to be more simple to read):
result_page = agent.page.link_with(:text => 'Private Reports').click
I get:
Mechanize::UnsupportedSchemeError
But if I were to replace :click with :text it responds correctly and result_page will equal the link's text "Private Reports"
Why does it respond to :text correctly but give an error for :click? Is there a way to get around this or should I rethink my solution to this problem?
Here's the code:
class AccountFetcher
EDLINE_LOGIN_URL = 'http://edline.net/Index.page'
def self.fetch_form(agent)
# uses #agent of an Account object
page = agent.get(EDLINE_LOGIN_URL)
form = page.form('authenticationEntryForm')
end
# edline's login form has to have several pre-conditions met before submitting the form
def self.initialize_form(agent)
form = AccountFetcher.fetch_form(agent)
submit_event = form.field_with(:name => 'submitEvent')
enter_clicked = form.field_with(:name => 'enterClicked')
ajax_support = form.field_with(:name => 'ajaxSupported')
ajax_support.value = 'yes'
enter_clicked.value = true
submit_event.value = 1
return form
end
# logs the user in and returns the homepage
def self.fetch_homepage(u_username, u_password, agent)
form = AccountFetcher.initialize_form(agent)
username = form.field_with(:name => 'screenName')
password = form.field_with(:name => 'kclq')
username.value = u_username
password.value = u_password
form.submit
end
end
# class Account will be expanded later on but here are the bare bones for a user to log in to their account
class Account
attr_accessor :report, :agent, :username, :password
def initialize(u_username, u_password)
#agent = Mechanize.new
#username = u_username
#password = u_password
end
def login
page = AccountFetcher.fetch_homepage(self.username, self.password, self.agent)
#report = page
end
end
my_account = Account.new('ex_username', 'ex_password')
my_account.login
page = my_account.report
agent = my_account.agent
page = agent.page.link_with(:text => 'Private Reports').click
Where does the link point to? Ie, what's the href?
I ask because "scheme" usually refers to something like http or https or ftp, so maybe the link has a weird scheme that mechanize doesn't know how to handle, hence Mechanize::UnsupportedSchemeError

How to use .tap method to build associated record?

I have an omniauth authentication model I'm building that's associated to a user.
aka user has many authentications.
I wish to build up key-value pairs of this authentication models using tap because twitter provides a secret key while facebook does not.
So if I have this, I want to accomplish the following conditional statement using the .tap method instead.
class User < ActiveRecord::Base
def apply_omniauth(omni)
if omni['credentials']['secret']
self.authentications.build(:provider => omni['provider'],
:uid => omni['uid'],
:token => omni['credentials']['token'],
:token_secret => omni['credentials']['secret']
else
self.authentications.build(:provider => omni['provider'],
:uid => omni['uid'],
:token => omni['credentials']['token']
end
end
end
UPDATE:
I'm trying it this way. Does this accomplish the same as the above?
self.authentications.build.tap do |auth|
auth[:provider] = omni['provider'] if omni['provider']
auth[:uid] = omni['uid'] if omni['uid']
auth[:token] = omni['credentials']['token'] if omni['credentials']['token']
auth[:token_secret] = omni['credentials']['secret'] if omni['credentials']['secret']
end
I think you could simply do (self is obsolete here):
authentications.build(:provider => omni['provider'],
:uid => omni['uid'],
:token => omni['credentials']['token'],
:token_secret => omni['credentials']['secret'])
If any key is missing, this will simply assign nil value. Unless you have some custom logic for setting those attributes, there is no difference between assigning nil or not assigning anything for a new records.

Rho Mobile: Parse and create a model using JSON

I am able to parse a JSON using the following code
$httpresult = #params['body']
$jsonresult = Rho::JSON.parse($httpresult)
But I don't know how to create a model from $jsonresult.
First, using app_info you can print the result coming from the server to check if the response is valid JSON string.
Second, i think you must decode the url in order to parse it by using:
Rho::JSON.parse(Rho::RhoSupport.url_decode(#params['body']))
Once you've the data in json_result, you can put them in a pre-existing Model.
Supposing that you've already created a model with the name "Product", you can use transactions to speed up the process.
At the beginning of your module you've to require the model name:
require_source 'Product'
Then you can do this callback:
def get_callback
if #params['status'] == "ok"
json_result = Rho::JSON.parse(#params['body'])
db = ::Rho::RHO.get_src_db('Product')
db.start_transaction
Product.delete_all
begin
json_result.each do |item|
Product.create({:Brand => item["B rand"], :Name => item["Name"], :SKU => d["SKU"]})
end
db.commit
rescue Exception => e
trace_msg = e.backtrace.join("\n")
puts 'Application initialize failed: ' + e.inspect + ";Trace: #{trace_msg}"
db.rollback
end
WebView.navigate Rho::RhoConfig.start_path
else
WebView.navigate url_for :action => :show_error
end
end

Chef Recipes - Setting node attributes in ruby_block

I have a Chef recipe for a multi-node web service, each node of which needs to get the hostname and IP of the other nodes, to put it into its own local configuration.
The code is shown below. The problem is that when the node.set[][] assignments are made in the ruby_block as shown, the values are empty when the template that relies upon them is created. If I want to create that template, I have to move all of the ruby_block code outside, and have it "loose" in the recipe. Which makes it harder to do unit-testing with Chefspec and the like.
Can any Chef guru set me straight? Is it just impossible to do node.set[] like this inside of a ruby_block? And if so, why doesn't it say so in the docs?
$cm = { :name => "web", :hostname => "" , :ip_addr => "" }
$ca = { :name => "data", :hostname => "" , :ip_addr => "" }
$cg = { :name => "gateway", :hostname => "" , :ip_addr => "" }
$component_list = [$cm, $ca, $cg]
ruby_block "get host addresses" do
block do
for cmpnt in $component_list
# do REST calls to external service to get cmpnt.hostname, ip_addr
# .......
node.set[cmpnt.name]['name'] = cmpnt.name
node.set[cmpnt.name]['host'] = cmpnt.hostname
node.set[cmpnt.name]['ip'] = cmpnt.ip_addr
end
end
end
template "/etc/app/configuration/config.xml" do
source "config.xml.erb"
variables( :dataHost => node['data']['host'],
:webHost => node['web']['host'],
:gatewayHost => node['gateway']['host'] )
action :create
end
I also added
subscribes :create, "ruby_block[get host addresses]", :immediately
to the template definition to ensure that the ruby_block ran before the template was created. This didn't make a difference.
I realize this is an old post, however for future reference, I just ran across this gist which gives a nice example of node variable assignments in the Compile vs. Converge phases. To adapt the gist to your example, you'll need to add code like the following to your ruby_block:
template_r = run_context.resource_collection.find(:template => "/etc/app/configuration/config.xml")
template_r.content node['data']['host']
template_r.content node['web']['host']
template_r.content node['gateway']['host']
For Chef 11, also see Lazy Attribute Evaluation.
The problem seems to be that attribute values inside your template resource definition get evaluated before actually invoking any resources.
I.e. the file is first executed as simple Ruby, compiling the resources, and only the the resource actions gets invoked. By that time, it is too late already.
I ran into the same problem when trying to encapsulate certain attribute manipulations into a resource. It simply does not work. Should anyone know a solution to this problem, I would appreciate it very much.
EDIT:
b = ruby_block...
...
end
b.run_action(:create)
Could possibly do the trick. It invokes the resource immediately.
The simplest answer to this is to not use chef attributes and not use ruby_block to do the work of talking to the REST API. The code can also be moved to a custom resource for better reuse:
unified_mode true
provides :my_resource
action :run do
cm = { :name => "web", :hostname => "" , :ip_addr => "" }
ca = { :name => "data", :hostname => "" , :ip_addr => "" }
cg = { :name => "gateway", :hostname => "" , :ip_addr => "" }
component_list = [cm, ca, cg]
hash = {}
for cmpnt in component_list
# do REST calls to external service to get cmpnt.hostname, ip_addr
# .......
hash[cmpnt.name] = {}
hash[cmpnt.name]['name'] = cmpnt.name
hash[cmpnt.name]['host'] = cmpnt.hostname
hash[cmpnt.name]['ip'] = cmpnt.ip_addr
end
template "/etc/app/configuration/config.xml" do
source "config.xml.erb"
variables( :dataHost => hash['data']['host'],
:webHost => hash['web']['host'],
:gatewayHost => hash['gateway']['host'] )
action :create
end
end
By using unified_mode and moving into a custom resource, it also makes it easier to use a node attribute without requiring the use of lazy {} or ruby_blocks. It also still allows chef configuration (like setting up resolv.conf or other network requirements before doing the REST calls) prior to calling this code while not having to think about compile/converge two pass issues in recipe context.
There is also no reason to use a resource like ruby_block to do pure ruby processing which does not change the system under management. In this case the ruby_block is hitting a REST service purely to collect data. That does not need to be placed into a Chef resource. It isn't clear from the question if that was being done because the questioner though it was a "best practice" (in this case it is not), or if it was being done to move execution to compile time in order to allow other chef resources that aren't part of the question to fire first (in which case using a custom resource is a much better solution than using a ruby_block).
It's been a while since this question, but in case someone is still looking for it, lazy evaluate is your friend:
template '/tmp/sql_file.sql' do
source "sql_file.sql.erb"
mode 0700
variables lazy {
# Create a new instance of MySQL library
mysql_lib = Acx::MySQL.new(
'127.0.0.1', 'root', node['mysql']['service']['pass']
)
password = node['mysql']['service']['support_admin']['ct_password']
# It returns the encrypted password after evaluate it, to
# be used in template variables
{ admin_password: mysql_lib.encrypted_password(password) }
}
end
https://docs.chef.io/resource_common.html#lazy-evaluation

Resources