I have designed code in ruby which converts my xls to json partially through spreadsheet and code. I need to upload this json to my data bag on chef-server. I am using knife commands from ruby code and running it. The json files upload to my local chef-repo correctly, but for transferring to chef-server I am getting this error:
ERROR: Chef::Exceptions::ValidationFailed: Data Bag Items must contain a Hash or Mash!
I have validated the json, id matches name of file and tried using [] braces at start and end but doesn't work. This is start of my json:
{
"id": "default_1",
"Sr.No" : "1", ....}
My ruby code essentials look like :
require 'spreadsheet'
book = Spreadsheet.open('BI.xls')
sheet1 = book.worksheet('Sheet1')
.
.
.
cmd1 = "cd #{current_dir}/chef-repo"
cmd2 = "knife data_bag create TestDB" #tried knife data bag too
cmd4 = "knife data_bag from file TestDB default_1.json" #tried knife data bag too
upload = %x[#{cmd1} && #{cmd2} && #{cmd4} ]
puts "#{upload}"
The command knife node list shows nodes correctly. I am new to chef and ruby , searched and tried many things but not working.
It surprisingly got sorted out! I stored all docs in chef-repo, and separated one file into two, the first to convert .xls to json , and the other to execute knife commands . I now run both ruby scripts from chef-repo and the work just as expected. Phew!
Related
New to Chef, bare wi/ me. Building my cookbook.
How did you set the databag?
are you able to get the credential out using :
$ knife vault show nameOfVault nameOfItem
or
$ knife data bag show nameOfVault nameOfItem_keys
Glad you have your data bag loading and indeed, with Test Kitchen you do not need to use knife to upload to Chef Server, as Test Kitchen uses Chef Zero / Solo.
The issue you have here is that you have not correctly formatted reading from the data bag object once you have read. You need to do this instead:
ruby_block "insert_line" do
block do
file = Chef::Util::FileEdit.new('/var/lib/net-snmp/snmpd.conf')
file.insert_line_if_no_match("/www.example.com/", "createUser
#{snmp3usercreds['user']}
SHA #{snmp3usercreds['auth_pssword']}
AES #{snmp3usercreds['enc_password']} ")
file.write_file
end
end
So, you will see I have changed snmp3usercreds[user] to snmp3usercreds['user'] with quotes around the user to show it is a string (rather than a variable as is the case with your code).
I want to get the recipes that a cookbook contains, through chef-server-api. Following is the code I'm using for getting the cookbook list, individual cookbook details through the api :
require 'rubygems'
require 'chef/config'
require 'chef/log'
require 'chef/rest'
require 'chef/cookbook_version'
client_name = "admin"
signing_key_filename="c:/chef-repo/.chef/admin.pem"
server_url = "https://10.132.17.244:443"
rest = Chef::REST.new(server_url, client_name, signing_key_filename)
cookbooks = rest.get_rest("/cookbooks?all_versions")
cookbooks.keys.each do |name|
cookbook_versions = rest.get_rest("/cookbooks/#{name}")
print "#{name}\n"
cookbook_versions[name]["versions"].each do |cv|
version = cv["version"]
cookbook = rest.get_rest("/cookbooks/#{name}/#{version}")
print "\t#{cookbook}\n"
#parsed = JSON[cookbook]
end
end
The problem I'm facing is to get the recipe list from the 'cookbook' object. I tried parsing it to ruby hash and then read, but of no use. If I directly print the 'cookbook' variable, the output is something like the screenshot
I'm not able to get how to interpret the output I am getting by hitting the '/cookbooks/NAMEW/VERSION' endpoint, and get the recipes present in an individual cookbooks.
When using the Chef gem it automatically decodes some responses into Ruby objects for you. You can either use the object directly (specifically you want #recipe_filenames and then parse those to the cookbook_name::recipe_name format) or you could use a better API client like Chef-API or PyChef.
Need a ruby solution? The following example uses jq to filter the JSON resultset returned by knife:
$ knife cookbook show apache2 2.0.0 recipes -Fj | jq '.[]|.name'
"mod_cgi.rb"
"mod_proxy_http.rb"
"mod_proxy_html.rb"
"mod_access_compat.rb"
"mod_authz_dbd.rb"
"mod_proxy_express.rb"
..
..
require 'net/ftp'
require 'nokogiri'
server = "xxxxxx"
user = "xxxxx"
password = "xxxxx"
ftp = Net::FTP.new(server, user, password)
files = ftp.nlst('File*.xml')
files.each do |file|
ftp.getbinaryfile(file)
doc = Nokogiri::XML(open(file))
# some operations with doc
end
With the code above I'm able to parse/read XML file, because it first downloads a file.
But how can I parse remote XML file without downloading it?
The code above is a part of rake task that loads rails environment when run.
UPDATE:
I'm not going to create any file. I will import info into the mongodb using mongoid.
If you simply want to avoid using a temporary local file, it is possible to to fetch the file contents direct as a String, and process in memory, by supplying nil as the local file name:
files.each do |file|
xml_string = ftp.getbinaryfile( file, nil )
doc = Nokogiri::XML( xml_string )
# some operations with doc
end
This still does an FTP fetch of the contents, and XML parsing happens at the client.
It is not really possible to avoid fetching the data in some form or other, and if FTP is the only protocol you have available, then that means copying data over the network using an FTP get. However, it is possible, but far more complicated, to add capabilities to your FTP (or other net-based) server, and return the data in some other form. That could include Nokogiri parsing done remotely on the server, but you'd still need to serialise the end result, fetch it and deserialise it.
I am writing an application in Ruby on Elastic Beanstalk in which I download a file from a remote server and write it to an object in a bucket.
require 'open-uri'
...
s3 = AWS::S3.new
bucket = s3.buckets['mybucket']
f = open(params[:url]) #using open-uri
obj = bucket.objects[params[:key]]
obj.write[f] #<< fails here
The last line, however, fails with the following exception in the log:
:data must be provided as a String, Pathname, File, or an object that responds to #read and #eof?
I know, however, from executing the same #open on my machine, that f is a StringIO object, which does have #read and #eof?.
I was getting same error during zip file upload on S3 and finally this worked for me:
zip_data = File.read(zip_file_path)
means, zip_data will be the object at the zip file path that is located in your tmp directory.
Hope, this will work for you also.
I am trying to work with two worksheets at the same time.
So I have code
require 'parseexcel'
#Open the excel file passed in from the commandline
workbook = Spreadsheet::ParseExcel.parse(ARGV[0])
workbook2 = Spreadsheet::ParseExcel.parse(ARGV[1])
#Get the first worksheet
worksheet = workbook.worksheet(0)
worksheet2 = workbook2.worksheet(0)
However, when I run this code I get an error: array is not implemented
This error goes away when I comment out line:
workbook2 = Spreadsheet::ParseExcel.parse(ARGV[1])
Why is this happeneing?
Way I am running script is: ruby -rubygems traverse.rb excel.xls so.xls
i fixed it by copy pasting so.xls in excel.xls as a different workbook. then just accessed it by workbook.worksheet(1) that worked