In Ruby, how to upload multiple files in single request using RESTClient - ruby

I have to upload multiple files as form request. I am using the Rest Client to post my request. I am able to upload single file but I am not sure how to add multiple files in a single request.
I searched/googled for such option and I am not finding any solution that solves my problem.
Below is my code.
It has variable argument (*yamlfile) which takes one or more files. I have to upload all the files together.
The issue now is , I am getting syntax error when I add the loop to extract the file within the payload.
my assumption is now to form this outside the payload and include it inside the payload block but I am not sure how to do it.
Can someone help me with that.
( I have tried net/http/post/multipart library too and I don't find much documents around it)
def uploadRest(endpoint,archive_file_path,,yaml_file_path,*yamlfile)
$arg_len=yamlfile.length
request = RestClient::Request.new(
:method => :post,
:url => endpoint,
:payload => {
:multipart => true,
:job_upload_archive => File.new(archive_file_path,'rb'),
:job_upload_path => "/tmp",
# Trying to add multiple file, but I get syntax error
yamlfile.each_with_index { |yaml, index|
:job_upload_yaml_file+index => File.new("#{yaml_file_path}/#{pmml}")
}
})
response=request.execute
puts response.code
end

uploadRest(endpoint,archive_file_path,yaml_file_path,*yamlfile)
#files=Array.new
yamlfile.each{ |yaml_file|
#files.push(File.new("#{yaml_file_path}/#{yaml_file}"))
}
request = RestClient::Request.new(
:method => :post,
:url => endpoint,
:payload => { :multipart => true, :job_upload_archive => File.new(archive_file_path,'rb'),
:job_upload_path => "/tmp", :job_upload_yaml_file => #files })
response=request.execute
end

I had a similar problem and was able to get this to work by passing an array of arrays as a requests.
file1 = File.new("#{yaml_file_path}/#{yaml_file1}", 'rb')
file2 = File.new("#{yaml_file_path}/#{yaml_file}", 'rb')
request_body = [["files", file1], ["files", file2]]
RestClient.post url, request_body, request_headers

There were two issues with your question code:
1) Attempt to add a symbol to an integer
2) Attempt to insert contents of yamlfile direct into the hash (because that is what yamlfile.each_with_index returns, as opposed to how it calls your block. The return value from the block is not used)
Both of these code issues read as if you have gained experience in HAML or another templating language, and are using structures/ideas that would work in that?
There are lots of possble solutions in Ruby, but a simple approach to build up the hash in parts, as opposed to generate it in one go with clever hash-returning routines embedded. Try something like this:
payload_hash = {
:multipart => true,
:job_upload_archive => File.new(archive_file_path,'rb'),
:job_upload_path => "/tmp",
}
# This does not use the return value from each_with_index, instead it relies
# on the block to make changes to the hash by adding new key/value pairs
yamlfile.each_with_index { |yaml, index|
# This concatenates two strings, and then converts the combined
# string into the symbol that you want
file_key = ("job_upload_yaml_file"+index.to_s).to_sym
payload_hash[file_key] = File.new("#{yaml_file_path}/#{yaml}")
}
request = RestClient::Request.new(
:method => :post,
:url => endpoint,
:payload => payload_hash
)
For added code cleanliness, you could make the first two parts a separate method, and call it where it currently has payload_hash.
This should get you over current syntax hurdles. However, I have made no attempt to check whether this will allow you to upload multiple files via RESTClient.

Section1:
#params = {
"FacialImage" => UploadIO.new(File.new('C:\temp\ATDD\Test\test\sachin.jpg'), "image/jpeg"),
"Login" => UploadIO.new(File.new('C:\temp\ATDD\Test\test\login.txt'), "application/json")
}

Related

How can I download multiple .xlsx files using axlsx gem?

Hi I'm having trouble downloading multiple files with axlsx. The problem is I'm sending an array of Id's to the controller and asking it to download the report using the render command. It raises an AbstractController::DoubleRenderError. I was thinking of overriding the error but realized it's a bad idea, I don't know what else to do... Any suggestions? Thanks.
My controller code looks like this:
def download_report
params[:user_id].each do |user_id|
#report = Report.find_by(:user_id => user_id)
render :xlsx => "download_report", :filename => "#{#report.user.last_name}.xlsx"
end
end
My axlsx template:
wb = xlsx_package.workbook
wb.add_worksheet(name: "Reports") do |sheet|
wb.styles do |s|
# template code
end
end
It is the built in expectation of Rails that you would call render once per request. And, the browser is going to expect one response per request. So, you are going to have to do something else!
You can use render_to_string, and combine the results into a zip file, serving that. See the bottom of this response.
Or, you could create a single spreadsheet and have each user's report show up on their own worksheet.
Or, on the client side, you could use javascript to request each spreadsheet and download each one separately.
The zip one would be something like this code, which uses render_to_string, rubyzip, and send_data:
def download_report
compressed_filestream = Zip::ZipOutputStream.write_buffer do |zos|
params[:user_id].each do |user_id|
#report = Report.find_by(:user_id => user_id)
content = render_to_string :xlsx => "download_report", :filename => "#{#report.user.last_name}.xlsx"
zos.put_next_entry("user_#{user_id}.xlsx")
zos.print content
end
end
compressed_filestream.rewind
send_data compressed_filestream.read, :filename => 'download_report.zip', :type => "application/zip"
end
Axlsx requires rubyzip, so you should have it already. And you probably want to lookup each user and use their name for the spreadsheet, unless you have it otherwise.

DELETE method with a payload using Ruby Rest:Client?

While I don't think it is very restful to have to include a payload in a DELETE request. I ran into an instance where I am testing a service that requires a payload for DELETE. Might there be a way using Ruby's Rest Client to accomplish this? Unfortunately, I am having a hard time with this one.
#json_request = '{"user_id": 5, "meta_data": "foo"}'
resource = RestClient::Resource.new "http://www.foo.com/some/process"
#response_update = resource.delete(#json_request, :content_type => :json, :accept => :json)
Output:
ArgumentError:
wrong number of arguments (2 for 0..1)
Try this
RestClient::Request.execute(:method => 'delete', :url => "http://www.foo.com", :payload => json_data)
Currently it's not possible with that gem. You can see a PL addressing that. Maybe you could fork it and pull those changes to your own fork of the rest-client gem.
The pull request https://github.com/rest-client/rest-client/pull/98
As a very modern update, from the ReadMe
RestClient::Request.execute(method: :delete, url: 'http://example.com/resource',
payload: 'foo', headers: {myheader: 'bar'})

Chef Recipes - Setting node attributes in ruby_block

I have a Chef recipe for a multi-node web service, each node of which needs to get the hostname and IP of the other nodes, to put it into its own local configuration.
The code is shown below. The problem is that when the node.set[][] assignments are made in the ruby_block as shown, the values are empty when the template that relies upon them is created. If I want to create that template, I have to move all of the ruby_block code outside, and have it "loose" in the recipe. Which makes it harder to do unit-testing with Chefspec and the like.
Can any Chef guru set me straight? Is it just impossible to do node.set[] like this inside of a ruby_block? And if so, why doesn't it say so in the docs?
$cm = { :name => "web", :hostname => "" , :ip_addr => "" }
$ca = { :name => "data", :hostname => "" , :ip_addr => "" }
$cg = { :name => "gateway", :hostname => "" , :ip_addr => "" }
$component_list = [$cm, $ca, $cg]
ruby_block "get host addresses" do
block do
for cmpnt in $component_list
# do REST calls to external service to get cmpnt.hostname, ip_addr
# .......
node.set[cmpnt.name]['name'] = cmpnt.name
node.set[cmpnt.name]['host'] = cmpnt.hostname
node.set[cmpnt.name]['ip'] = cmpnt.ip_addr
end
end
end
template "/etc/app/configuration/config.xml" do
source "config.xml.erb"
variables( :dataHost => node['data']['host'],
:webHost => node['web']['host'],
:gatewayHost => node['gateway']['host'] )
action :create
end
I also added
subscribes :create, "ruby_block[get host addresses]", :immediately
to the template definition to ensure that the ruby_block ran before the template was created. This didn't make a difference.
I realize this is an old post, however for future reference, I just ran across this gist which gives a nice example of node variable assignments in the Compile vs. Converge phases. To adapt the gist to your example, you'll need to add code like the following to your ruby_block:
template_r = run_context.resource_collection.find(:template => "/etc/app/configuration/config.xml")
template_r.content node['data']['host']
template_r.content node['web']['host']
template_r.content node['gateway']['host']
For Chef 11, also see Lazy Attribute Evaluation.
The problem seems to be that attribute values inside your template resource definition get evaluated before actually invoking any resources.
I.e. the file is first executed as simple Ruby, compiling the resources, and only the the resource actions gets invoked. By that time, it is too late already.
I ran into the same problem when trying to encapsulate certain attribute manipulations into a resource. It simply does not work. Should anyone know a solution to this problem, I would appreciate it very much.
EDIT:
b = ruby_block...
...
end
b.run_action(:create)
Could possibly do the trick. It invokes the resource immediately.
The simplest answer to this is to not use chef attributes and not use ruby_block to do the work of talking to the REST API. The code can also be moved to a custom resource for better reuse:
unified_mode true
provides :my_resource
action :run do
cm = { :name => "web", :hostname => "" , :ip_addr => "" }
ca = { :name => "data", :hostname => "" , :ip_addr => "" }
cg = { :name => "gateway", :hostname => "" , :ip_addr => "" }
component_list = [cm, ca, cg]
hash = {}
for cmpnt in component_list
# do REST calls to external service to get cmpnt.hostname, ip_addr
# .......
hash[cmpnt.name] = {}
hash[cmpnt.name]['name'] = cmpnt.name
hash[cmpnt.name]['host'] = cmpnt.hostname
hash[cmpnt.name]['ip'] = cmpnt.ip_addr
end
template "/etc/app/configuration/config.xml" do
source "config.xml.erb"
variables( :dataHost => hash['data']['host'],
:webHost => hash['web']['host'],
:gatewayHost => hash['gateway']['host'] )
action :create
end
end
By using unified_mode and moving into a custom resource, it also makes it easier to use a node attribute without requiring the use of lazy {} or ruby_blocks. It also still allows chef configuration (like setting up resolv.conf or other network requirements before doing the REST calls) prior to calling this code while not having to think about compile/converge two pass issues in recipe context.
There is also no reason to use a resource like ruby_block to do pure ruby processing which does not change the system under management. In this case the ruby_block is hitting a REST service purely to collect data. That does not need to be placed into a Chef resource. It isn't clear from the question if that was being done because the questioner though it was a "best practice" (in this case it is not), or if it was being done to move execution to compile time in order to allow other chef resources that aren't part of the question to fire first (in which case using a custom resource is a much better solution than using a ruby_block).
It's been a while since this question, but in case someone is still looking for it, lazy evaluate is your friend:
template '/tmp/sql_file.sql' do
source "sql_file.sql.erb"
mode 0700
variables lazy {
# Create a new instance of MySQL library
mysql_lib = Acx::MySQL.new(
'127.0.0.1', 'root', node['mysql']['service']['pass']
)
password = node['mysql']['service']['support_admin']['ct_password']
# It returns the encrypted password after evaluate it, to
# be used in template variables
{ admin_password: mysql_lib.encrypted_password(password) }
}
end
https://docs.chef.io/resource_common.html#lazy-evaluation

Get values from server response in ruby

I send request to server and server returns me response. If I print this response, it looks exactly as mentioned below (with array and braces). I'm new to Ruby so I have two questions:
1. To what structure should I add this response?
2. How to I get values from this response (eg value of user_id or user_status). How to get rid of quotes in value
Request code:
def userGet(user_id_or_email)
uri = URI(SRV + '/userGet')
http = Net::HTTP.new(uri.host,uri.port)
req = Net::HTTP::Post.new(uri.path)
req['bla-bla'] = 'bla-bla'
req.set_form_data('search' => user_id_or_email)
res = http.request(req)
puts(res.read_body)
end
Output of puts(res)
array (
'user_id' => 301877459,
'login' => '0301877459',
'email' => 'YS5raG96eWFfdHZhc2lsaWlAY29ycC5iYWRvby5jb20=',
'passwd' => 'cc03e747a6afbbcbf8be7668acfebee5',
'partner_id' => '105',
'user_status' => 'active',
'nickname' => 'Test',
'fullname' => 'Test',
)
As other commentors have mentioned, the first step is to determine the encoding of the response. If you can easily change the way that the data is returned by the server, you could output valid JSON and use a gem such as this. If you cannot, then an ad-hoc method for parsing responses of this type would be to define a function like this:
def parseResult(res)
# Remove the array wrapper and any leading/trailing whitespace
parsed_string = res.gsub(/^\s*array\s*\(/, "").gsub(/[\s,]*\)[\s,]*$/, "")
# Split the string into an array of key-value tuples
parsed_array = parsed_string.split(',').collect do |tuple|
tuple.split("=>").collect do |x|
x.match(/^[\s',]*([^',]*)[\s',]*$/)[1]
end
end
# Convert the array of tuples into a hash for easy access
Hash[parsed_array]
end
This is similar sawa's method, but it assumes that you cannot trust the data being returned by the server and therefore cannot use eval safely.
Not sure what that array ( ... ) means, but assuming it means a hash, you can do:
string.eval(
string
.sub(/\A\s*array\s*\(/, "{")
.sub(/\)\s*\z/, "}")
)

Ruby RestClient converts XML to Hash

I need to send a POST request as an XML string but I get odd results. The code:
require 'rest_client'
response = RestClient.post "http://127.0.0.1:2000", "<tag1>text</tag1>", :content_type => "text/xml"
I expect to receive "<tag1>text</tag1>" as the parameter on the request server. Instead, I get "tag1"=>"text". It converts the XML to a hash. Why is that? Any way around this?
Try this:
response = RestClient.post "http://127.0.0.1:2000",
"<tag1>text</tag1>",
{:accept => :xml, :content_type => :xml}
I think you just needed to specify the ":accept" to let it know you wanted to receive it in the XML format. Assuming it's your own server, you can debug on the server and see the request format used is probably html.
Hope that helps.
Instead of using RestClient, use Ruby's built-in Open::URI for GET requests or something like Net::HTTP or the incredibly powerful Typhoeus:
uri = URI('http://www.example.com/search.cgi')
res = Net::HTTP.post_form(uri, 'q' => 'ruby', 'max' => '50')
In Typhoeus, you'd use:
res = Typhoeus::Request.post(
'http://localhost:3000/posts',
:params => {
:title => 'test post',
:content => 'this is my test'
}
)
Your resulting page, if it's in XML will be easy to parse using Nokogiri:
doc = Nokogiri::XML(res.body)
At that point you'll have a fully parsed DOM, ready to be searched, using Nokogiri's search methods, such as search and at, or any of their related methods.

Resources