I have an array that store some data like this and this is the exact input I'm using
{
"accepted" => {
"number_of_order" => {
"condition" => "between_number", "value" => "0&5"
}
}, "removed" => {
}
}
Now I'm trying to convert this array into ruby hash access like this and this is my expected output it's not what I'm getting now
--- !ruby/hash:ActiveSupport::HashWithIndifferentAccess
accepted: !ruby/hash:ActiveSupport::HashWithIndifferentAccess
number_of_order: !ruby/hash:ActiveSupport::HashWithIndifferentAccess
condition: between_number
value: 0&5
removed: !ruby/hash:ActiveSupport::HashWithIndifferentAccess {}
But I'm unable to convert the array in ruby hash access. I also try YAML dump Yaml::dump($array); to convert into YAML then ruby hash access but nothing works.
Basically, I'm trying to do reverse of this question
How to get data from ruby hash access to laravel 8?
You can refer above link I convert this ruby hash into a laravel array but now I also wanted to convert this array into ruby hash.
My current code for converting an array to a ruby hash
$jsonEncoded = '{"accepted":{"number_of_order":{"consition":"between_number","value":"0&5"}}}';
$array = json_decode($jsonEncoded, true);
dump(Yaml::dump($array));
My current code for converting ruby hash to array
$filter_data = str_replace("!ruby/hash:ActiveSupport::HashWithIndifferentAccess", "", $filter_data);
$yamlContents = Yaml::parse($filter_data);
To convert hash to yaml in ruby we can use yaml library
require 'yaml'
data = {
"accepted" => {
"number_of_order" => {
"condition" => "between_number", "value" => "0&5"
}
}, "removed" => {
}
}
puts data.to_yaml
To convert associated array to yaml in php we can use Symfony Yaml
use Symfony\Component\Yaml\Yaml;
$jsonEncoded = '{"accepted":{"number_of_order":{"consition":"between_number","value":"0&5"}}}';
$array = json_decode($jsonEncoded, true);
print Yaml::dump($array);
To load php output yaml in ruby same yaml library can be used
require 'yaml'
yaml_str = ... # output generate from Symfony Yaml::dump
yaml_hash = YAML.load(yaml_str)
To load ruby output yaml in php we can use the same Symfony Yaml
$yaml_str = ...; # output generate from ruby rhash.to_yaml
$yaml_associated_array = Yaml.parse(yaml_str);
Load ruby hash to php
$data = "
--- !ruby/hash:ActiveSupport::HashWithIndifferentAccess
accepted: !ruby/hash:ActiveSupport::HashWithIndifferentAccess
number_of_order: !ruby/hash:ActiveSupport::HashWithIndifferentAccess
condition: between_number
value: 0&5
removed: !ruby/hash:ActiveSupport::HashWithIndifferentAccess {}
";
print $data;
$data = str_replace("---", "", preg_replace("/!ruby.*Access/", "", $data));
print $data;
$result = Yaml::parse($data);
var_dump($result);
Same can be achieved in ruby
data = %Q(
--- !ruby/hash:ActiveSupport::HashWithIndifferentAccess
accepted: !ruby/hash:ActiveSupport::HashWithIndifferentAccess
number_of_order: !ruby/hash:ActiveSupport::HashWithIndifferentAccess
condition: between_number
value: 0&5
removed: !ruby/hash:ActiveSupport::HashWithIndifferentAccess {}
)
result = str.gsub(/!ruby.*Access/, '').gsub("--- ", "")
puts result
puts YAML.load(result)
Note: We can also use other libraries
I just found the answer to my queustion and I'm sharing it with you all.
Laravel Code:
$jsonEncoded = '{"accepted":{"first_order_number":{"consition":"between_date","value":"2021-11-28 00:00:00&2021-11-22 00:00:00"}},"removed":{}}';
$process = new Process(['ruby', 'convert_hashActiveSupport.rb'],null,null, $jsonEncoded);
$process->run();
// executes after the command finishes
if (!$process->isSuccessful()) {
throw new ProcessFailedException($process);
}
$data = $process->getOutput();
Ruby Code:
require 'yaml'
require 'json'
require "active_support/core_ext/hash/indifferent_access"
yaml_str = "#{ ARGF.read }"
parsed_output = JSON.parse(yaml_str)
yaml_output = parsed_output.with_indifferent_access.to_yaml
puts yaml_output
Package required in laravel:
composer require symfony/process
Packages required in ruby:
gem install activesupport
gem install 'yaml'
And here I'm getting my expected output:
--- !ruby/hash:ActiveSupport::HashWithIndifferentAccess
accepted: !ruby/hash:ActiveSupport::HashWithIndifferentAccess
first_order_date: !ruby/hash:ActiveSupport::HashWithIndifferentAccess
consition: between_date
value: 2021-11-28 00:00:00&2021-11-22 00:00:00
removed: !ruby/hash:ActiveSupport::HashWithIndifferentAccess {}
Related
This question already has answers here:
How do I push unique copies of array into another array?
(2 answers)
Ruby Arrays: Why is the value I print correct, but the value I push incorrect?
(1 answer)
Closed 4 months ago.
I am making json formater for sonarqube, here is my script:
require 'json'
tr_report = File.open('./tes.json').read
tr_report.gsub!(/\r\n?/, "\n")
sq_generic_format = {'issues' => []}
sq_issue_format = {
'engineId' => '', # CONST='brakeman'
'ruleId' => '', #[check_name (warning_code)] warning_type [confidence]
'severity':'MAJOR', # MAJOR
'type':'VULNERABILITY', # CONST='VULNERABILITY'
'primaryLocation' => {},
'effortMinutes' => 0, #CONST=0
}
primary_location_format = {
'message' => '', # message + CONST='\nCode:' + code + CONST='\nUser Input:' + user_input + CONST='\nLink: ' + link
'filePath' => '', # file
'textRange' => {}
}
text_range_format = {
'startLine' => 1,# line
'endLine' => 1,# line
'startColumn' => 0,
'endColumn' => 1
}
issues = []
tr_report.each_line do |line|
tr_data = JSON.parse(line)
# puts tr_data
# puts parsed["SourceMetadata"]["Data"]["Filesystem"]["file"]
issue = sq_issue_format
issue['engineId'] = 'trufflehog'
issue['ruleId'] = 'Sensitive Data Exposure - %s' %[tr_data['Raw']]
issue['severity'] = 'MAJOR' # MAJOR
issue['type'] = 'VULNERABILITY' # CONST='VULNERABILITY'
issue['effortMinutes'] = 0
issue['primaryLocation'] = {}
# filling up nested data lvl1 ^
primary_location = primary_location_format
primary_location['message'] = 'Sensitive Data Exposure'
primary_location['filePath'] = tr_data["SourceMetadata"]["Data"]["Filesystem"]["file"] # file
primary_location['textRange'] = {}
# filling up nested data lvl2 ^
text_range = text_range_format
# text_range['startLine'] = w['line']
# text_range['endLine'] = w['line']
# sticking all together
primary_location['textRange'] = text_range
issue['primaryLocation'] = primary_location
issues.append(issue)
end
# puts issues
sq_generic_format['issues'] = issues
puts JSON.dump(sq_generic_format)
File.write('./trufflehog-sq-report.json', JSON.dump(sq_generic_format))
and here is my jsonline tes.json:
{"SourceMetadata":{"Data":{"Filesystem":{"file":"../ruby/railsgoat/dependency-check-report.html"}}},"SourceID":15,"SourceType":15,"SourceName":"trufflehog - filesystem","DetectorType":9,"DetectorName":"Gitlab","Verified":false,"Raw":"vulnerable-to-driveby-","Redacted":"","ExtraData":null,"StructuredData":null}
{"SourceMetadata":{"Data":{"Filesystem":{"file":"../ruby/railsgoat/dependency-check-report.html"}}},"SourceID":15,"SourceType":15,"SourceName":"trufflehog - filesystem","DetectorType":800,"DetectorName":"Atera","Verified":false,"Raw":"39a6bda16ef9583fba2696cc3efde0da","Redacted":"","ExtraData":null,"StructuredData":null}
But everytime I try run it, I always got the first line of the parse, I cant get the next line and make the result redundace. how to capture my parse for the next line? and not just the first line.
Also i make a simple script to parse the jsonl, and it successfully like my expected. here is the script:
require 'json'
text=File.open('tes.json').read
text.gsub!(/\r\n?/, "\n")
text.each_line do |line|
parsed = JSON.parse(line)
puts parsed["Raw"]
end
result:
vulnerable-to-driveby-
39a6bda16ef9583fba2696cc3efde0da
The current result: its parse just only the first line,
expected result: I got all of the parse properly.
My expected result for my formatter script:
{"issues":[{"engineId":"trufflehog","ruleId":"Sensitive Data Exposure - vulnerable-to-driveby-","severity":"MAJOR","type":"VULNERABILITY","primaryLocation":{"message":"Sensitive Data Exposure","filePath":"../ruby/railsgoat/dependency-check-report.html","textRange":{"startLine":1,"endLine":1,"startColumn":0,"endColumn":1}},"effortMinutes":0,"severity":"MAJOR","type":"VULNERABILITY"},{"engineId":"trufflehog","ruleId":"Sensitive Data Exposure - 39a6bda16ef9583fba2696cc3efde0da","severity":"MAJOR","type":"VULNERABILITY","primaryLocation":{"message":"Sensitive Data Exposure","filePath":"../ruby/railsgoat/dependency-check-report.html","textRange":{"startLine":1,"endLine":1,"startColumn":0,"endColumn":1}},"effortMinutes":0,"severity":"MAJOR","type":"VULNERABILITY"}]}
and here what i got right now:
{"issues":[{"engineId":"trufflehog","ruleId":"Sensitive Data Exposure - 39a6bda16ef9583fba2696cc3efde0da","severity":"MAJOR","type":"VULNERABILITY","primaryLocation":{"message":"Sensitive Data Exposure","filePath":"../ruby/railsgoat/dependency-check-report.html","textRange":{"startLine":1,"endLine":1,"startColumn":0,"endColumn":1}},"effortMinutes":0,"severity":"MAJOR","type":"VULNERABILITY"},{"engineId":"trufflehog","ruleId":"Sensitive Data Exposure - 39a6bda16ef9583fba2696cc3efde0da","severity":"MAJOR","type":"VULNERABILITY","primaryLocation":{"message":"Sensitive Data Exposure","filePath":"../ruby/railsgoat/dependency-check-report.html","textRange":{"startLine":1,"endLine":1,"startColumn":0,"endColumn":1}},"effortMinutes":0,"severity":"MAJOR","type":"VULNERABILITY"}]}
PS: see the ruleId for the difference.
First, I ran your json through a formatter and it was reported as invalid. If you're going to have multiple objects you should use an array. So I've adjusted it to be: [{...},{...}]. (this is because JSON expects there to only be 1 root eleemnt.)
I think it's easiest to then say, you're doing the work that the JSON.parser is already meant to do. You can iterate through the object directly off the parser: JSON.parse(File.read("/tmp/tes.json")).map{ |obj| obj["Raw"] }
This gives me the results of => ["vulnerable-to-driveby-", "39a6bda16ef9583fba2696cc3efde0da"]
I have defined a custom function currently based on the very simple example here: https://docs.puppet.com/guides/custom_functions.html
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
filename = args[0]
hash_to_be_transformed = args[1]
File.open(filename, 'a') {|fd| fd.puts hash_to_be_transformed }
end
end
This kinda works. I can call it like this:
$my_hash = { key => "value1" , key2 => "value2" }
notify{ "new hash!! $my_hash" :}
transform_service_hash('/var/tmp/blah',$my_hash)
and the file displays:
mgt21 ~ # cat /var/tmp/blah
keyvalue1key2value2
But, if I try to access elements of the hash, nothing changes:
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
filename = args[0]
hash_to_be_transformed = args[1]
element1 = hash_to_be_transformed["key"]
File.open(filename, 'a') {|fd| fd.puts element1 }
end
end
The above block outputs the exact same data to /var/tmp/blah.
And, interestingly, if I remove the filename pass and define it statically in the module:
$my_hash = { key => "value1" , key2 => "value2" }
notify{ "new hash!! $my_hash. element1 is: $my_hash.key" :}
transform_service_hash($my_hash)
and
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
hash_to_be_transformed = args[0]
element1 = hash_to_be_transformed["key"]
File.open('/var/tmp/blah2', 'a') {|fd| fd.puts element1 }
end
end
I get the following error: "Error 400 on SERVER: can't convert Hash into String" with a line reference pointing to "transform_service_hash($my_hash)"
I am new to both puppet and ruby...so I'm unsure I am not passing the element properly, if I am not receiving it properly, or if it something that puppet cannot handle. Please note that I am using version 3.8 of puppet and 1.8.7 of ruby.
Thanks for any help. I've been banging my head against this, and google hasn't been forthcoming yet.
---Edit to clarify my goals (I also edited my code a bit for specificity): I am attempting to pass a hash into a custom ruby function within puppet. The "test" hash has two elements: one string and one array. It is defined as such:
$my_hash = { key => "value1" , key2 => ['array_value1', 'array_value2'] }
$my_display_element=$my_hash["key2"][0]
notify{ "new hash!! $my_hash. the first value of the array stored in element2 is: $my_display_element" :}
transform_service_hash('/var/tmp/blah',$my_hash)
The function appears like so:
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
filename = args[0]
hash_to_be_transformed = args[1]
element1 = args[1]["key"]
element2 = args[1]["key2"][0]
#element1 = hash_to_be_transformed["key"]
#element2 = hash_to_be_transformed["key2"][0]
File.open(filename, 'a') {|fd| fd.puts "hash_to_be_transformed: #{hash_to_be_transformed}\n" }
File.open(filename, 'a') {|fd| fd.puts "element1: #{element1}\n" }
File.open(filename, 'a') {|fd| fd.puts "element2: #{element2}\n" }
end
end
For now, I just want to be able to see that I am able to access elements within the passed hash like a hash. So I'd love for the output file to look like:
hash_to_be_transformed: keyvalue1key2array_value1array_value2
element1: value1
element2: array_value1
However, in the output file, I see:
mgt21 ~ # cat /var/tmp/blah
keyvalue1key2array_value1array_value2
Clearly, something is off here as my text is not being added and the full hash is just printed out just once and seemingly in string form.
I believe that this may be related to the error that I get when I don't pass in a file name (see above). I think that my hash is getting interpreted (or passed) as a string and, as such, I am unable to access the elements. Unfortunately, I still have been unable to verify this or figure out why it might be happening.
---Edit2 based on Matt's answer below.
I decided to simplify my code to isolate this "can't convert Hash into String error". I also made his suggested changes to remove the ambiguity from my key declarations.
$my_hash = { 'key' => "value1" , 'key2' => ['array_value1', 'array_value2'] }
$my_display_element=$my_hash["key2"][0]
notify{ "new hash!! $my_hash. the first value of the array stored in element2 is: $my_display_element" :}
transform_service_hash($my_hash)
and
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
hash_to_be_transformed = args[0]
element1 = args[0]['key']
element2 = args[0]['key2'][0]
File.open('/var/tmp/blah', 'a') {|fd| fd.puts "hash_to_be_transformed: #{hash_to_be_transformed}\n" }
File.open('/var/tmp/blah', 'a') {|fd| fd.puts "element1: #{element1}\n" }
File.open('/var/tmp/blah', 'a') {|fd| fd.puts "element2: #{element2}\n" }
end
end
But, I still end up with the same "Hash to String error". It is worth noting that I also tried simplifying my hash to:
$my_hash = { 'key' => "value1" , 'key2' => "value2" }
and I still get the "Hash to String error".
I quickly took your custom parser function and converted it into pure ruby like the following:
hash = { 'key' => 'value1', 'key2' => %w(array_value1 array_value2) }
def newfunction(filename, a_hash)
element1 = a_hash['key']
element2 = a_hash['key2'][0]
File.open(filename, 'a') do |fd|
fd.puts "hash_to_be_transformed: #{a_hash}"
fd.puts "element1: #{element1}"
fd.puts "element2: #{element2}"
end
end
newfunction('foo.txt', hash)
This results in the output text file like the following:
hash_to_be_transformed: {"key"=>"value1", "key2"=>["array_value1", "array_value2"]}
element1: value1
element2: array_value1
This seems to confirm my initial suspicion about what is going wrong here. Your hash in Puppet of:
$my_hash = { key => "value1" , key2 => ['array_value1', 'array_value2'] }
has keys of implicit/ambiguous types. In the ruby code I used to test, I explicitly established them as strings. This also correlates strongly with these lines in your code failing:
element1 = args[1]["key"]
element2 = args[1]["key2"][0]
and your error message of:
Error 400 on SERVER: can't convert Hash into String
because you are specifying in your ruby code that you expect the keys to be string. Changing your hash in Puppet to:
$my_hash = { 'key' => "value1" , 'key2' => "value2" }
should fix this.
On an unrelated note, I recommend the use of linters to help you learn these languages. Puppet-Lint, Rubocop, and Reek will all help point out suboptimal and messy parts of your code to help you learn the new languages.
On a related note, you may want to put something like this at the top of your custom parser function:
raise(Puppet::ParseError, 'newfunction expects two arguments') if args.length != 2
After much gnashing of teeth (and some very helpful pointers from #MattSchuchard), I realized that none of the changes to my function were going into effect. One needs to restart the puppetmaster service after each change to a custom function: docs.puppet.com/guides/custom_functions.html (appropriately under "Gotchas").
Once I started restarting this service after each change to the function, my hash was able to be parsed properly:
from the .pp file:
$filename = "/var/tmp/test"
$my_hash = { 'key' => "value1" , 'key2' => ["M\'lady\n*doffs cap*", 'array_value2'] }
transform_service_hash($filename, $my_hash)
from the ruby file:
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
filename = args[0]
hash_to_be_transformed = args[1]
array_val = hash_to_be_transformed['key2'][0]
File.open(filename, 'a') {|fd| fd.puts "#{array_val}\n" }
end
end
and output:
mgt21 tmp # cat test
M'lady
*doffs cap*
I am making a JSON call to a Ruby on Rails server via a client side ruby script, which returns the following JSON:
get_data.rb
require 'net/http'
require 'open-uri'
require 'json'
def get_functions(serial_number, function)
request_uri = "http://localhost:3000/devices/#{serial_number}"
buffer = open(request_uri).read
result = JSON.parse(buffer)
puts result
end
{ "serial_number" => "111aaa",
"device_functions" => [
{ "can_scan" => true,
"can_halt" => true
}
],
"host_options" => [
{ "exclude_ip" => "10.10.10.100-110",
"scan_ip" => "10.10.10.1"
}
]
}
Now, I'm wanting to just extract certain values from the response to determine what can/cannot be done on the client side:
scan.rb
if get_functions('111aaa', 'can_scan')
result = %x( ping 10.10.10.1 )
else
result = "Not allowed to perform action!"
end
I'm stuck with how I can extract the value of can_scan from my JSON in get_data.rb for my get_functions method to be able to run its if statement correctly.
Your get_functions is returning nil as the last line is an I/O operation (puts). Change your function to:
def get_functions(serial_number, function)
request_uri = "http://localhost:3000/devices/#{serial_number}"
buffer = open(request_uri).read
result = JSON.parse(buffer)
puts result
result
end
And access the Hash:
result = get_functions(serial_number, function)
result["device_functions"].first["can_scan"]
Code:
#!/usr/bin/ruby
require 'rubygems'
require 'open-uri'
require 'json'
def getData
file = open("http://goo.gl/BI6h7a")
#json = JSON.parse(file.read)
end
getData
cveIds = #json['cve_id']
puts cveIds
You can see the JSON response here: http://goo.gl/BI6h7a
Console:
./cve.rb:13:in `[]': can't convert String into Integer (TypeError) from ./cve.rb:13:in `<main>'
I don't know why this is happening. "Convert String into Integer"? WHAT?
The #json gets the content fine, but the cveIds doesn't.
The top element in the json that you're reading is actually an Array, each of its elements is actually a hash, it's like this:
[
{
"cve_id": "CVE-2014-3976"
// other key/value pairs
}
{
"cve_id": "CVE-2014-3975"
// other key/value pairs
}
{
"cve_id": "CVE-2014-3974"
// other key/value pairs
}
// .... more hashes
]
so #json is an array. And if you want to access any of its elements you have to access it with a numeric integer index like, so:
#json[0] # => { "cve_id": "CVE-2014-3976", // other key/value pairs }
I think you are trying to collect the cve_id fields of all these hashes, this can be done as follows:
cveIds = #json.collect { |h| h["cve_id"] }
# The result:
=> ["CVE-2014-3976", "CVE-2014-3975", "CVE-2014-3974", "CVE-2014-3962", "CVE-2014-3961",
"CVE-2014-3878", "CVE-2014-3871", "CVE-2014-3842", "CVE-2014-3806", "CVE-2014-3792",
"CVE-2014-3791", "CVE-2014-3443", "CVE-2014-3247", "CVE-2014-3246", "CVE-2014-3225",
"CVE-2014-3216", "CVE-2014-3139", "CVE-2014-3138", "CVE-2014-3008", "CVE-2014-2996",
"CVE-2014-2994", "CVE-2014-2976", "CVE-2014-2850", "CVE-2014-2847", "CVE-2014-2671",
"CVE-2014-2668", "CVE-2014-2588", "CVE-2014-2587","CVE-2014-2586", "CVE-2014-2579"]
I'm not a ruby developer but what you have there is a list if dictionaries.
My guess in order for you to read cve_id you need to create some kind of a for loop.
for example in python I would write it like this:
for line in my_data:
print line['cve_id']
I guess in ruby it would look like this:
for i in #json do
cveIds = i['cve_id']
puts cveIds
end
cveIds = #json['cve_id']
What are you doing here is equivalent to:
arr = [1, 2, 3, 4]
puts arr["hello"] # using a string here on an indexed based array!
Hence your error message about Ruby trying to convert a String to an int.
Try the following instead
cveIds = #json.first['cve_id'] # equivalent to #json[0]['cve_id']
puts cveIds
In the above code sample, we are getting the first element from the array, which is a hash we can then access cve_id from.
I have a string stored in a database like so:
images = '[{"id":1,"type":"Image","image_id":"asdf123"},{"id":2,"type":"Image","image_id":"asdf456"},{"id":3,"type":"Image","image_id":"asdf890"}]'
And would like to convert it to an array so I can do something like:
images.each do |image|
puts image.image_id
end
Is it really just a matter of removing the outer square brackets and then following the procedure from this question Converting a Ruby String into an array or is there a more direct/elegant method?
That format is called JavaScript Object Notation (JSON) and can be parsed by a builtin Ruby library:
require 'json'
images_str = '[{"id":1,"type":"Image","image_id":"asdf123"},{"id":2,"type":"Image","image_id":"asdf456"},{"id":3,"type":"Image","image_id":"asdf890"}]'
images = JSON.parse(images_str)
images.size # => 3
images[0].class # => Hash
images[0]['image_id'] # => "asdf123"
images.each { |x| puts "#{x['id']}: #{x['image_id']}" }
# 1: asdf123
# 2: asdf456
# 3: asdf890