Currently I want to create an md5 hash from an argument. Then I want to write the hash into a file (the path is another argument).
That is the custom function:
module Puppet::Parser::Functions
newfunction(:write_line_to_file) do |args|
require 'md5'
filename = args[0]
str = MD5.new(lookupvar(args[1])).to_s
File.open(filename, 'a') {|fd| fd.puts str }
end
end
And the call in the puppet manifest:
write_line_to_file('/tmp/some_hash', "Hello world!")
The result I get is a file and the content is not the hash but the original string. (In the example Hello World!)
I know that this custom function has no practical use. I just want to understand how the md5 hash works.
---UPD---
new Function (it works properly):
require 'digest'
module Puppet::Parser::Functions
newfunction(:lxwrite_line_to_file) do |args|
filename = args[0]
str = Digest::MD5.hexdigest args[1]
File.open(filename, 'w') {|fd| fd.puts str }
end
end
Which ruby you are using?
In Ruby 2.0+ there is a Digest module (documentation here) - why you don't use it instead?.
You can use any hash, available in Digest, like this:
Digest::MD5.digest '123'
=> " ,\xB9b\xACY\a[\x96K\a\x15-#Kp"
or use hexdigest if you prefer hex representation
Digest::MD5.hexdigest '123'
=> "202cb962ac59075b964b07152d234b70"
There are also other hash-functions available there:
Digest::SHA2.hexdigest '123'
=> "a665a45920422f9d417e4867efdc4fb8a04a1f3fff1fa07e998e86f7f7a27ae3"
Related
E.G.
def do_the_thing(file_to_load, hash_path)
file = File.read(file)
data = JSON.parse(file, { symbolize_names: true })
data[sections.to_sym]
end
do_the_thing(file_I_want, '[:foo][:bar][0]')
Tried a few methods but failed so far.
Thanks for any help in advance :)
Assuming you missed the parameters names...
Lets assume our file is:
// test.json
{
"foo": {
"bar": ["foobar"]
}
}
Recomended solution
Does your param really need to be a string??
If your code can be more flexible, and pass arguments as they are on ruby, you can use the Hash dig method:
require 'json'
def do_the_thing(file, *hash_path)
file = File.read(file)
data = JSON.parse(file, symbolize_names: true)
data.dig(*hash_path)
end
do_the_thing('test.json', :foo, :bar, 0)
You should get
"foobar"
It should work fine !!
Read the rest of the answer if that doesn't satisfy your question
Alternative solution (using the same argument)
If you REALLY need to use that argument as string, you can;
Treat your params to adapt to the first solution, it won't be a small or fancy code, but it will work:
require 'json'
BRACKET_REGEX = /(\[[^\[]*\])/.freeze
# Treats the literal string to it's correspondent value
def treat_type(param)
# Remove the remaining brackets from the string
# You could do this step directly on the regex if you want to
param = param[1..-2]
case param[0]
# Checks if it is a string
when '\''
param[1..-2]
# Checks if it is a symbol
when ':'
param[1..-1].to_sym
else
begin
Integer(param)
rescue ArgumentError
param
end
end
end
# Converts your param to the accepted pattern of 'dig' method
def string_to_args(param)
# Scan method will break the match results of the regex into an array
param.scan(BRACKET_REGEX).flatten.map { |match| treat_type(match) }
end
def do_the_thing(file, hash_path)
hash_path = string_to_args(hash_path)
file = File.read(file)
data = JSON.parse(file, symbolize_names: true)
data.dig(*hash_path)
end
so:
do_the_thing('test.json', '[:foo][:bar][0]')
returns
"foobar"
This solution though is open to bugs when the "hash_path" is not on an acceptable pattern, and treating it's bugs might make the code even longer
Shortest solution (Not safe)
You can use Kernel eval method which I EXTREMELY discourage to use for security reasons, read the documentation and understand its danger before using it
require 'json'
def do_the_thing(file, hash_path)
file = File.read(file)
data = JSON.parse(file, symbolize_names: true)
eval("data#{hash_path}")
end
do_the_thing('test.json', '[:foo][:bar][0]')
If the procedure you were trying to work with was just extracting the JSON data to an object, you might find yourself using either of the following scenarios:
def do_the_thing(file_to_load)
file = File.read(file)
data = JSON.parse(file, { symbolize_names: true })
data[sections.to_sym]
end
do_the_thing(file_I_want)[:foo][:bar][0]
or use the dig function of Hash :
def do_the_thing(file_to_load, sections)
file = File.read(file)
data = JSON.parse(file, { symbolize_names: true })
data.dig(*sections)
end
do_the_thing(file_I_want, [:foo, :bar, 0])
I have defined a custom function currently based on the very simple example here: https://docs.puppet.com/guides/custom_functions.html
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
filename = args[0]
hash_to_be_transformed = args[1]
File.open(filename, 'a') {|fd| fd.puts hash_to_be_transformed }
end
end
This kinda works. I can call it like this:
$my_hash = { key => "value1" , key2 => "value2" }
notify{ "new hash!! $my_hash" :}
transform_service_hash('/var/tmp/blah',$my_hash)
and the file displays:
mgt21 ~ # cat /var/tmp/blah
keyvalue1key2value2
But, if I try to access elements of the hash, nothing changes:
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
filename = args[0]
hash_to_be_transformed = args[1]
element1 = hash_to_be_transformed["key"]
File.open(filename, 'a') {|fd| fd.puts element1 }
end
end
The above block outputs the exact same data to /var/tmp/blah.
And, interestingly, if I remove the filename pass and define it statically in the module:
$my_hash = { key => "value1" , key2 => "value2" }
notify{ "new hash!! $my_hash. element1 is: $my_hash.key" :}
transform_service_hash($my_hash)
and
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
hash_to_be_transformed = args[0]
element1 = hash_to_be_transformed["key"]
File.open('/var/tmp/blah2', 'a') {|fd| fd.puts element1 }
end
end
I get the following error: "Error 400 on SERVER: can't convert Hash into String" with a line reference pointing to "transform_service_hash($my_hash)"
I am new to both puppet and ruby...so I'm unsure I am not passing the element properly, if I am not receiving it properly, or if it something that puppet cannot handle. Please note that I am using version 3.8 of puppet and 1.8.7 of ruby.
Thanks for any help. I've been banging my head against this, and google hasn't been forthcoming yet.
---Edit to clarify my goals (I also edited my code a bit for specificity): I am attempting to pass a hash into a custom ruby function within puppet. The "test" hash has two elements: one string and one array. It is defined as such:
$my_hash = { key => "value1" , key2 => ['array_value1', 'array_value2'] }
$my_display_element=$my_hash["key2"][0]
notify{ "new hash!! $my_hash. the first value of the array stored in element2 is: $my_display_element" :}
transform_service_hash('/var/tmp/blah',$my_hash)
The function appears like so:
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
filename = args[0]
hash_to_be_transformed = args[1]
element1 = args[1]["key"]
element2 = args[1]["key2"][0]
#element1 = hash_to_be_transformed["key"]
#element2 = hash_to_be_transformed["key2"][0]
File.open(filename, 'a') {|fd| fd.puts "hash_to_be_transformed: #{hash_to_be_transformed}\n" }
File.open(filename, 'a') {|fd| fd.puts "element1: #{element1}\n" }
File.open(filename, 'a') {|fd| fd.puts "element2: #{element2}\n" }
end
end
For now, I just want to be able to see that I am able to access elements within the passed hash like a hash. So I'd love for the output file to look like:
hash_to_be_transformed: keyvalue1key2array_value1array_value2
element1: value1
element2: array_value1
However, in the output file, I see:
mgt21 ~ # cat /var/tmp/blah
keyvalue1key2array_value1array_value2
Clearly, something is off here as my text is not being added and the full hash is just printed out just once and seemingly in string form.
I believe that this may be related to the error that I get when I don't pass in a file name (see above). I think that my hash is getting interpreted (or passed) as a string and, as such, I am unable to access the elements. Unfortunately, I still have been unable to verify this or figure out why it might be happening.
---Edit2 based on Matt's answer below.
I decided to simplify my code to isolate this "can't convert Hash into String error". I also made his suggested changes to remove the ambiguity from my key declarations.
$my_hash = { 'key' => "value1" , 'key2' => ['array_value1', 'array_value2'] }
$my_display_element=$my_hash["key2"][0]
notify{ "new hash!! $my_hash. the first value of the array stored in element2 is: $my_display_element" :}
transform_service_hash($my_hash)
and
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
hash_to_be_transformed = args[0]
element1 = args[0]['key']
element2 = args[0]['key2'][0]
File.open('/var/tmp/blah', 'a') {|fd| fd.puts "hash_to_be_transformed: #{hash_to_be_transformed}\n" }
File.open('/var/tmp/blah', 'a') {|fd| fd.puts "element1: #{element1}\n" }
File.open('/var/tmp/blah', 'a') {|fd| fd.puts "element2: #{element2}\n" }
end
end
But, I still end up with the same "Hash to String error". It is worth noting that I also tried simplifying my hash to:
$my_hash = { 'key' => "value1" , 'key2' => "value2" }
and I still get the "Hash to String error".
I quickly took your custom parser function and converted it into pure ruby like the following:
hash = { 'key' => 'value1', 'key2' => %w(array_value1 array_value2) }
def newfunction(filename, a_hash)
element1 = a_hash['key']
element2 = a_hash['key2'][0]
File.open(filename, 'a') do |fd|
fd.puts "hash_to_be_transformed: #{a_hash}"
fd.puts "element1: #{element1}"
fd.puts "element2: #{element2}"
end
end
newfunction('foo.txt', hash)
This results in the output text file like the following:
hash_to_be_transformed: {"key"=>"value1", "key2"=>["array_value1", "array_value2"]}
element1: value1
element2: array_value1
This seems to confirm my initial suspicion about what is going wrong here. Your hash in Puppet of:
$my_hash = { key => "value1" , key2 => ['array_value1', 'array_value2'] }
has keys of implicit/ambiguous types. In the ruby code I used to test, I explicitly established them as strings. This also correlates strongly with these lines in your code failing:
element1 = args[1]["key"]
element2 = args[1]["key2"][0]
and your error message of:
Error 400 on SERVER: can't convert Hash into String
because you are specifying in your ruby code that you expect the keys to be string. Changing your hash in Puppet to:
$my_hash = { 'key' => "value1" , 'key2' => "value2" }
should fix this.
On an unrelated note, I recommend the use of linters to help you learn these languages. Puppet-Lint, Rubocop, and Reek will all help point out suboptimal and messy parts of your code to help you learn the new languages.
On a related note, you may want to put something like this at the top of your custom parser function:
raise(Puppet::ParseError, 'newfunction expects two arguments') if args.length != 2
After much gnashing of teeth (and some very helpful pointers from #MattSchuchard), I realized that none of the changes to my function were going into effect. One needs to restart the puppetmaster service after each change to a custom function: docs.puppet.com/guides/custom_functions.html (appropriately under "Gotchas").
Once I started restarting this service after each change to the function, my hash was able to be parsed properly:
from the .pp file:
$filename = "/var/tmp/test"
$my_hash = { 'key' => "value1" , 'key2' => ["M\'lady\n*doffs cap*", 'array_value2'] }
transform_service_hash($filename, $my_hash)
from the ruby file:
module Puppet::Parser::Functions
newfunction(:transform_service_hash) do |args|
filename = args[0]
hash_to_be_transformed = args[1]
array_val = hash_to_be_transformed['key2'][0]
File.open(filename, 'a') {|fd| fd.puts "#{array_val}\n" }
end
end
and output:
mgt21 tmp # cat test
M'lady
*doffs cap*
I wrote this bit of code that reverses XOR-based encryption in Ruby. The chipertext is XORed with 'key' and the output is passed to Zlib.deflate.
require 'zlib'
def bin_to_hex(s)
s.unpack('H*').first
end
def hex(s)
s.scan(/../).map { |x| x.hex }.pack('c*')
end
chipertext = "Encrypted data"
key = "Some encryption key"
puts hex((bin_to_hex(Zlib::Inflate.inflate(code)).to_i(16) ^ ((bin_to_hex(key) * (bin_to_hex(Zlib::Inflate.inflate(chipertext)).length/bin_to_hex(key).length)) + bin_to_hex(key)[0, bin_to_hex(Zlib::Inflate.inflate(chipertext)).length%bin_to_hex(key).length]).to_i(16)).to_s(16))
The code runs perfectly when I specify chipertext as a string, in the example above. But when I use code like chipertext = File.open(ARGV[0], 'rb') { |f| f.read }, I get a inflate: incorrect header check (Zlib::DataError).
How can I prevent this from happening?
I've got a complex XML file, and I want to extract a content of a specific tag from it.
I use a ruby script with XmlSimple gem. I retrieve an XML file with HTTP request, then strip all the unnecessary tags and pull out necessary info. That's the script itself:
data = XmlSimple.xml_in(response.body)
hash_1 = Hash[*data['results']]
def find_value(hash, value)
hash.each do |key, val|
if val[0].kind_of? Hash then
find_value(val[0], value)
else
if key.to_s.eql? value
puts val
end
end
end
end
hash_1['book'].each do |arg|
find_value(arg, "title")
puts("\n")
end
The problem is, that when I change replace puts val with return val, and then call find_value method with puts find_value (arg, "title"), i get the whole contents of hash_1[book] on the screen.
How to correct the find_value method?
A "complex XML file" and XmlSimple don't mix. Your task would be solved a lot easier with Nokogiri, and be faster as well:
require 'nokogiri'
doc = Nokogiri::XML(response.body)
puts doc.xpath('//book/title/text()')
I am using ruby gpgme gem (1.0.8). My passphrase callback isn't called:
def passfunc(*args)
fd = args.last
io = IO.for_fd(fd, 'w')
io.puts "mypassphrase"
io.flush
end
opts = {
:passphrase_callback => method(:passfunc)
}
GPGME.decrypt(input,output, opts)
Does someone have working example of passphrase callback?
Sample of callback you can find in the following working example. It signs a file in detached mode, i.e., the signature file is separated from the original file. It uses the default keyring at ~/.gnupg or something like that. To use a different directory for your keyring, set the environment variable ENV["GNUPGHOME"]="" before call GPGME::sign().
#!/usr/bin/ruby
require 'rubygems'
require 'gpgme'
puts "Signing #{ARGV[0]}"
input = File.open(ARGV[0],'r')
PASSWD = "abc"
def passfunc(hook, uid_hint, passphrase_info, prev_was_bad, fd)
puts("Passphrase for #{uid_hint}: ")
io = IO.for_fd(fd, 'w')
io.write(PASSWD+"\n")
io.flush
end
output = File.open(ARGV[0]+'.asc','w')
sign = GPGME::sign(input, {
:passphrase_callback => method(:passfunc),
:mode => GPGME::SIG_MODE_DETACH
})
output.write(sign)
output.close
input.close
Here's another working example for you that doesn't use a detached signature. To test this, simply change 'user#host.name' to the identifier of your key and do this: GPG.decrypt(GPG.encrypt('some text', :armor => true))
require 'gpgme'
require 'highline/import'
module GPG
ENCRYPT_KEY = 'user#host.com'
#gpg = GPGME::Crypto.new
class << self
def decrypt(encrypted_data, options = {})
options = { :passphrase_callback => self.method(:passfunc) }.merge(options)
#gpg.decrypt(encrypted_data, options).read
end
def encrypt(data_to_encrypt, options = {})
options = { :passphrase_callback => self.method(:passfunc), :armor => true }.merge(options)
#gpg.encrypt(data_to_encrypt, options).read
end
private
def get_passphrase
ask("Enter passphrase for #{ENCRYPT_KEY}: ") { |q| q.echo = '*' }
end
def passfunc(hook, uid_hint, passphrase_info, prev_was_bad, fd)
begin
system('stty -echo')
io = IO.for_fd(fd, 'w')
io.puts(get_passphrase)
io.flush
ensure
(0 ... $_.length).each do |i| $_[i] = ?0 end if $_
system('stty echo')
end
$stderr.puts
end
end
end
Cheers!,
--
Carl
It is important to note that as of GnuPG 2.0 (and in 1.4 when the use-agent option is used) pinentry is used for passphrase collection. This means that the gpgme passphrase callback will not be invoked. This is described here and an example of usage can be found in the gpgme-tool example.