osquery extension in Ruby - create new table - ruby

I'm trying to implement an extension for osquery in Ruby.
I found some libs and examples doing the same in Java, Node and Python, but nothing helpful implemented in Ruby language.
According to this documention, it's possible generating the code using Thrift: https://osquery.readthedocs.io/en/stable/development/osquery-sdk/#thrift-api
The steps I did, so far:
Generated the code using thrift -r --gen rb osquery.thrift
Created a class and some code to connect to the server and register the extension
This is the code of the class
# include thrift-generated code
$:.push('./gen-rb')
require 'thrift'
require 'extension_manager'
socket = Thrift::UNIXSocket.new(<path_to_socket>)
transport = Thrift::FramedTransport.new(socket)
protocol = Thrift::BinaryProtocol.new(transport)
client = ExtensionManager::Client.new(protocol)
transport.open()
info = InternalExtensionInfo.new
info.name = "zzz"
info.version = "1.0"
extension = ExtensionManager::RegisterExtension_args.new
extension.info = info
client.registerExtension(extension, {'table' => {'zzz' => [{'name' => 'TEXT'}]}})
To get the <path_to_socket> you can use:
> osqueryi --nodisable_extensions
osquery> select value from osquery_flags where name = 'extensions_socket';
+-----------------------------------+
| value |
+-----------------------------------+
| /Users/USERNAME/.osquery/shell.em |
+-----------------------------------+
When I try to get this table using osqueryi, I don't see the table when I run select * from osquery_registry;.
Have anybody by any chance implemented an osquery extension already? I'm stuck and I don't know how to proceed from here.

I don't think I've seen anyone make a ruby extension, but once you have the thrift side, it should be pretty simple.
As a tool, osquery supports a lot of options. So there's no single way to do this. Generally speaking, extensions run as their own process and the communicate over that thrift socket.
Usually, they're very simple and osquery invokes extensions directly with appropriate command line arguments. This is alluded to in the doc you linked with the example accepting --interval, --socket, and --timeout. If you do this, you'll want to look at osquery's --extensions_autoload and --extensions_require option. (I would recommend this route)
Less common, is to start osquery with a specified socket path using --extensions_socket. And then your extension can use that. This way is more common is the extension cannot be a simple binary, and instead is a large complex system.

I find myself playing around with thrift via ruby. And it seems to work if I used a BufferedTransport:
socket = Thrift::UNIXSocket.new('/tmp/osq.sock')
transport = Thrift::BufferedTransport.new(socket)
protocol = Thrift::BinaryProtocol.new(transport)
client = ExtensionManager::Client.new(protocol)
transport.open()
client.ping()
client.query("select 1")

Related

Loading protocol buffer in ruby or java similar to node

I have a .proto file that contains my schema and service definition. I'm looking for a method in ruby/java that is similar to how Node loads and parses it (code below). Looking at the grpc ruby gem, I don't see anything that can replicate how Node does it.
Digging around I see this (https://github.com/grpc/grpc/issues/6708) which states that dynamically loading .proto files is only available in node. Hopefully, someone can provide me with an alternative.
Use case: Loading .proto files dynamically as provided in the client but I can only use either ruby or java to do it.
let grpc = require("grpc");
let loader = require("#grpc/proto-loader");
let packageDefinition = loader.loadSync(file.file, {});
let parsed = grpc.loadPackageDefinition(packageDefinition);
I've been giving this a try for the past few month and it seems like Node is the only way to read a protobuf file on runtime. Hopefully this helps anyone in the future that needs it.

Upload to S3 with progress in plain Ruby script

This question is related to this one: Tracking Upload Progress of File to S3 Using Ruby aws-sdk,
However since there is no clear solution to this I was wondering if there's a better/easier way (if one exists) of getting file upload progress with S3 using Ruby in 2018?
In my current setup I'm basically creating a new Resource, fetch my bucket and call upload_file but I haven't yet found any options for passing blocks which would help in yielding some sort of progress.
...
#connection = Aws::S3::Resource.new
#s3_bucket = #connection.bucket(bucket)
#s3_bucket.object(path).upload_file(data, {acl: 'public-read'})
...
Is there a way to do this using the newest sdk-for-ruby v3?
Any help (or even better a small example) would be great.
The example Trevor gives in https://stackoverflow.com/a/12147709/153886 is not hacky from what I can see - just wiring things together. The SDK simply does not provide a feature for passing progress details on all operations. Plus, Trevor is the maintainer of the Ruby SDK at AWS so I trust his judgement.
Expanding on his example
bar = ProgressBar.create(:title => "Uploading action", :starting_at => 0, :total => file.size)
obj = s3.buckets['my-bucket'].objects['object-key']
obj.write(:content_length => file.size) do |writable, n_bytes|
writable.write(file.read(n_bytes))
bar.progress += n_bytes
end
If you want to have a progress block right in the upload_file method I believe you will need to open a PR to the SDK. It is not that strange that is not the case for Ruby (or for any other runtime) because, for example, there could be an optimisation in the HTTP client library that uses IO.copy_stream from your source body argument to the destination socket, which does not relay progress anywhere.

I am trying to upgrade my script from Cloudera hbase 4(CDH4) version to (CDH5)

def getRegions(config, servername)
connection = HConnectionManager::getConnection(config)
parts = servername.split(',')
puts parts
rs = connection.getHRegionConnection(parts[0], parts[1].to_i)
return rs.getOnlineRegions()
end
I am trying to make this code compatible with CDH5. I have looked into CDH5 library but unable to find exact solution.
I am using
connection = ConnectionFactory::createConnection(config) which returns Connection object.
I want list of onlineRegions on given server.
Have a look the following api's
Admin.html#getClusterStatus()
ClusterStatus.html#getServers()
Admin.html#getOnlineRegions(org.apache.hadoop.hbase.ServerName)
Note : In older versions, Some of the Admin functions live in HBaseAdmin class. (Rest of the usage should be same/similar)
Hopefully, that should help you.

How should I implement IDownloadProgressChangedCallback using WIN32OLE?

I'd like to download Windows updates by use of WIN32OLE in Ruby. I'm running into issues knowing which COM object to use in order to get the arguments needed by IUpdateDownloader::BeginDownload . For the time being, I can execute the synchronous version of downloading updates, but I would really like to know how I can go about using the asynchronous method.
Something such as the following works now:
require 'win32ole'
muSession = WIN32OLE.new('Microsoft.Update.Session')
availableUpdates = muSession.CreateUpdateSearcher().Search("IsInstalled=0 and Type='Software'").Updates
muUpdateColl = WIN32OLE.new('Microsoft.Update.UpdateColl')
availableUpdates.each do |update|
update.AcceptEula()
muUpdateColl.Add(update)
end
updateDownloader = WIN32OLE.new('Microsoft.Update.Session').CreateUpdateDownloader()
updateDownloader.Updates = muUpdateColl
downloadResult = updateDownloader.Download()
However, instead of invoking "Download()", I would like to use "BeginDownload()". How can I instantiate IDownloadProgressChangedCallback (for example). I think doing so may be obvious in C#, but using WIN32OLE, I am not sure how to create the object.

how to log work for a given issue using jira soap?

I'm working on a ruby on rails app which connects to Jira through jira soap. I browsed through the Jira SOAP API documentation but could not figure out a way to log time for an issue. What method(s) do I've to use to allow users to log time for some issue?
addWorklogAndAutoAdjustRemainingEstimate is the one I'm using in my scripts
This is a snippet of python code that might be useful to you:
d = datetime.datetime.now()
timestamp = "%s-%02d-%02dT%02d:%02d:00+01:00"%(d.year,d.month,d.day,d.hour,d.minute);
log = self.client.factory.create("ns0:RemoteWorklog")
log.comment = comment # a string
log.timeSpent = time # in the usual format (2h30m )
log.startDate = timestamp
# task is the name of the tast (a string)
self.client.service.addWorklogAndAutoAdjustRemainingEstimate(self.auth,task,log)
Via the addWorkLogXxx methods, where Xxx is what to do with the estimate.
for ruby developers,
worklog class from jira4r gem can be accessed via : Jira4R::V2::RemoteWorklog

Resources