Use controller method in ActionCable channel - ruby

I have a rails api that I am trying to create a live stream feature for a git gui. I have a controller and a channel for commit updates and I want to use the live stream function inside the subscribe action in the channel. Is this possible? Or is there a better Rails way to do this. I think the correct way was to use ActionCable.server.broadcast('commit_updates', { body: return_data }) inside my controller method but I am not calling my controller method anywhere. Where/how do I call it so that it is run on a client subscribing to it? or should I place the logic inside the channel?
here is my commit_updates_controller.rb
class CommitUpdatesController < ApplicationController
include ActionController::Live
require 'listen'
require 'json'
$file_path = "#{Dir.getwd}/.git/objects"
##commits = {}
##commits_array = []
##edges_array = []
def index
end
def live_stream
# get folders exlcluding ".","..", "pack","info"
folders = Dir.entries($file_path).select {|folder| /^[a-z0-9]{2}$/i.match(folder) }
folders.each do |folder|
files = Dir.children("#{$file_path}/#{folder}")
files.each do |file|
CommitUpdate.find_commit_info(folder, file, ##commits_array)
end
end
generate_edges
ActionCable.server.broadcast('commit_updates', { body: return_data })
p return_data
# listens to any changes that happen to the git folder while the program is open
listener = Listen.to($file_path) do |modified, added, removed|
# puts(modified: modified, added: added, removed: removed)
added.each do |new_file_path|
split_new_file_path = new_file_path.split("/").reject!{|item| item.empty?}
folder = split_new_file_path[split_new_file_path.length() - 2]
file = split_new_file_path[split_new_file_path.length - 1]
CommitUpdate.find_commit_info(folder, file, ##commits_array)
add_edge(##edges_array, CommitUpdate.get_commit_info(folder, file))
ActionCable.server.broadcast('commit_updates', { body: return_data })
end
end
listener.start
sleep
end
private
def generate_edges
if ##commits_array.length != 0
##commits_array.each do |commit|
add_edge(##commits_array, commit)
end
end
end
def add_edge(array, commit)
if commit[:parents] != []
commit[:parents].each {|parent| ##edges_array.push({from: parent, to: commit[:sha1]})}
end
end
def return_data
ret = {
:nodes => ##commits_array,
:links => ##edges_array
}
return ret.to_json
end
end
here is my commit_updates_channel.rb
class CommitUpdatesChannel < ApplicationCable::Channel
def subscribed
stream_from 'commit_updates'
end
end
here is my commit_updates.rb Model
class CommitUpdate < ApplicationRecord
def self.find_commit_info(folder, file, array)
file_type = self.check_sha1_type(folder, file)
if file_type == "commit"
array.push(
self.get_commit_info(folder, file)
)
end
end
def self.get_commit_info(folder, file)
author = ""
parents = []
commit_message = ""
unixtime = ""
decoded_file = `git cat-file -p #{folder}#{file}`
file_data_array = decoded_file.split("\n").reject!{|item| item.empty?}
p file_data_array
file_data_array.each do |item|
split_item = item.split(" ")
case split_item[0]
when "author"
author = split_item[1..split_item.length() - 4].join(" ")
when "parent"
parents.push(split_item[1])
when "tree"
next
when "blob"
next
when "committer"
unixtime = split_item[split_item.length - 2]
else
commit_message = split_item.join(" ")
end
end
commit_info = {
:sha1 => "#{folder}#{file}",
:parents => parents,
:commit_message => commit_message,
:author => author,
:unixtime => unixtime,
:id => "#{folder}#{file}",
:label => "#{folder}#{file[0..4]}...",
:font => "18px verdana blue",
}
return commit_info
end
private
def self.check_sha1_type(folder, file)
return `git cat-file -t #{folder}#{file}`.chomp
end
end

To use controller's live_stream, you need to request this method, for example, write js programme that is calling it periodically. Or some bash script that is calling it. Not good.
Another solution is to put live_stream logic into background job that will check commit updates every X minute. And if job finds change, it broadcasts.
And more common is to place broadcasting in model callbacks after_save, after_commit. But it looks it is not your solution.

Related

Chef delay attribute assignment via data bag

So i have a bit of a pickle.
I have an encrypted data bag to store LDAP passwords. In my node run list, one of my recipes installs the secret key onto my client machine.
In my problematic cookbook, i have a helper (in /libraries) that pulls data from AD (using LDAP). Problem is, i can't find a way to delay the assignment of my node attribute after initial compile phase.
Take this line of code as example :
node.override['yp_chefserver']['osAdminUser'] = node['yp_chefserver']['osAdminUser'] + get_sam("#{data_bag_item('yp_chefserver', 'ldap', IO.read('/etc/chef/secret/yp_chefserver'))['ldap_password']}")
Im trying to override an attribute by adding an array returned by my helper function "get_sam" which returns an array, but it needs to run AFTER the compile phase since the file "/etc/chef/secret/yp_chefserver" doesnt exist before the convergence of my runlist.
So my question : Is there a way to assign node attributes via data_bag_items during the execution phase?
Some things i've tried :
ruby_block 'attribution' do
only_if { File.exist?('/etc/chef/secret/yp_chefserver')}
block do
node.override['yp_chefserver']['osAdminUser'] = node['yp_chefserver']['osAdminUser'] + get_sam("#{data_bag_item('yp_chefserver', 'ldap', IO.read('/etc/chef/secret/yp_chefserver'))['ldap_password']}")
Chef::Log.warn("content of osAdminUser : #{node['yp_chefserver']['osAdminUser']}")
end
end
This doesn't work because the custom resource ruby_block doesn't have the method "data_bag_item". I've tried using lazy attributes in my "chef_server" custom resource, but same problem.
I also tried having the attribution done directly in my helper module, but since the helper module compiles before the exec phase, the file doesn't exist when it assigns the variable.
Here is the helper function in question should anyone wonder, it pulls the SamAccountName from LDAP to assign admin users to my chef server. :
module YpChefserver
module LDAP
require 'net-ldap'
#ldap
def get_ldap(ldap_password)
if #ldap.nil?
#ldap = Net::LDAP.new :host => "ADSERVER",
:port => 389,
:auth => {
:method => :simple,
:username => "CN=USERNAME,OU=East Service Accounts,OU=System Accounts,DC=ad,DC=ypg,DC=com",
:password => "#{ldap_password}"
}
end
#ldap
end
def get_ldap_users(ldap_password)
filter = Net::LDAP::Filter.eq("cn", "DevOps")
treebase = "dc=ad, dc=ypg, dc=com"
get_ldap(ldap_password).search(:base => treebase, :filter => filter) do |entry|
#puts "DN: #{entry.dn}"
entry.each do |attribute, values|
return values if attribute == :member
end
end
end
def get_sam(ldap_password)
samacc = Array.new
get_ldap_users(ldap_password).entries.each{ |elem|
y = elem.to_s.split(/[,=]/)
filter = Net::LDAP::Filter.eq("cn", y[1])
treebase = "OU=Support Users and Groups,OU=CGI Support,DC=ad,DC=ypg,DC=com"
get_ldap(ldap_password).search(:base => treebase, :filter => filter, :attributes => "SamAccountName") do |entry|
samacc << entry.samaccountname
end
}
return samacc
end
end
end
Turns out you can actually call it inside a ruby block, just by using the actual Chef call instead of the resource name, as follow :
ruby_block 'attributes' do
only_if {File.exist?('/etc/chef/secret/yp_chefserver')}
block do
dtbg = Chef::EncryptedDataBagItem.load('yp_chefserver','ldap',"IO.read('/etc/chef/secret/yp_chefserver')")
end
end
Leaving this here for those who might need it
EDIT :
Here is final function using the code mentionned above to pull accounts from AD, using encrypted data bags to provide the password and to then pass those results to my node attributes, all during the execution phase :
ruby_block 'attributes' do
extend YpChefserver::LDAP
only_if {File.exist?('/etc/chef/secret/yp_chefserver')}
block do
# Chef::Config[:encrypted_data_bag_secret] = '/etc/chef/secret/yp_chefserver'
dtbg = Chef::EncryptedDataBagItem.load('yp_chefserver','ldap')
node.override['yp_chefserver']['ldap_pw'] = dtbg['ldap_password']
userarray = Array.new
userarray.push("#{node['yp_chefserver']['osAdminUser']}")
get_sam("#{node['yp_chefserver']['ldap_pw']}").each { |i| userarray.push(i[0]) }
node.override['yp_chefserver']['authorized_users'] = userarray
node.override['yp_chefserver']['local_admin_pw'] = dtbg['local_admin_pw']
end
end

Ruby method variable declaration

I'm trying to define methods to parse through an apache log file and pull ip addresses, URLs, requests per hour, and error codes. I've got everything working outside of methods, but when attempting to put that code into the methods I keep getting the error message "Stack level too deep." Here is the code in question.
class CommonLog
def initialize(logfile)
#logfile = logfile
end
def readfile
#readfile = File.readlines(#logfile).map { |line|
line.split()
}
#readfile = #readfile.to_s.split(" ")
end
def ip_histogram
#ip_count = 0
#readfile.each_index { |index|
if (#readfile[index] =~ /\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/ )
puts #readfile[index]
puts #ip_count += 1
end
}
end
def url_histogram
url_count = 0
cleaned_file.each_index { |index|
if (cleaned_file[index] =~ /\/{1}(([a-z]{4,})|(\~{1}))\:{0}\S+/ )
puts cleaned_file[index]
puts url_count += 1
end
}
end
def requests_per_hour
end
def sorted_list
end
end
my_file = CommonLog.new("test_log")
cleaned_file = my_file.readfile
puts cleaned_file.ip_histogram
It looks like the problem lies on you CommonLog#readfile method:
def readfile
#readfile = File.readlines(#logfile).map { |line|
line.split()
}
#readfile = readfile.to_s.split(" ")
end
Notice that inside the implementation of readfile your calling readfile recursively? When it executes it reads the lines from the file, maps them and assign the result the #readfile; then it calls readfile and the method starts to execute again; this goes on forever, until you stack blows up because of too many recursive method calls.
I assume what you actually meant is:
#readfile = #readfile.to_s.split(" ")

Calling multiple methods on a CSV object

I have constructed an Event Manager class that performs parsing actions on a CSV file, and produces html letters using erb. It is part of a jumpstart labs tutorial
The program works fine, but I am unable to call multiple methods on an object without the earlier methods interfering with the later methods. As a result, I have opted to create multiple objects to call instance methods on, which seems like a clunky inelegant solution. Is there a better way to do this, where I can create a single new object and call methods on it?
Like so:
eventmg = EventManager.new("event_attendees.csv")
eventmg.print_valid_phone_numbers
eventmg_2 = EventManager.new("event_attendees.csv")
eventmg_2.print_zipcodes
eventmg_3 = EventManager.new("event_attendees.csv")
eventmg_3.time_targeter
eventmg_4 = EventManager.new("event_attendees.csv")
eventmg_4.day_of_week
eventmg_5 = EventManager.new("event_attendees.csv")
eventmg_5.create_thank_you_letters
The complete code is as follows
require 'csv'
require 'sunlight/congress'
require 'erb'
class EventManager
INVALID_PHONE_NUMBER = "0000000000"
Sunlight::Congress.api_key = "e179a6973728c4dd3fb1204283aaccb5"
def initialize(file_name, list_selections = [])
puts "EventManager Initialized."
#file = CSV.open(file_name, {:headers => true,
:header_converters => :symbol} )
#list_selections = list_selections
end
def clean_zipcode(zipcode)
zipcode.to_s.rjust(5,"0")[0..4]
end
def print_zipcodes
puts "Valid Participant Zipcodes"
#file.each do |line|
zipcode = clean_zipcode(line[:zipcode])
puts zipcode
end
end
def clean_phone(phone_number)
converted = phone_number.scan(/\d/).join('').split('')
if converted.count == 10
phone_number
elsif phone_number.to_s.length < 10
INVALID_PHONE_NUMBER
elsif phone_number.to_s.length == 11 && converted[0] == 1
phone_number.shift
phone_number.join('')
elsif phone_number.to_s.length == 11 && converted[0] != 1
INVALID_PHONE_NUMBER
else
phone_number.to_s.length > 11
INVALID_PHONE_NUMBER
end
end
def print_valid_phone_numbers
puts "Valid Participant Phone Numbers"
#file.each do |line|
clean_number = clean_phone(line[:homephone])
puts clean_number
end
end
def time_targeter
busy_times = Array.new(24) {0}
#file.each do |line|
registration = line[:regdate]
prepped_time = DateTime.strptime(registration, "%m/%d/%Y %H:%M")
prepped_time = prepped_time.hour.to_i
# inserts filtered hour into the array 'list_selections'
#list_selections << prepped_time
end
# tallies number of registrations for each hour
i = 0
while i < #list_selections.count
busy_times[#list_selections[i]] += 1
i+=1
end
# delivers a result showing the hour and the number of registrations
puts "Number of Registered Participants by Hour:"
busy_times.each_with_index {|counter, hours| puts "#{hours}\t#{counter}"}
end
def day_of_week
busy_day = Array.new(7) {0}
d_of_w = ["Monday:", "Tuesday:", "Wednesday:", "Thursday:", "Friday:", "Saturday:", "Sunday:"]
#file.each do |line|
registration = line[:regdate]
# you have to reformat date because of parser format
prepped_date = Date.strptime(registration, "%m/%d/%y")
prepped_date = prepped_date.wday
# adds filtered day of week into array 'list selections'
#list_selections << prepped_date
end
i = 0
while i < #list_selections.count
# i is minus one since days of week begin at '1' and arrays begin at '0'
busy_day[#list_selections[i-1]] += 1
i+=1
end
#busy_day.each_with_index {|counter, day| puts "#{day}\t#{counter}"}
prepared = d_of_w.zip(busy_day)
puts "Number of Registered Participants by Day of Week"
prepared.each{|date| puts date.join(" ")}
end
def legislators_by_zipcode(zipcode)
Sunlight::Congress::Legislator.by_zipcode(zipcode)
end
def save_thank_you_letters(id,form_letter)
Dir.mkdir("output") unless Dir.exists?("output")
filename = "output/thanks_#{id}.html"
File.open(filename,'w') do |file|
file.puts form_letter
end
end
def create_thank_you_letters
puts "Thank You Letters Available in Output Folder"
template_letter = File.read "form_letter.erb"
erb_template = ERB.new template_letter
#file.each do |line|
id = line[0]
name = line[:first_name]
zipcode = clean_zipcode(line[:zipcode])
legislators = legislators_by_zipcode(zipcode)
form_letter = erb_template.result(binding)
save_thank_you_letters(id,form_letter)
end
end
end
The reason you're experiencing this problem is because when you apply each to the result of CSV.open you're moving the file pointer each time. When you get to the end of the file with one of your methods, there is nothing for anyone else to read.
An alternative is to read the contents of the file into an instance variable at initialization with readlines. You'll get an array of arrays which you can operate on with each just as easily.
"Is there a better way to do this, where I can create a single new object and call methods on it?"
Probably. If your methods are interfering with one another, it means you're changing state within the manager, instead of working on local variables.
Sometimes, it's the right thing to do (e.g. Array#<<); sometimes not (e.g. Fixnum#+)... Seeing your method names, it probably isn't.
Nail the offenders down and adjust the code accordingly. (I only scanned your code, but those Array#<< calls on an instance variable, in particular, look fishy.)

How do you upload a zip file and unzip to s3?

I am working on an application where I have to upload a zip file. The zip file is basically a static website so it has many files and a couple subdirectories. I have been playing with the rubyzip gem for a while now and can not figure out how to simply extract the files from it. Any pointers on where I can read up on some examples? I am sure someone has ran in to this problem before. the documentation for rubyzip is not very good so I am hoping someone can give me some pointers.
Here you go, one super magical multithreaded zip-to-S3 uploader which I haven't tested at all - go nuts! Looks like I'm three years too late though.
class S3ZipUploader
require 'thread'
require 'thwait'
require 'find'
attr_reader *%i{ bucket s3 zip failed_uploads }
def initialize(zipfilepath, mys3creds)
# next 4 lines are important
#s3 = AWS::S3.new(access_key_id: mys3creds[Rails.env]['aws_access_key'],
secret_access_key: mys3creds[Rails.env]['aws_secret_access_key'],
region: 'us-west-2')
#bucket = #s3.buckets[ mys3creds[Rails.env]['bucket'] ]
#failed_uploads = []
#zip = Zip::File.open(zipfilepath)
end
def upload_zip_contents
rootpath = "mypath/"
desired_threads = 10
total_entries = zip.entries.count
slice_size = (total_entries / desired_threats).ceil
threads = []
zip.entries.each_slice(slice_size) do |e_arr|
threads << Thread.new do |et|
e_arr.each do |e|
result = upload_to_s3(rootpath + e.name, e.get_input_stream.read)
if !result
#failed_uploads << {name: e.name, entry: e, error: err}
end
end
end
end
ThreadsWait.all_waits(*threads)
end
def upload_file_to_s3(filedata,path, rewrite_basepath)
retries = 0
success = false
while !success && retries < 3
success = begin
obj = bucket.objects[path]
obj.write(Pathname.new(outputhtml))
obj.acl = :public_read
success = true
rescue
retries += 1
success = false
end
end
return success
end
end
uploader = S3ZipUploader.new("/path/to/myzip.zip", MYS3CREDS)
uploader.upload_zip_contents

Using passphrase callback in ruby gpgme

I am using ruby gpgme gem (1.0.8). My passphrase callback isn't called:
def passfunc(*args)
fd = args.last
io = IO.for_fd(fd, 'w')
io.puts "mypassphrase"
io.flush
end
opts = {
:passphrase_callback => method(:passfunc)
}
GPGME.decrypt(input,output, opts)
Does someone have working example of passphrase callback?
Sample of callback you can find in the following working example. It signs a file in detached mode, i.e., the signature file is separated from the original file. It uses the default keyring at ~/.gnupg or something like that. To use a different directory for your keyring, set the environment variable ENV["GNUPGHOME"]="" before call GPGME::sign().
#!/usr/bin/ruby
require 'rubygems'
require 'gpgme'
puts "Signing #{ARGV[0]}"
input = File.open(ARGV[0],'r')
PASSWD = "abc"
def passfunc(hook, uid_hint, passphrase_info, prev_was_bad, fd)
puts("Passphrase for #{uid_hint}: ")
io = IO.for_fd(fd, 'w')
io.write(PASSWD+"\n")
io.flush
end
output = File.open(ARGV[0]+'.asc','w')
sign = GPGME::sign(input, {
:passphrase_callback => method(:passfunc),
:mode => GPGME::SIG_MODE_DETACH
})
output.write(sign)
output.close
input.close
Here's another working example for you that doesn't use a detached signature. To test this, simply change 'user#host.name' to the identifier of your key and do this: GPG.decrypt(GPG.encrypt('some text', :armor => true))
require 'gpgme'
require 'highline/import'
module GPG
ENCRYPT_KEY = 'user#host.com'
#gpg = GPGME::Crypto.new
class << self
def decrypt(encrypted_data, options = {})
options = { :passphrase_callback => self.method(:passfunc) }.merge(options)
#gpg.decrypt(encrypted_data, options).read
end
def encrypt(data_to_encrypt, options = {})
options = { :passphrase_callback => self.method(:passfunc), :armor => true }.merge(options)
#gpg.encrypt(data_to_encrypt, options).read
end
private
def get_passphrase
ask("Enter passphrase for #{ENCRYPT_KEY}: ") { |q| q.echo = '*' }
end
def passfunc(hook, uid_hint, passphrase_info, prev_was_bad, fd)
begin
system('stty -echo')
io = IO.for_fd(fd, 'w')
io.puts(get_passphrase)
io.flush
ensure
(0 ... $_.length).each do |i| $_[i] = ?0 end if $_
system('stty echo')
end
$stderr.puts
end
end
end
Cheers!,
--
Carl
It is important to note that as of GnuPG 2.0 (and in 1.4 when the use-agent option is used) pinentry is used for passphrase collection. This means that the gpgme passphrase callback will not be invoked. This is described here and an example of usage can be found in the gpgme-tool example.

Resources