What would be the simplest approach to streaming the contents of a file to the web page, as it gets updated.
I currently have a .txt file that is constantly updated while a script is running, and I want to display that on the page for the users as it is updated.
Is there a simple gem or technology out there to accomplish this? Or is there an excellent pure Ruby approach?
My first thought was to use some kind of AJAX request to return diffs of the file, but I feel that would be a bad approach.
I used Celluloid's Reel by Tony Arcieri to accomplish this.
I run this in the background (received a lot of help from Adam Dalton on this part):
my_reel.rb
require 'reel'
CONNECTIONS = []
Reel::Server.supervise("0.0.0.0", 5000) do |connection|
while request = connection.request
case request
when Reel::Request
puts "Client requested: #{request.method} #{request.url} #{request.body}"
CONNECTIONS.each do |c|
c << request.body
end
request.respond :ok, 'YES! YOU GOT IT!'
when Reel::WebSocket
puts "Client made a WebSocket request to: #{request.url}"
CONNECTIONS << request
break
end
end
end
sleep
Then I wrote a command line script to send post requests to the Reel server. Had a lot of help from Writing Ruby Scripts That Respect Pipelines by Jesse Storimer.
~/bin/serve_it_up
#!/usr/bin/env ruby
require 'httparty'
ARGF.each_line do |line|
output_line = line.chop
HTTParty.post 'http://localhost:5000', body: output_line
end
Then to make it work, my Rails application uses Sidekiq (bundle exec sidekiq), and runs a script in the background when a button is clicked, which sends the output to a file, output.txt. So I run a forced tail and pipe it's output to my command line script.
tail -f output.txt | serve_it_up
In my web app, I have some JavaScript (CoffeeScript in this case) that hooks up to the web socket and puts the output on the page:
connection = new WebSocket('ws://localhost:5000')
connection.onmessage = (event) ->
$('#prompt').append(event.data + '<br/>')
prompt = document.getElementById('prompt')
prompt.scrollTop = prompt.scrollHeight
So anytime that output.txt changes, it'll take the output from tail -f and put it in the web browser. I also style the #prompt div to have a limited height, and I keep it always scrolled to the bottom.
Here is the style for it:
#prompt {
font-family: Consolas, Menlo, Monaco, Lucida Console, Liberation Mono, DejaVu Sans Mono, Bitstream Vera Sans Mono, Courier New, monospace, serif;
height: 200px;
overflow: auto;
}
You could look into using something like Faye or the gem private_pub (it makes Faye channels private) which allows you to make use of web sockets or long polling to push new data to the client. There is a lot of documentation on the Github page and also a RailsCast.
In your app or script you could then publish to the server which will push the new data to all subscribed channels.
Related
i have a server installation of redmine, i have installed shady mode plugin with 3.4 version and it is working but now i have update to 4.1 and doesn't work. This plugin allow me to enable shady mode and stop sending notification.
I found the problem of plugin code, I found two files where in both it does the check to see if the mode is active by going to retrieve the user's properties, in one file the check actually works and the condition returns the correct value, in the other file (the one that decides if to actually send the email or not) this setting is always false (or null).
I guess with the new version of redmine the plugin in that piece of code can't find the variables.
this is the file that work hook.rb:
module RedmineShady
module Hook
class ViewListener < Redmine::Hook::ViewListener
# necessary for using content_tag in Listener
attr_accessor :output_buffer
# If shady, show message on the top
def view_layouts_base_html_head(context = {})
session = context[:controller].session
if User.current.pref[:shady]
style = 'margin: 0; padding: 10px; border-width: 0 0 2px; background-image: none'
content_tag :div, id: 'shady-bar', class: 'flash error', style: style do
concat link_to l(:button_cancel), { controller: 'shady', action: 'destroy' },
method: :delete, style: 'float: right'
concat l(:notice_shady_mode)
end
end
end
end
end
end
and this is the file that doesn't work: mail_interceptor.rb
module RedmineShady
class MailInterceptor
def self.delivering_email(message)
message.perform_deliveries = false if User.current.pref[:shady]
end
end
end
Why does User.current.pref[:shady] work in hook.rb but not in mail_interceptor.rb?
Between Redmine 3.4 and Redmine 4.1, the way how email notifications are send has changed significantly.
Specifically, Redmine now renders and sends notification mails in the background rather than waiting in the update request. Also, a unique mail is now rendered for each recipient. While rendering the mail, User.current will be set to the respective recipient (rather than the sending user as in Redmine 3.4).
As such, the plugin needs to be updated for Redmine 4.1 and likely needs a significant re-architecture due to those Redmine changes.
I'm running a script that will open up on my localhost. My local server is a vulnerable web app test suite.
I'm trying to confirm a XSS popup from a JavaScript alert. For example:
http://127.0.0.1:65412/v?=0.2<script>alert("TEST");</script>
I need to confirm the popup happened using either Mechanize or Nokogiri. Is it possible to confirm that the popup is there with Nokogiri or Mechanize?
For example:
def page(site)
Nokogiri::HTML(RestClient.get(site))
end
puts page('http://127.0.0.1:65412/v?=0.2<script>alert("TEST");</script>')
Nokogiri, and Mechanize because it is built on top of Nokogiri, can parse the HTML and return the <script> tag's contents. The tag's content is text so at that point it's necessary to look inside the text to find what you want:
require 'nokogiri'
doc = Nokogiri::HTML(<<EOT)
<html>
<head>
<script>alert("TEST");</script>
</head>
</html>
EOT
script_content = doc.at('script').content # => "alert(\"TEST\");"
It's easy to check to see if a sub-string exists at that point:
script_content['alert("TEST");'] # => "alert(\"TEST\");"
or:
!!script_content['alert("TEST");'] # => true
Note: It's not possible with Nokogiri, or Mechanize, to tell if a pop-up occurred as that'd happen inside a browser as it runs the JavaScript. Neither Nokogiri or Mechanize understand or interpret JavaScript. Only a tool like Watir or that interprets JavaScript could do that.
Definitely not, and that's because neither Mechanize or Nokogiri run Javascript.
Instead, you could use Selenium.
Something like this:
require 'selenium-webdriver'
class AlertChecker
Driver = Selenium::WebDriver.for :firefox
def initialize(url)
Driver.navigate.to url
end
def raise_alert(text)
Driver.execute_script "alert('#{text}')"
self
end
def safely_get_alert
begin
Driver.switch_to.alert
rescue Selenium::WebDriver::Error::NoAlertOpenError
end
end
end
Usage:
alert_checker = AlertChecker.new("http://my.website")
alert = alert_checker.safely_get_alert
# => nil
alert_checker.raise_alert("hack")
alert = alert_checker.safely_get_alert
puts alert.text
# => 'hack'
# As far as I'm aware Selenium doesn't have a built-in way
# to tell you if it's an alert, confirm, or prompt.
# But you know it's a prompt, for example, you could also send
# keys before accepting or dismissing
alert.accept
alert = alert_checker.safely_get_alert
# => nil
There are some tricky things with Selenium's handling of alerts, though.
There's no way for your code to detect the type (prompt, confirm, or alert) without using something like rescue or try. Everything is reached through switch_to.alert.
Also, if your browser has an alert open you cannot run any subsequent commands unless you handle alert. Say you try and navigate.to while the alert is open; you'd get an error along the lines of You didn't handle the alert and your navigate.to command would have to be rerun. When this error is raised, the alert object will be lost as well.
It's a little unappealing to use rescue as a control structure in this way but I'm not aware of any other option
I'm trying to get an example of the following code from github that looks to be a dead topic for my Linux/Ubuntu install. I have been trying to scrape data from my company intranet using "mechanize" see stack question for details. Since I'm not smart enough to figure a way around my login issue I thought I would try and feed data from an excel sheet as a work around until I can figure out the mechanize route. Once again I'm not smart enough to get the provided code to work on Linux because I'm getting the following error:
`kqueue=': kqueue is not supported on this platform (EventMachine::Unsupported)
If I'm understanding correctly from the information provided in the original source, the problem is that kqueue isn't supported in Linux. The OP states that inotify is an alternative but I've had no luck finding a similar example using it to display Excel in a widget.
Here is the code that is shown on GitHub and would like help converting it to work on Linux:
require 'roo'
EM.kqueue = EM.kqueue?
file_path = "#{Dir.pwd}/spreadsheet.xls"
def fetch_spreadsheet_data(path)
s = Roo::Excel.new(path)
send_event('valuation', { current: s.cell(1, 2) })
end
module Handler
def file_modified
fetch_spreadsheet_data(path)
end
end
fetch_spreadsheet_data(file_path)
EM.next_tick do
EM.watch_file(file_path, Handler)
end
Okay, so I was able to get this working and to display my data on a Dashing Dashboard widget by doing the following:
First: I uploaded my spreadsheet.xls to the root directory of my dashboard.
Second: I replaced the /jobs/sample.rb code with:
#!/usr/bin/env ruby
require 'roo'
SCHEDULER.every '2s' do
file_path = "#{Dir.pwd}/spreadsheet.xls"
def fetch_spreadsheet_data(path)
s = Roo::Excel.new(path)
send_event('valuation', { current: s.cell('B',49) })
end
module Handler
def file_modified
fetch_spreadsheet_data(path)
end
end
fetch_spreadsheet_data(file_path)
end
Third: Make sure the /widgets/number is in your dashboard "this is part of the sample install".
Fourth: Add the following code to your /dashboards/sample.erb file "this is part of the sample install as well".
<li data-row="1" data-col="1" data-sizex="1" data-sizey="1">
<div data-id="valuation" data-view="Number" data-title="Current Valuation" data-prefix="$"></div>
</li>
I used this source to help me better understand how Roo works. I tested my widget by changing my values and re-uploading the spreadsheet.xls to server and seen instant changes on my dashboard.
Hope this helps someone and I'm still looking for help to automate this process by scraping the data. Reference this if you can help.
Thanks for sharing this code sample. I did not manage to make it work in my environment (Raspberry/Raspbian) but after some efforts I managed to come up something that works -- at least for me ;)
I had never worked with Ruby before this week, so this code may be a bit crappy. Please accept apologizes.
-- Christophe
require 'roo'
require 'rubygems'
require 'rb-inotify'
# Implement INotify::Notifier.watch as described here:
# https://www.go4expert.com/articles/track-file-changes-ruby-inotify-t30264/
file_path = "#{Dir.pwd}/datasheet.csv"
def fetch_spreadsheet_data(path)
s = Roo::CSV.new(path)
send_event('csvdata', { value: s.cell(1, 1) })
end
SCHEDULER.every '5s' do
notifier = INotify::Notifier.new
notifier.watch(file_path, :modify) do |event|
event.flags.each do |flag|
## convert to string
flag = flag.to_s
puts case flag
when 'modify' then fetch_spreadsheet_data(file_path)
end
end
end
## loop, wait for events from inotify
notifier.process
end
I have a Sinatra app that wraps a command line application. It doesn't have may users, so performance is not a problem.
I am using Sinatra's streaming api to allow me to stream out the HTML as the command is run. This means the user gets to see progress as the command runs.
post "/reorder" do
#project = params["project"]
#id_or_range = params["id_or_range"]
#output_log = "[OUTPUT]"
before, after = slim(:index).split(#output_log)
stream do |out|
out << before
run(#project, #id_or_range, StreamOutput.new(out))
out << after
end
end
https://gist.github.com/NigelThorne/04775270abd46b78e262
Currently I am doing a hack where I render the template (as if I had all the data), then split the template text where the data should be inserted. I then render out the beginning of the template, then render the data as I receive it (on a stream), then the end of the template.
Slim is supposed to support streaming...
I'd like to write.
post "/reorder" do
...
stream do |out|
out << slim(:index)
end
end
or better
post "/reorder" do
...
slim(:index, stream: true)
end
How do I get slim to yield to the stream of data when rendering, so I stream out the template in one go?
Yes, you can if you overwrite the slim helper in Sinatra. See:
https://github.com/slim-template/slim/issues/540
https://github.com/slim-template/sinatra-stream
I've crafted a basic TCP client using EventMachine. Code:
# run.rb
EventMachine::run do
EventMachine::connect $config_host, $config_port, Bot
end
# bot.rb
module Bot
def post_init
# log us in and do any other spinup.
sleep(1)
send_data $config_login + "\n"
EventMachine.add_periodic_timer($config_keepalive_duration) { send_data $config_keepalive_str + "\n" }
#valid_command = /^<#{$config_passphrase}:([^>:]+):(#\d+)>(.*)$/
end
def receive_data(data)
if(ma = #valid_command.match(data))
command, user, args = ma[1,3]
args.strip!
command.downcase!
p "Received: #{command}, #{user}, #{args}"
# and.. here we handle the command.
end
end
end
This all works quite well. The basic idea is that it should connect, listen for specially formatted commands, and execute the command; in executing the command, there may be any number of "actions" taken that result in various data sent by the client.
But, for my next trick, I need to add the ability to actually handle the commands that Bot receives.
I'd like to do this using a dynamic library of event listeners, or something similar to that; ie, I have an arbitrary number of plugins that can register to listen for a specific command and get a callback from bot.rb. (Eventually, I'd like to be able to reload these plugins without restarting the bot, too.)
I've looked at the ruby_events gem and I think this makes sense but I'm having a little trouble of figuring out the best way to architect things. My questions include...
I'm a little puzzled where to attach ruby_events listeners to - it just extends Object so it doesn't make it obvious how to implement it.
Bot is a module, so I can't just call Bot.send_data from one of the plugins to send data - how can I interact with the EM connection from one of the plugins?
I'm completely open to any architectural revisions or suggestions of other gems that make what I'm trying to do easier, too.
I'm not sure exactly what you're trying to do, but one common pattern of doing this in EM is to define the command handlers as callbacks. So then the command logic can be pushed up out of the Bot module itself, which just handles the basic socket communication and command parsing. Think of how a web server dispatches to an application - the web server doesn't do the work, it just dispatches.
For example something like this
EM::run do
bot = EM::connect $config_host, $config_port, Bot
bot.on_command_1 do |user, *args|
# handle command_1 in a block
end
# or if you want to modularize it, something like this
# where Command2Handler = lambda {|user, *args| #... }
bot.on_command_2(&Command2Handler)
end
So then you just need to implement #on_command_1, #on_command_2, etc. in your Bot, which is just a matter of storing the procs to instance vars; then calling them once you parse the passed commands out of your receive_data.
A good, very readable example of this in production is TwitterStream.