So let's say pretty often a script runs that opens a browser and does web things:
require 'watir-webdriver'
$browser = Watir::Browser.new(:firefox, :profile => "botmode")
=> #<Watir::Browser:0x7fc97b06f558 url="about:blank" title="about:blank">
It could end gracefully with a browser.close, or it could crash sooner and leave behind the memory-hungry Firefox process, unnoticed until they accumulate and slow the server to a crawl.
My question is twofold:
What is a good practice to ensure that even in case of script failure anywhere leading to immediate error exit, the subprocess will always get cleaned up (I already have lots of short begin-rescue-end blocks peppered for other unrelated small tests)
More importantly, can I simply remember this Watir::Browser:0x7fc97b06f558 object address or PID somehow and re-assign it to another $browser variable in a whole new Ruby process, for example irb? I.e. can an orphaned browser on webdriver be re-attached later in another program using watir-webdriver on the same machine? From irb I could then get in and re-attach to the browser left behind by the crashed Ruby script, to examine the website it was on, check what went wrong, what elements are different than expected, etc.
Another hugely advantageous use of the latter would be to avoid the overhead of potentially hundreds of browser startups and shutdowns per day...best to keep one alive as sort of a daemon. The first run would attempt to reuse a previous browser object using my specially prepared botmode profile, otherwise create one. Then I would deliberately not call $browser.close at the end of my script. If nothing else I run an at job to kill the Xvfb :99 display FF runs inside of at the end of the day anyway (giving FF no choice but to die with it, if still running). Yes I am aware of Selenium standalone jar, but trying to avoid that java service footprint too.
Apologies if this is more a basic Ruby question. I just wasn't sure how to phrase it and keep getting irrelevant search results.
I guess, U cant just remember the variable from another process. But the solution might be creating a master process and process your script in loop in thread, periodically checking the browser running state. I'm using some thing similar in my acceptance tests on Cucumber + watir. So it will be some thing like that:
require 'rubygems'
require 'firewatir' # or watir
#browser = FireWatir::Firefox.new
t = Thread.new do
#browser.goto "http://google.com"
#call more browser actions here
end
while not_exit?
if t.stop?
# error occurred in thread, restart or exit
end
if browser_live?
# browser was killed for a some reason
# restart or exit
end
end
#browser.close
not_exit? - can be over TRAP for the ctrl+C
browser_live? - you can check if firefox browser running with processes listings
It is quite tricky but might work for you
You can use DRb like this:
browsers pool:
require 'drb'
require 'watir'
browser = Watir::Browser.new :chrome
DRb.start_service 'druby://127.0.0.1:9395', browser
gets
and then from test script use this browser:
require 'drb'
browser = DRbObject.new_with_uri 'druby://127.0.0.1:9395'
browser.goto 'stackoverflow.com'
I'm pretty sure that at the point ruby exits, any handles or pointers to something like a browser object would become invalid. So re-using something in a later ruby process is likely not a good approach. In addition I might be wrong on this, but it does seem that webdriver is not very good at connecting to a running browser process. So for your approach to work it would really all need to be wrapped by some master process that was calling all the tests etc.. and hey wait a sec, that's starting to sound like a framework, which you might already (or perhaps should be) using in the first place.
So a better solution is probably to look at whatever framework you are using to run your tests and investigate any capability for 'setup/teardown' actions (which can go by different names) which are run before and after either each test, groups of tests, or all tests. Going this way is good since most frameworks are designed to allow you to run any single test, or set of tests that you want to. And if your tests are well designed they can be run singly without having to expect the system was left in some perfect state by a prior test. Thus these sorts of setup/teardown actions are designed to work that way as well.
As an example Cucumber has this at the feature level, with the idea of a 'background' which is basically intended as a way to dry out scenarios by defining common steps to run before each scenario in a feature file. (such as navigating to and logging into your site) This could include a call to a series of steps that would look to see if a browser object existed, and if not create one. However you'd need to put that in every feature file which starts to become rather non dry.
Fortunately cucumber also allows a way to do this in one place via the use of Hooks. You can define hooks to run before steps, in the event of specific conditions, 'before' and 'after' each scenario, as well as code that runs once before any scenarios, and code defined to run 'at_exit' where you could close the browser after all scenarios have run.
If I was using cucumber I'd look at the idea of a some code in env.rb that would run at the start to create a browser, complemented by at_exit code to close the browser. Then perhaps also code in a before hook which could check to see that the browser is still there and re-create it if needed, and maybe logout actions in a after hook. Leave stuff like logging in for the individual scenarios, or a background block if all scenarios in a feature login with the same sort of user.
Not so much a solution but a workaround for part 1 of my question, using pkill. Posting here since it turned out to be a lot less trivial than I had hoped.
After the ruby script exits, its spawned processes (which may not at all belong in the same PID tree anymore, like firefox-bin) have a predictable "session leader" which turned out to be the parent of the bash shell calling rubyprogram.rb in my case. Available as $PPID in Bash, for when you have to go higher than $$.
Thus to really clean up unwanted heavyweight processes eg. after a ruby crash:
#!/bin/bash
# This is the script that wraps on top of Ruby scripts
./ruby_program_using_watirwebdriver_browser.rb myparams & # spawn ruby in background but keep going below:
sleep 11 # give Ruby a chance to launch its web browser
pstree -panu $$ # prints out a process tree starting under Bash, the parent of Ruby. Firefox may not show!
wait # now wait for Ruby to exit or crash
pkill -s $PPID firefox-bin # should only kill firefox-bin's caused above, not elsewhere on the system
# Another way without pkill, will also print out what's getting killed if anything:
awk '$7=="firefox-bin" && $3=="'$PPID'" {print $1}' <(ps x -o pid,pgid,sess,ppid,tty,time,comm) | xargs -rt kill
OPTIONAL
And since I use a dedicated Xvfb Xwindows server just for webdriving on DISPLAY :99, I can also count on xkill:
timeout 1s xwininfo -display :99 -root -all |awk '/("Navigator" "Firefox")/ {print $1}' |xargs -rt xkill -display :99 -id
# the timeout is in case xkill decides to wait for user action, when window id was missing
Just an update on part 2 of my question.
It seems one CAN serialize a Watir:Browser object with YAML, and because it's text-based the contents were quite interesting to me (e.g. some things I've only dreamed of tweaking hidden inside private elements of private classes...but that's a separate topic)
Deserializing from YAML is still trouble. While I haven't tested beyond the first try it gives me some kind of reg exp parse error...not sure what that's about.
(more on that at at how to serialize an object using TCPServer inside? )
Meanwhile, even attempting to serialize with Marshal, which is also built-in to Ruby but stores in binary format, results in a very reasonable-sounding error about not being able to dump a TCPServer object (apparently contained within my Watir:Browser pointed to by $browser)
All in all I'm not surprised at these results, but still pretty confident there is a way, until Watir arrives at something more native (like PersistentWebdriver or how it used to be in the days of jssh when you could simply attach to an already running browser with the right extension)
Until then, if serialization + deserialization to a working object gets too thorny I'll resort to daemonizing a portion of my Ruby to keep objects persistent and spare the frequent and costly setup/teardowns. And I did take a gander at some established (unit testing) frameworks but none seem to fit well yet within my overall software structure--I'm not web testing after all.
Related
I want to use ruby and in one moment open another program (cfdisk) and let the user interact with it, then return to my code
I can use
exec "cfdisk; ruby another.rb"
but that is very hacky and certainly not the right thing to do
I know about Open3 but i dont know how to use it properly for my usecase. I know that when I use capture3 on irb on Windows it says:
["Running under a virtual console, enabling 256/true-color support\n", "Screen size 28640x499 is not supported; limit 500x500.\nCheck the TERM environment variable.\n", #]
In Arch it opens it but the graphical output isnt present, the keystrokes are registered, they work (for example mc exits with F10) but the window isnt there, just blinking cursor.
parted exits immediately
Tried popen3 but I had no luck with it either
What is wrong with system for this case?
The exec ruby command replaces the running process, so it will not return to your code.
The Open3 library is used when you want to capture stdout and stderr.
Isn't this what you are looking for?
puts "here"
system "cfdisk"
puts 'there'
If you have some screen related issues, this is another issue that you might be able to resolve with different TERM value in the environment variable.
Whenever I drop a debugging breakpoint with byebug, I am noticing inconsistencies with the text that I type. For instance, I am able to type the first 2-3 characters. However, after the first 2-3 characters, the terminal starts adding random square brackets and other characters. For instance when I type env, Byebug outputs: eenv, or when I try to access a key in env Byebug outputs something even worse: ^[[Aenv['QUERY_STRING'']^[[.
Has anyone ran into this problem with Byebug?
A similar issue occurred to me while working in ruby on rails, using multiple workers. I am making the assumption here that OP is using rails because env and query strings are mentioned in the output.
So, if you are running multiple workers while debugging, they can cause this sort of behaviour, because code is still being executed after the byebug breakpoint is hit, and more importantly: while you try to type letters in the terminal, causing input to get jumbled. Debugging middleware can exacerbate this behaviour, since middleware is always hit, causing this issue to happen only while debugging certain components but not for an example controllers.
Ensure that only one worker is hitting your breakpoint at a time. This could be done by setting your development environment to never run more than 1 worker at least for the time being
This is a parallelization issue. Either with multiple workers while running rails server, OR in my case it was:
class ActiveSupport::TestCase
# Run tests in parallel with specified workers
parallelize(workers: :number_of_processors) # <--- this is a madness freaking root
end
byebug (and any bb-based gem) terminal input going nuts after hitting a break point in this case
I have a script named program.rb and would like to write a script named main.rb that would do the following:
system("ruby", "program.rb")
constantly check if program.rb is running until it is done
if program.rb has reached completion
exit main.rb
end
otherwise keep doing this until program.rb reaches completion{
if program.rb is not running and stopped before completing
restart program.rb from where it left off
end}
I've looked into Pidify but could not find a way to apply it to fit this exactly the right way...
Any help in how to approach this script would be greatly appreciated!
Update:
I could figure out how to resume running the script from where it left off in program.rb if there's no way to do it in main.rb
It's impossible to "restart script from where it left off" without full cooperation from the program.rb. That is, it should be able to advertise its progress (by writing current state to a file, maybe?) and be able to start correctly from a step specified in ARGV. There's no external ruby magic that can replace this functionality.
Also, if a program terminated abnormally, it means one of two things:
the error is (semi-)permanent (disk is full, no appropriate access rights to a file, etc). In this case, simply restarting the program would cause it to fail again. And again. Infinite fail loop.
the error is temporary (shaky internet connection). In this case, program should do better job with exception handling and retry on its own (instead of terminating).
In either case, there's no need for restarting, IMHO.
Well, here is one way.
Modify program.rb to take an optional flag argument --restart or something.
When program.rb starts up without this argument it will initialize a file to record its current state. Periodically, it will write whatever it needs into this file to record some kind of checkpoint.
When program.rb starts up with the restart flag, it will read its checkpoint file and start processing at that point. For this to work, it must either checkpoint all state changes or arrange for all processing between checkpoints to be idempotent so it can be repeated without ill effect.
There are lots of ways to monitor the health of program.rb. The best way is with some sort of ping, perhaps something like GET /health_check or a dummy message via a socket or pipe. You could just have a locked file to detect if the lock is still held, or you could record the PID on startup and check that it still exists.
I am writing an application, which shall work with networks.
As a GUI I am using rubyqt.
To determine if a Server is up I have to ping it (with net/ping).
But I ran in to a problem. If the server is down,
the GUI freezes for the timeout, even if I put the code in a Thread or IO.popen loop eg.
Thread.new('switch') do
if Net::PingExternal.new("195.168.255.244",timeout=0.9).ping then
down = false
else
down = true
end
end
will freeze for 0.9 seconds. As the QtThreads are not yet working with rubyqt,
does somebody have an idea to make the GUI don't freeze (apart from reducing the timeout)?
I was thinking about putting the pinging-part in an external program, which writes the status (up/down) in a file, which the actual program then reads, but this solution seems to be a bit clumsy.
Have you considered abstracting that operation from the request altogether? If you move the costly operation to an external library you could easily queue it up and execute it using something like delayed_job (http://github.com/tobi/delayed_job/tree/master) which would remove the risk of it halting the request at all.
Maybe this is what you are looking for...?
I've had some trouble forking of processes from a Perl CGI script when running on Windows. The main issue seems to be that 'fork' is emulated when running on windows, and doesn't actually seem to create a new process (just another thread in the current one). This means that web servers (like IIS) which are waiting for the process to finish continue waiting until the 'background' process finishes.
Is there a way of forking off a background process from a CGI script under Windows? Even better, is there a single function I can call which will do this in a cross platform way?
(And just to make life extra difficult, I'd really like a good way to redirect the forked processes output to a file at the same time).
If you want to do this in a platform independent way, Proc::Background is probably the best way.
Use Win32::Process->Create with DETACHED_PROCESS parameter
perlfork:
Perl provides a fork() keyword that
corresponds to the Unix system call of
the same name. On most Unix-like
platforms where the fork() system call
is available, Perl's fork() simply
calls it.
On some platforms such as Windows
where the fork() system call is not
available, Perl can be built to
emulate fork() at the interpreter
level. While the emulation is designed
to be as compatible as possible with
the real fork() at the the level of
the Perl program, there are certain
important differences that stem from
the fact that all the pseudo child
``processes'' created this way live in
the same real process as far as the
operating system is concerned.
I've found real problems with fork() on Windows, especially when dealing with Win32 Objects in Perl. Thus, if it's going to be Windows specific, I'd really recommend you look at the Thread library within Perl.
I use this to good effect accepting more than one connection at a time on websites using IIS, and then using even more threads to execute different scripts all at once.
This question is very old, and the accepted answer is correct. However, I just got this to work, and figured I'd add some more detail about how to accomplish it for anyone who needs it.
The following code exists in a very large perl CGI script. This particular sub routine creates tickets in multiple ticketing systems, then uses the returned ticket numbers to make an automated call via Twilio services. The call takes awhile, and I didn't want the CGI users to have to wait until the call ended to see the output from their request. To that end, I did the following:
(All the CGI code that is standard stuff. Calls the subroutine needed, and then)
my $randnum = int(rand(100000));
my $callcmd = $progdir_path . "/aoff-caller.pl --uniqueid $uuid --region $region --ticketid $ticketid";
my $daemon = Proc::Daemon->new(
work_dir => $progdir_path,
child_STDOUT => $tmpdir_path . '/stdout.txt',
child_STDERR => $tmpdir_path . '/stderr.txt',
pid_file => $tmpdir_path . '/' . $randnum . '-pid.txt',
exec_command => $callcmd,
);
my $pid = $daemon->Init();
exit 0;
(kill CGI at the appropriate place)
I am sure that the random number generated and attached to the pid is overkill, but I have no interest in creating issues that are extremely easily avoided. Hopefully this helps someone looking to do the same sort of thing. Remember to add use Proc::Daemon at the top of your script, mirror the code and alter to the paths and names of your program, and you should be good to go.