is there a problem with doing? will there be some constraint on resources?
#main.rb
(1..100000).each do |loop|
`ruby dosomething.rb`
end
The reason i am doing this is because main.rb needs to be run in Jruby.
Somescript.rb runs faster using less resource in just Ruby, hence i am running it as shell command.
So starting up the ruby process, parsing the script, executing it and exiting 100,000 times is faster than importing the script into the loop under JRuby? Well, fine if you've measured that then there isn't too much wrong with what you're doing. But if you've only measured running the script once in JRuby and once under ruby (or maybe averaged 5 runs, not in loops of 100,000 times, then there may well be something wrong with what you're doing because you've partially compared the JRuby startup time to the ruby startup time, which wouldn't be a fair comparison since you must run JRuby and then ruby in what you've actually written.
From you comments it seems you're having trouble clearing the memory used by each run when run in JRuby. In that case, you might try a varient of running the loop in the external ruby if that handles the memory correctly it's better than starting up ruby 100,000 times.
#main.rb
`ruby dosomething.rb`
----
#dosomething.rb
(1..100000).each do |loop|
doingSomething
end
There's not much 'wrong' with doing it. It's not a great way to work around a memory bug and I fear for the environment into which you have to deploy. If you have to run in JRuby it seems like it's probably because your sysadmin doesn't want to have a build of Ruby MRI installed so requiring that to run is odd.
But yeah, if it works it works. I would talk to your sysadmin and make sure that it's cool to be running MRI as well as JRuby.
Related
I am running code shown in this question. I expected it to run faster second and third time (on first run it takes time to compile the code). However, it seems to be taking same amount of time as the first time. How can I make this code run faster?
Edit: I am running the code by giving command on Linux terminal: julia mycode.jl
I tried following instructions in the answer by #Przemyslaw Szufel but got following error:
julia> create_sysimage(["Plots"], sysimage_path="sys_plots.so", precompile_execution_file="precompile_plots.jl")
ERROR: MethodError: no method matching create_sysimage(::Array{String,1}; sysimage_path="sys_plots.so", precompile_execution_file="precompile_plots.jl")
Closest candidates are:
create_sysimage() at /home/cardio/.julia/packages/PackageCompiler/2yhCw/src/PackageCompiler.jl:462 got unsupported keyword arguments "sysimage_path", "precompile_execution_file"
create_sysimage(::Union{Array{Symbol,1}, Symbol}; sysimage_path, project, precompile_execution_file, precompile_statements_file, incremental, filter_stdlibs, replace_default, base_sysimage, isapp, julia_init_c_file, version, compat_level, soname, cpu_target, script) at /home/cardio/.julia/packages/PackageCompiler/2yhCw/src/PackageCompiler.jl:462
Stacktrace:
[1] top-level scope at REPL[25]:1
I am using Julia on Debian Stable Linux: Debian ⛬ julia/1.5.3+dfsg-3
In Julia packages are compiled each time they are run withing a single Julia session. Hence starting a new Julia process means that each time Plots.jl get compiled. This is quite a big package so will take a significant time to compile.
In order to circumvent it, use the PackageCompiler and compile Plots.jl into a static system image that can be used later by Julia
The basic steps include:
using PackageCompiler
create_sysimage(["Plots"], sysimage_path="sys_plots.so", precompile_execution_file="precompile_plots.jl")
After this is done you will need to run your code as:
julia --sysimage sys_plots.so mycode.jl
Similarly you could have added MultivariateStats and RDatasets to the generated sysimage but I do not think they cause any significant delay.
Note that if the subsequent runs are part of your development process (rather than your production system implementation) and you are eg. developing a Julia module than you could rather consider using Revise.jl in the development process rather than precompile the sysimage. Basically, having the sysimage means that you will need to rebuild it each time you update your Julia packages so I would consider this approach rather for production than development (depends on your exact scenario).
I had this problem and almost went back to Python but now I run scripts in the REPL with include. It is much faster this way.
Note: First run will be slow but subsequent runs in the same REPL session will be fast even if the script is edited.
Fedora 36, Julia 1.8.1
I have compiled and tested an open-source command line SIP client for my machine which we can assume has the same architecture as all other machines in our shop. By this I mean that I have successfully passed a compiled binary to others in the shop and they were able to use them.
The tool has a fairly esoteric invocation, a simple bash script piped to it prior to execution as follows:
(sleep 3; echo "# 1"; sleep 3; echo h) | pjsua sip:somephonenumber#ip --flag_1 val --flag_2 val
Note that the leading bash script is an essential part of the functioning of the program and that the line itself seems to be the best practice for use.
In the framing of my problem I am considering the following:
I don't think I can expect very many others in the shop to
compile the binary for themselves
Having a common system architecture in the shop it is reasonable to think that a repo can house the most up-to-date version
Having a way to invoke the tool using Ruby would be the most useful and the most accessible to the
most people.
The leading bash script being passed needs to be wholly extensible. These signify modifiable "scenarios" e.g. in this case:
Call
Wait three seconds
Press 1
Wait three seconds
Hang up
There may be as many as a dozen flags. Possibly a configuration file.
Is it a reasonable practice to create a gem that carries at its core a command line tool that has been previously compiled?
It seems reasonable to create a gem that uses a command line tool. The only thing I'd say is to check that the command is available using system('which psjua') and raising an informative error if it hasn't been installed.
So it seems like the vocabulary I was missing is extension. Here is a great stack discussion on wrapping up a Ruby C extension in a Ruby Gem.
Here is a link to the Gem Guides on creating Gems with Extensions.
Apparently not only is it done but there are sets of best practices around its use.
I have a ruby script, apparently correct, that sometimes stops working (probably on some calls to Postgresql through the pg gem). The problem is that it freezes but doesn't produce any error, so I can't see the line number and I always have to isolate the line by using puts "ok1", puts "ok2", etc. and see where the script stops.
Is there any better way to see the current line being executed (without changing the script)? And maybe the current stack?
You could investigate ruby-debug a project that has been rewritten several times for several different versions of ruby, should allow you to step through your code line by line. I personally prefer printf debugging in a lot of cases though. Also, if I had to take an absolutely random guess at your problem, I might investigate whether or not you're running into a race condition and/or deadlock in your DB.
I've been learning Ruby for the past few days, and I've run into a few issues concerning the implementation of threads. I've programmed in other languages before (mainly Java and C), and I still couldn't figure out what the issue is. I'm running ruby 2.1.2p95 on Ubuntu Server 14.10. The code in question is from Mr. Neighborly's Humble Little Ruby Book:
mate = Thread.new do
puts "Ahoy! Can I be dropping the anchor sir?"
Thread.stop
puts "Aye sir, dropping anchor!"
end
Thread.pass
puts "CAPTAIN: Aye, laddy!"
mate.run
mate.join
The output should be:
Ahoy! Can I be dropping the anchor sir?
CAPTAIN: Aye, laddy!
Aye sir, dropping anchor!
But instead, I'm receiving the following join and deadlock error:
CAPTAIN: Aye, laddy!
Ahoy! Can I be dropping the anchor sir?
ex2.rb:12:in `join': No live threads left. Deadlock? (fatal)
from ex2.rb:12:in `<main>'
I've run into errors with other threading examples from other resources as well, and have tried running the examples on other Ubuntu machines as well as trying Ruby 2.2. Is there a blatant concept that I'm missing out on? Has something changed in recent revisions of Ruby that would deem the examples out-of-date? Thank you for your help!
Has something changed in recent revisions of Ruby that would deem the examples out-of-date?
Yes. It looks like this book was written for Ruby 1.8, which used green threads. Ruby 1.9 onwards uses native threads (where the threads are scheduled by the OS).
Compare the documentation for the Thread.pass method in Ruby 1.8.7:
Invokes the thread scheduler to pass execution to another thread.
In Ruby 2.1.2 (the version you are using), this methods documentation looks like this:
Give the thread scheduler a hint to pass execution to another thread. A running thread may or may not switch, it depends on OS and processor.
So in current versions the scheduling is not deterministic in the way it was in Ruby 1.8.7, the OS is free to ignore the call to Thread.pass and run the main thread first, which causes the problems.
Running this script on my machine (Mac OS 10.9, Ruby 2.2.0) I get both results, sometimes it works and I see:
Ahoy! Can I be dropping the anchor sir?
CAPTAIN: Aye, laddy!
Aye sir, dropping anchor!
Other times it fails with:
CAPTAIN: Aye, laddy!
Ahoy! Can I be dropping the anchor sir?
capt-thread.rb:12:in `join': No live threads left. Deadlock? (fatal)
from capt-thread.rb:12:in `<main>'
I have a Perl script that runs via a system() command from C. On a specific site (SunOS 5.10), when that script is run, it nearly always takes 6 seconds or more. On other sites, it runs pretty much instantly (0.1s). If I run the script manually, i.e. not from the C code, it also runs instantly. I eventually tracked the slowness down (by spitting out the time a whole bunch in a lot of different places), to a single require line. The file that it is requiring is another Perl script we wrote. The script consists of a single require (this file here), 3 scalars that are assigned integer values, and a handful of time/date conversion routines. The file ends with a 1;. That single require appears to take as much as 6 seconds on occasion, but as I said, not always even on the same machine. I'm absolutely stumped here. My only last thought is to turn on profiling, but the site doesn't have Devel::Profiler and my only other option (that I know of) would be to add it to the Perl command which would require me altering and recompiling the C code (doable but non-trivial).
Anybody have ANY idea what could be going on here? I don't think I can/want to put the entire date.pl that is being required, but it's pretty much exactly as I described; I could answer any questions about it that you have.
Thanks in advance.
You might be interested in A Timely Start by Jean-Louis Leroy. He had a similar problem and tracked it down to a long and deep module search path where perl usually found the modules in the last entries in #INC.
Six seconds is a long time. Have you checked what your network is doing during this?
My first thought was that spawning the new process when using the system() command could be the problem, but six seconds is too long.
I don't know much about perl, but I could imagine that for any reason, the access of the time module could invoke a call to a network time server. Just to get synchronized. Maybe this takes so long or maybe it is getting a time out.
It could be that this only happens for a newly spawned process -- hence only when you use the system() command.
just wild guessing...
So, this does nothing to answer your question directly, but please tell me that you're not actually running on perl 4? Assuming you're on perl 5, you could remove the entire file and replace the require with use POSIX qw(ctime) to get the version that comes with Perl.
If you do have to support perl4, I'll merely grumble something about version 5 being fifteen years old now and go away. :)