I have recently started studying parallel processing in Julia and I am having an issue which I don't really understand how to fix.
After having executed Julia with julia -p 4 I want to load the Distributions module in all the processes and I would like to define a Type which depends on Distributions.
The following apparently works correctly when I include it:
#everywhere using Distributions
type TypeDistrib{T <: Float64}
d::Distributions.Normal{T}
end
If I write exactly the same code as a module, then it doesn't:
module test
#everywhere using Distributions
type TypeDistrib{T <: Float64}
d::Distributions.Normal{T}
end
export TypeDistrib
end
The above gives the following error message:
ERROR: LoadError: UndefVarError: Distributions not defined
in include_from_node1(::String) at ./loading.jl:488
in include_from_node1(::String) at /Applications/Julia-0.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
while loading /***/test.jl, in expression starting on line 4
Would you please clarify what I am not doing correctly here?
Note: I replaced the full path in the error message with *** since it was confusing.
#everywhere something evaluates something in the Main module across all processes. And in this case, not in test module, and therefore the error in the second case and not in the first.
Perhaps
#everywhere module test
using Distributions
type TypeDistrib{T <: Float64}
d::Distributions.Normal{T}
end
export TypeDistrib
end
does what is meant.
Related
I'm trying to make some imported function available on all processes.
module EverywhereTest
#everywhere importall SubTest
function output()
test_print()
end
end
module SubTest
export test_print
function test_print()
print("hello")
end
end
On one process, this works without the #everywhere
using EverywhereTest
EverywhereTest.output()
hello
But with #everywhere I get UndefVarError: test_print not defined. Same behaviour with using instead of importall.
Does anyone know why this happens?
Working following the direction suggested after a recent question of mine (Parallel processing: #everywhere, distributions and types) I tried to set up a module using #everywhere, with a type depending on a package.
However, this arises a different question - as suggested in meta I created a new post. Let's say I have the following module (in test.jl):
#everywhere module test
using Distributions
type TypeDistrib{T <: Float64}
d::Distributions.Normal{T}
end
include('./folder1/fun1.jl');
export TypeDistrib, fun1
end
where fun1 is a function (in fun1.jl) and a simple script (in script.jl):
include("./testdir/test.jl");
using test
In addition, let's assume that these file are stored as follows:
-> example/
- script.jl
-> testdir/
- test.jl
-> folder1/
- fun1.jl
When I run it using example as my working dir without #everywhere, then it works with no errors. If I run it with #everywhere it says that the file fun1.jl doesn't exist.
I noticed that when I use #everywhere the relative path is linked with my working dir and not with the directory in which the module is stored. Am I missing something? Could you please clarify how I should set correctly the relative path using #everywhere - or more generally in case my settings are not accurate?
For MRI 1.9+ and Rubinius implementation, Ruby source code is compiled into byte-code, and then that byte-code is interpreted by the VM. I want to know the details of this mechanism when running a Ruby script from command-line using the interpreter.
Does the interpreter compile all related source files required in the script first and then it run everything? Or does it execute some code and then compile other files while needed in a lazy way?
If it's the latter (which I suspect), is this process done by file or by a block of code?
At which point it stops the execution of byte-code and runs the compilation process again?
Does this process differ from MRI to Rubinius?
For example, if I run "ruby my_main_script.rb", which requires 3 other rb source files (and this files itself do not have any requirement), the possibility I imagine would be:
A: The interpreter parses my_main_script.rb and the 3 files. After parsing then, it compiles all AST trees to byte-code. It then proceeds to run the byte-code using the VM.
B: Ruby parses my_main_script.rb and compiles it into byte-code. It then runs the byte-code. When encountering a call to a method in another files, it first parse and compiles that files and the continues with the execution. If this is the case, I would like to understand this in detail.
C: Ruby parses and compiles some piece of code from my_main_script.rb according to some (unkwnon to me) criteria, it runs that byte-code and then parses-and-compiles another piece when needed. This process and that "when needed" condition detection method is what would be interesting for me to understand.
Update 30/03/16
I wrote this little experiment script to try to check if B is the right answer:
class RubyVM
class InstructionSequence
class << self
alias :old_compile_file :compile_file
def compile_file(code, opt)
puts "Injecting code..."
old_compile_file(code, opt)
end
alias :old_compile :compile
def compile(code)
puts "Injecting code..."
old_compile(code)
end
end
end
end
require_relative 'say_hi'
'say_hi.rb' only contains the line "puts 'hello'".
If B is the right answer, shouldn't the output be the following?
Injecting code...
hello
It just outputs "hello"...
For me B is the right answer.
Ruby allows us to load dynamically our code via autoload and to execute strings as code (eval) so it must be able to parse and execute code at any time.
Therefore first it will transform all the files required by your main program to YARV instructions, but if you use autoload or eval those files/code will be transformed later.
A very good book about that process is Ruby under a microscope
I have a simple chef cookbook and all it does is it sets the MOTD on a CentOS machine. It takes the content of the /tmp/mymotd.txt and turns it into the MOTD.
I also have a simple ruby script (a full-fledged ruby script) that simply reads the text from the web-server and puts in into the /tmp/mymotd.txt.
My questions are:
how do I run this ruby script from within the cookbook?
how do I pass some parameters to the script (e.g. the address of the web-server)
Thanks a lot beforehand.
Ad 1.
You can use libraries directory in scripts to place there your ruby script and declare it in a module. Example:
# includes
module MODULE_NAME
# here some code using your script
# Example function
def example_function (text)
# some code
end
end
You can use then
include MODULE_NAME
in your recipe to import those functions and just use it like
example_function(something)
What's good - you can use there also Chef functions and resources.
IMPORTANT INFO: Just remember that Chef has 2 compilation phases. First will be all of Ruby code, second all of Chef resources. This means, that you have to remember priority of code. I won't write here more info about it, since you haven't asked for this, but if you want, you can find it here.
Ad 2.
You can do this in several ways, but it seems to me, that the best option for you would be to use environments. You can find more info in here. Basically, you can set up environment for script before it will run - this way you can define some variables you would use later.
Hope this helps.
Can I replace an executable (accessed via a system call from ruby) with an executable that expects certain input and supplies the expected output in a consistent amount of time? I'm mainly operating on Mac OSX 10.6 (Snow Leopard), but I also have access to Linux and Windows. I'm using MRI ruby 1.8.7.
Background: I'm looking at doing several DNA sequence alignments, one in each thread. When I try using BioRuby for this, either BioRuby or ruby's standard library's tempfile sometimes raise exceptions (which is better than failing silently!).
I set up a test that reproduces the problem, but only some of the time. I assume the main sources of variability between tests are the threading, the tempfile system, and the executable used for alignment (ClustalW). Since ClustalW probably isn't malfunctioning, but can be a source of variability, I'm thinking that eliminating it may aid reproducibility.
For those thinking select isn't broken - that's what I'm wondering too. However, according to the changelog, there was concern about tempfile's thread safety in August 2009. Also, I've checked on the BioRuby mailing list whether I'm calling the BioRuby code correctly, and that seems to be the case.
I really don't understand what the problem is or what exactly are you after, can't you just write something like
#!/bin/sh
#Test for input (syntax might be wrong, but you get the idea)
if [ $* ne "expected input" ]; then
echo "expected output for failure"
exit -1
fi
#have it work in a consistent amount of time
$CONSISTENT_AMOUNT_OF_TIME = 20
sleep $CONSISTENT_AMOUNT_OF_TIME
echo "expected output"
You can. In cases where I'm writing a functional test for program A, I may need to "mock" a program, B, that A runs via system. What I do then is to make program B's pathname configurable, with a default:
class ProgramA
def initialize(argv)
#args = ParseArgs(argv)
#config = Config.new(#args.config_path || default_config_path)
end
def run
command = [
program_b_path,
'--verbose',
'--do_something_wonderful',
].join(' ')
system(command)
...
end
def program_a_path
#config.fetch('program_b_path', default_program_b_path)
end
end
Program A takes a switch, "--config PATH", which can override the default config file path. The test sets up a configuration file in /tmp:
program_b_path: /home/wayne/project/tests/mock_program_b.rb
And passes to program A that configuration file:
program_a.rb --config /tmp/config.yaml
Now program A will run not the real program B, but the mock one.
Have you tried the Mocha gem? It's used a lot for testing, and you describe it perfectly. It "fakes" the method call of an object (which includes just about anything in ruby), and returns the result you want without actually running the method. Take this example file:
# test.rb
require 'rubygems'
require 'mocha'
self.stubs(:system).with('ls').returns('monkey')
puts system('ls')
Running this script outputs "monkey" because I stubbed out the system call. You can use this to bypass parts of an application you don't want test, to factor out irrelevant parts.