I am trying to execute the inner double loop in parallel (in serial there is no problem):
DO J=1,NY
DO I=1,NX
A2=0.0d0
A1=S(I,J)
A2=dble(S(IP(I),J)+S(IM(I),J)+S(I,JP(J))+S(I,JM(J)))
EL=0.0d0
!$OMP parallel DO SHARED(LAMDA,S,COUL) PRIVATE(I1,J1) reduction(+:EL)
DO J1=1,NY
DO I1=1,NX
IF ((J1/=J.OR.I1/=I).AND.(J1/=J.OR.I1/=IP(I)).AND.(J1/=J.OR.I1/=IM(I)).AND.(J1/=JP(J).OR.I1/=I).AND.(J1/=JM(J).OR.I1/=I)) THEN
IF (ABS(FLOAT(I)-FLOAT(I1)) <= ABS(FLOAT(I)+LEN-FLOAT(I1))) THEN
ID= INT(ABS(FLOAT(I)-FLOAT(I1)))
ELSE
ID= INT(ABS(FLOAT(I)+LEN-FLOAT(I1)))
END IF
IF (ABS(FLOAT(J)-FLOAT(J1)) <= ABS(FLOAT(J)+LEN-FLOAT(J1))) THEN
JD= INT(ABS(FLOAT(J)-FLOAT(J1)))
ELSE
JD= INT(ABS(FLOAT(J)+LEN-FLOAT(J1)))
END IF
EL=EL + LAMDA*COUL(ID,JD)*dble(S(I1,J1))
END IF
END DO
END DO
!$OMP END PARALLEL DO
....
where COUL(ID,JD) is a matrix that has been computed earlier in the code, LAMDA a parameter and the matrices S, IP etc. have all known values.
When I compile it (Intel Fortran) I get the following weird errors:
error #7655: A variable is required in this OpenMP* context. [3.0D0]
error #7656: Subobjects are not allowed in this OpenMP* clause; a named variable must be specified.
Anyone has any idea about these errors? The funny thing is that I wrote a simple program just doing these loops, and it works fine in parallel with the above OpenMP statements.
Related
I want to run several jobs with Condor, my executable take as an argument b such that: b1=50+ $(($(Process)/41)), where $(())stands for the quotient of $(Process) divided by 41. b is defined in quotient.sh. Here is my submit file:
# Unix submit description file
include : PATH/quotient.sh
executable = PATH/script_test.sh
arguments = $(b) $(Process)
log = fit_it_data_$INT(b)_$(Process).log
output = outfile_fit_$INT(b)_$(Process).txt
error = errors_fit_$INT(b)_$(Process).txt
transfer_input_files = PATH
should_transfer_files = Yes
when_to_transfer_output = ON_EXIT
queue 81
However I am getting the error Submitting job(s)ERROR at Queue statement on Line 13: $INT() macro: 50+ $((0/41)) does not evaluate to an integer!. I don't understand why it complains that is does not evaluate to an integer, since b should be equal to 50 here...
Any idea how to fix that issue?
b1=50+ $(($(Process)/41))
I think you have an extra "$" in there. Try this:
b1=50+ ($(Process)/41)
In Julia, I want to use addprocs and pmap inside a function that is defined inside a module. Here's a silly example:
module test
using Distributions
export g, f
function g(a, b)
a + rand(Normal(0, b))
end
function f(A, b)
close = false
if length(procs()) == 1 # If there are already extra workers,
addprocs() # use them, otherwise, create your own.
close = true
end
W = pmap(x -> g(x, b), A)
if close == true
rmprocs(workers()) # Remove the workers you created.
end
return W
end
end
test.f(randn(5), 1)
This returns a very long error
WARNING: Module test not defined on process 4
WARNING: Module test not defined on process 3
fatal error on fatal error on WARNING: Module test not defined on process 2
43: : WARNING: Module test not defined on process 5
fatal error on fatal error on 5: 2: ERROR: UndefVarError: test not defined
in deserialize at serialize.jl:504
in handle_deserialize at serialize.jl:477
in deserialize at serialize.jl:696
...
in message_handler_loop at multi.jl:878
in process_tcp_streams at multi.jl:867
in anonymous at task.jl:63
Worker 3 terminated.
Worker 2 terminated.ERROR (unhandled task failure): EOFError: read end of file
WARNING: rmprocs: process 1 not removed
Worker 5 terminated.ERROR (unhandled task failure): EOFError: read end of file
4-element Array{Any,1}:Worker 4 terminated.ERROR (unhandled task failure): EOFError: read end of file
ERROR (unhandled task failure): EOFError: read end of file
ProcessExitedException()
ProcessExitedException()
ProcessExitedException()
ProcessExitedException()
What I'm trying to do is write a package that contains functions that perform operations that can be optionally parallelized at the user's discretion. So a function like f might take an argument par::Bool that does something like I've shown above if the user calls f with par = true and loops otherwise. So from within the definition of f (and within the definition of the module test), I want to create workers and broadcast the Distributions package and the function g to them.
What's wrong with using #everywhere in your function? The following, for example, works fine on my computer.
function f(A, b)
close = false
if length(procs()) == 1 # If there are already extra workers,
addprocs() # use them, otherwise, create your own.
#everywhere begin
using Distributions
function g(a, b)
a + rand(Normal(0, b))
end
end
close = true
end
W = pmap(x -> g(x, b), A)
if close == true
rmprocs(workers()) # Remove the workers you created.
end
return W
end
f(randn(5), 1)
Note: when I first ran this, I needed to recompile the Distributions package since it had been updated since I had last used it. When I first tried the above script right after the recompiling, it failed. But, I then quit Julia and reopened it and it worked fine. Perhaps that was what is causing your error?
I am using the msutter DSC module for puppet. While reading through the source code, I come across code like this (in dsc_configuration_provider.rb):
def create
Puppet.debug "\n" + ps_script_content('set')
output = powershell(ps_script_content('set'))
Puppet.debug output
end
What file defines the powershell function or method? Is it a ruby builtin? A puppet builtin? Inherited from a class? I know that it is being used to send text to powershell as a command and gathering results, but I need to see the source code to understand how to improve its error logging for my purposes, because certain powershell errors are being swallowed and no warnings are being printed to the Puppet log.
These lines in file dsc_provider_helpers.rb may be relevant:
provider.commands :powershell =>
if File.exists?("#{ENV['SYSTEMROOT']}\\sysnative\\WindowsPowershell\\v1.0\\powershell.exe")
"#{ENV['SYSTEMROOT']}\\sysnative\\WindowsPowershell\\v1.0\\powershell.exe"
elsif File.exists?("#{ENV['SYSTEMROOT']}\\system32\\WindowsPowershell\\v1.0\\powershell.exe")
"#{ENV['SYSTEMROOT']}\\system32\\WindowsPowershell\\v1.0\\powershell.exe"
else
'powershell.exe'
end
Surely this defines where the Powershell executable is located, but gives no indication how it is called and how its return value is derived. Are stdout and stderr combined? Am I given the text output or just the error code? etc.
This is core Puppet logic. When a provider has a command, like
commands :powershell => some binary
That is hooked up as a function powershell(*args).
You can see it with other providers like Chocolatey:
commands :chocolatey => chocolatey_command
def self.chocolatey_command
if Puppet::Util::Platform.windows?
# must determine how to get to params in ruby
#default_location = $chocolatey::params::install_location || ENV['ALLUSERSPROFILE'] + '\chocolatey'
chocopath = ENV['ChocolateyInstall'] ||
('C:\Chocolatey' if File.directory?('C:\Chocolatey')) ||
('C:\ProgramData\chocolatey' if File.directory?('C:\ProgramData\chocolatey')) ||
"#{ENV['ALLUSERSPROFILE']}\chocolatey"
chocopath += '\bin\choco.exe'
else
chocopath = 'choco.exe'
end
chocopath
end
Then other locations can just call chocolatey like a function with args:
chocolatey(*args)
I came across the Timeout module in Ruby, and wanted to test it out. I looked at their official source code at http://ruby-doc.org/stdlib-2.1.1/libdoc/timeout/rdoc/Timeout.html
Here is the code I had
require 'timeout'
require 'benchmark'
numbers = [*1..80]
Timeout::timeout(5) { numbers.combination(5).count }
=> 24040016
I did some benchmarking tests, and got the following.
10.828000 0.063000 10.891000 11.001676
According to the documentation, this method is supposed to return an exception if the block is not executed within 5 seconds. If it is executed within the time frame, it will return the result of the code block
For what it's worth, I've tried timeout with 1 second, instead of 5 seconds, and I still get returned the result of the code block.
Here is the official documentation
timeout(sec, klass=nil)
Performs an operation in a block, raising an error if it takes longer than sec seconds to complete.
sec: Number of seconds to wait for the block to terminate. Any number may be used,
including Floats to specify fractional seconds. A value of 0 or nil will execute the
block without any timeout.
klass: Exception Class to raise if the block fails to terminate in sec seconds. Omitting
will use the default, Timeout::Error
I am mystified as to why this doesn't work.
The problem is the way MRI (Matz's Ruby Implementation) thread scheduling works. MRI uses a GIL (Global Interpreter Lock), which in practice means only one thread is truly running at a time.
There are some exception, but for the majority of the time there is only one thread executing Ruby code at any one time.
Normally you do not notice this, even during heavy computations that consume 100% CPU, because the MRI keeps time-slicing the threads at regular intervals so that each thread gets a turn to run.
However there's one exception where time-slicing isn't active and that's when a Ruby thread is executing native C-code instead of Ruby code.
Now it so happens that Array#combination is implemented in pure C:
[1] pry(main)> show-source Array#combination
From: array.c (C Method):
static VALUE
rb_ary_combination(VALUE ary, VALUE num)
{
...
}
When we combine this knowledge with how Timeout.timeout is implemented we can start to get a clue of what is happening:
[7] pry(main)> show-source Timeout#timeout
From: /opt/ruby21/lib/ruby/2.1.0/timeout.rb # line 75:
75: def timeout(sec, klass = nil) #:yield: +sec+
76: return yield(sec) if sec == nil or sec.zero?
77: message = "execution expired"
78: e = Error
79: bl = proc do |exception|
80: begin
81: x = Thread.current
82: y = Thread.start {
83: begin
84: sleep sec
85: rescue => e
86: x.raise e
87: else
88: x.raise exception, message
89: end
90: }
91: return yield(sec)
92: ensure
93: if y
94: y.kill
95: y.join # make sure y is dead.
96: end
97: end
98: end
99: ...
1xx: end
Your code running Array.combination most likely actually starts executing even BEFORE the timeout thread runs sleep sec on line 84. Your code is launched on line 91 through yield(sec).
This means the order of execution actually becomes:
1: [thread 1] numbers.combination(5).count
# ...some time passes while the combinations are calculated ...
2: [thread 2] sleep 5 # <- The timeout thread starts running sleep
3: [thread 1] y.kill # <- The timeout thread is instantly killed
# and never times out.
In order to make sure the timeout thread starts first you can try this, which will most likely trigger the timeout exception this time:
Timeout::timeout(5) { Thread.pass; numbers.combination(5).count }
This is because by running Thread.pass you allow the MRI scheduler to start and run the code on line 82 before the native combination C-code executes. However even in this case the exception won't be triggered until combination exits because of the GIL.
There is no way around this unfortunately. You would have to use something like JRuby instead, which has real concurrent threads. Or you could run the combination calculation in a Process instead of a thread.
When following Fortran code is executed on the Intel Fortran Composer 2013 the compiler triggers a breakpoint at write function and retuns code 408:
character*20 date_char
character*10 LADATE
...
if (date_char(3:3) .EQ. "") date_char(3:3)="0"
if (date_char(7:7) .EQ. "") date_char(7:7)="0"
write(LADATE,"(2A2,A4)")
S date_char(3:4),date_char(7:8),date_char(9:12)
It is a fixed line-length format and the S represents the line continuation.
The date_char has a value of ' 29 012013 ' and the LADATE ' '
As soon as the write statement is reached the debugger triggers a breakpoint and the Call Stack shows following system functions being called:
for_issue_diagnostics()
_for_emit_diagnostics()
Your time is appreciated
The problem was that the LADATE variable was actually a call-by-reference argument (FORTRAN77 default passing convention):
SUBROUTINE MDATE(LADATE)
character*20 date_char
character*10 LADATE
...
write(LADATE,"(2A2,A4)")
S date_char(3:4),date_char(7:8),date_char(9:12)
RETURN
END
and it was passed as an argument several subroutines above as a just an 8-character string. Simply written, the call would be equivalent to:
...
CHARACTER VAR*20
...
CALL MDATE(VAR(10:17))
...
The program started, but after an attempt to access an inaccessible array addresses by the write function the breakpoint was triggered.