With N=P+Q compute nodes, can ipython send a message to the distinct R nodes and Q nodes at the same time and for each set of nodes to do some computation in parallel and for both set of nodes to return the result to the controller?
Yes, assuming direct view
from IPython.parallel import Client
c = Client()
c.ids
c[:].apply_sync(lambda : "Hello, World")
result in
['Hello, World',
'Hello, World',
'Hello, World',
'Hello, World',
'Hello, World',
'Hello, World',
'Hello, World',
'Hello, World']
Partitionning on even/odd engines
(
c[0::2].apply_sync(lambda : "Hello"),
c[1::2].apply_sync(lambda : "World")
)
result in
(['Hello', 'Hello', 'Hello', 'Hello'], ['World', 'World', 'World', 'World'])
here I apply sync, so it's not exactly at the same time, but if you apply async, you can collect at once after.
You can also push a variable on each set of node that will "decide" which computation to do.
Related
I am trying to limit the number of threads that must be executed simultaneously or to separate the elements of the array in a certain amount and send them separately (splice) but I have no idea where and how to start. The code is executing the Threads quickly and this is hindering many.
My code:
file = File.open('lista.txt').read
file.gsub!(/\r\n?/, "")
arr = []
file.each_line do |log|
arr << Thread.new{
puts login = Utils.req("GET","http://httpbin.org/GET",{
'Host': 'api.httpbin.org',
'content-type': 'application/json',
'accept': '*/*',
},
false)
}
end
arr.each(&:join)
If you want to limit how many threads are running concurrently, use a thread pool. This creates a "pool" of worker threads and limits the number of threads running at the same time. You can write your own, or use a gem like celluloid.
class Whatever
include Celluloid
def login_request
Utils.req(
"POST",
"http://httpbin.org/post",
{
'Host': 'api.httpbin.org',
'content-type': 'application/json',
'accept': '*/*',
},
false
)
end
end
# Only 5 threads will run at a time.
pool = Whatever.pool(size: 5)
file.each_line do |log|
split = log.split("|")
id = split[0]
number = split[1]
# If there's a free working in the pool, this will execute immediately.
# If not, it will wait until there is a free worker.
pool.login_request
end
You could use a thread pool. Here's a very basic one using Queue:
queue = Queue.new
pool = Array.new(5) do
Thread.new do
loop do
line = queue.pop
break if line == :stop
# do something with line, e.g.
# id, number = line.split('|')
# Utils.req(...)
end
end
end
It creates 5 threads and stores them in an array for later reference. Each thread runs a loop, calling Queue#pop to fetch a line from the queue. If the queue is empty, pop will suspend the thread until data becomes available. So initially, the threads will just sit there waiting for work to come:
queue.num_waiting #=> 5
pool.map(&:status) #=> ["sleep", "sleep", "sleep", "sleep", "sleep"]
Once the thread has retrieved a line, it will process it (to be implemented) and fetch a new one (or fall asleep again if there is none). If the line happens to be the symbol :stop, the thread will break the loop and terminate. (a very pragmatic approach)
To fill the queue, we can use a simple loop in our main thread:
file.each_line { |line| queue << line }
Afterwards, we push 5 :stop symbols to the end of the queue and then wait for the threads to finish:
pool.each { queue << :stop }
pool.each(&:join)
At this point, no threads are waiting for the queue – they all terminated normally:
queue.num_waiting #=> 0
pool.map(&:status) #=> [false, false, false, false, false]
Note that Queue isn't limited to strings or symbols. You can push any Ruby object to it. So instead of pushing the raw line, you could push the pre-processed data:
file.each_line do |line|
id, number = line.split('|')
queue << [id, number]
end
This way, your threads don't have to know about the log format. They can just work on id and number:
loop do
value = queue.pop
break if value == :stop
id, number = value
# do something with id and number
end
Instead of writing your own thread pool, you could of course use one of the various concurrency gems.
I'm not getting the consistent output in two cases:
Scenario 1:
humen = {"hand" => 1, "eye" => 2, "head" => 3, "hair"=>4}
puts "enter any body part name"
internal = gets.chomp.downcase.to_s
body = humen[internal]
puts body
#if input is "eye", it comes out 2
Scenario 2:
humen = {hand:1, eye:2, head:3, hair:4}
puts "enter any body part name"
internal = gets.chomp.downcase.to_s
body = humen[internal]
puts body
I see nothing in irb console. Can anyone please explain why that's the case?
keys are symbol in second case -
{:hand=>1, :eye=>2, :head=>3, :hair=>4}
whereas internal is a string.
humen[internal] is expecting the string assigned to internal to be present in hash humen which is not the case.
:hand != 'hand'
You should convert the string to symbol by:
humen[internal.to_sym]
String#to_sym converts a string into a symbol.
I am trying to make an interactive telnet client for Ruby. The current library is extremely lacking so I have been trying to add onto it by creating an interactive gem that allows a user to streamline data in real time with telnet. To do this I need to use multithreading:
t1 accepts user input. Users must always have capacity to input data through entire application. Once user data is sent we will receive data back right away which will be caught with our block { |c| print c }. The problem is that we want data to be streamlined to us. In other words, right now we only get data that is sent back after we send something, we want data to be received up to a minute after we sent a command. We want data to constantly be flowing to us.
I made t2 for this purpose. t2 waits for data to be received and then displays it when its regex pattern is matched. The problem with t2 is, if data is never received then the user cannot enter information into t1.
t3 operates on t1 and t2.
My Question is how can I organize my threads in such a way where the user can constantly type in console and submit commands, and simultaneously constantly receive information back from the server?
t1 = Thread.new {
while true
input = gets.chomp
localhost.cmd(input) { |c| print c }
end
}
t2 = Thread.new {
puts localhost.waitfor("Prompt" => /[$%#>:?.|](\e\[0?m\s*)* *\z/)
}
t3 = Thread.new {
t1.join
t2.join
}
t3.join
The problem is that we want data to be streamlined to us. In other
words, right now we only get data that is sent back after we send
something,
require 'thread'
user_data = Queue.new
t1 = Thread.new do
loop do
print "Enter data: "
line = gets.chomp
if line == ""
user_data << "END_OF_DATA"
break
else
user_data << line
end
end
end.join
t2 = Thread.new do
processed_data = []
loop do
line = user_data.shift
break if line == "END_OF_DATA"
processed_data << line
end
p processed_data
end.join
You might want to read this:
https://www.rfc-editor.org/rfc/rfc854
I am writing a command line script in ruby and I am trying to color a section of lines. Currently, I am using the 'colorize' gem, but from the documentation it only lets you color one line of text at a time
puts "test".colorize(:green)
puts "test".colorize(:green)
puts "test".colorize(:green)
But, that seems a bit redundant to me and I would like to color all the lines of text, but only call 'colorize(:green)' once and not 3 times.
How can this be done in Ruby?
Define a method for this:
def putsg(text)
puts text.colorize(:green)
end
And than call that method:
putsg "test"
putsg "test"
putsg "test"
puts ["test", "test", "test"].join($/).colorize(:green)
or
puts ["test", "test", "test"].map{|s| s.colorize(:green)}
Regarding your statement "...but from the documentation it only lets you color one line of text at a time"
You can colorize different parts of the same line with different colors.
s = "Hello"
ss = "world"
puts "#{s.red} #{"there".white} #{s.blue}"
You can also accomplish your aim like this:
s = "test"
puts "#{s}\n#{s}\n#{s}".green
Or:
s1 = "check"
s2 = "this"
s3 = "out"
puts "#{s1}\n#{s2}\n#{s3}".green
You can also define a method like this:
The input is the color you want as a string and the text you want outputed. This will work in the command prompt.
def colorized_text(color,text)
#Find colors here: https://en.wikipedia.org/wiki/ANSI_escape_code#8-bit
color_code_hash = {
'red' => 196,
'green' => 40,
'yellow' => 226,
'blue' => 27,
}
puts"\e[38;5;#{color_code_hash[color]}m#{text}\e[0m"
end
I found this to be much better than using the colorize gem since that only gives you 8 colors. You can find all the colors on the ansi escape wiki page under the 8 bit section.
I've got this as my code
openAll = File.open('N:\Josh\Blondie\db.txt')
allNumbers = Array.new
allNumbers=[]
openAll.each_line {|line|
allNumbers.push line
}
puts allNumbers
and I'd like to be able to display the output of this code in a new window with Ruby Shoes, I can't seem to get it to display anything though. The contents of the file are names and phone numbers.
Any ideas?
Here's an example of outputting text to a shoes window. Using a puts statement just outputs to the shell, not to the Shoes app.
Shoes.app :title => "GUI RAW file converter, for the CLI challenged",
:resizable => true do
background white
stack do
flow {
background gray, :height => 30
caption "Caption", :margin => 8, :stroke => white
stack {
para 'This is a fancy line I just printed to the window'
####### Here's an example line you could use to put out the array...
allNumbers.each do |number|
para "#{number}"
end
}
}
end
end
I guess you should use the method Kernel#alert instead of Kernel#puts.
http://shoesrb.com/manual/Built-in.html