I'm benchmarking a "hello world" equivalent using sinatra and rack.
Command in question wrk -t12 -c400 -d30s: 12 threads, 400 open HTTP connections, 30 seconds.
Rack:
require 'rack'
app = Proc.new do |env|
['200', {'Content-Type' => 'text/html'}, ['A barebones rack app.']]
end
Rack::Handler::Thin.run app
# wrk $ wrk -t12 -c400 -d30s http://localhost:8080
# Running 30s test # http://localhost:8080
# 12 threads and 400 connections
# Thread Stats Avg Stdev Max +/- Stdev
# Latency 11.82ms 38.97ms 488.51ms 99.32%
# Req/Sec 705.04 568.62 2.20k 61.82%
# 16576 requests in 30.08s, 1.55MB read
# Socket errors: connect 157, read 274, write 0, timeout 0
# Requests/sec: 551.05
# Transfer/sec: 52.74KB
Sinatra:
require 'sinatra'
get '/' do
status 200
headers \
'Content-Type' => 'text/html'
'A barebones rack app.'
end
# wrk $ wrk -t12 -c400 -d30s http://localhost:4567
# Running 30s test # http://localhost:4567
# 12 threads and 400 connections
# Thread Stats Avg Stdev Max +/- Stdev
# Latency 40.12ms 90.46ms 1.39s 98.67%
# Req/Sec 265.47 147.50 1.17k 73.15%
# 90322 requests in 30.08s, 18.78MB read
# Socket errors: connect 157, read 333, write 0, timeout 0
# Requests/sec: 3002.52
# Transfer/sec: 639.21KB
Specs:
If both Rack and Sinatra run Thin, how come Sinatra manages 3002.52~ req/s while pure Rack manages only 551.05 req/s? What am I missing?
I didn't install wrk, I wasn't in the mood for an install, but I ran both your examples using time:
# run.sh
for i in {1..1000}
do
curl localhost:8080/ > /dev/null 2>&1
done
# sinatra, run via `bundle exec ruby sinatra.rb -p 8080`
$ time sh run.sh
sh run.sh 3.67s user 2.79s system 66% cpu 9.768 total
# rack, run via `bin/rackup config.ru`
$ time sh run.sh
sh run.sh 3.65s user 2.87s system 74% cpu 8.799 total
# sinatra with the puma server, by adding the line `set :server, "puma"`
$ time sh run.sh
sh run.sh 3.67s user 2.71s system 92% cpu 6.924 total
I got very similar results, so perhaps it's something to do with wrk or your system. I'm running OSX Mavericks, but I found this answer that says to benchmark with time you should really use chrt to avoid contention with other processes. I'm not sure what the OSX equivalent of chrt is though. Perhaps that's what is happening on your system, or perhaps the ports are running at different speeds, for whatever reason.
I also sandboxed my gems using bundle install --binstubs --path vendor.noindex, perhaps that would have an effect on your system too.
Otherwise, I can't see any reason for such disparity. I a little surprised by the results I got as I'd expect Sinatra to be quite a bit heavier than Rack, but perhaps things seem different with more complex or larger apps. Or perhaps not, Sinatra is such a neat little library.
Related
Using Ruby (tested with versions 2.6.9, 2.7.5, 3.0.3, 3.1.1) and forking processes to handle socket communication there seems to be a huge difference between MacOS OSX and a Debian Linux.
While running on Debian, the forked processes get called in a balanced manner - that mean: if having 10 tcp server forks and running 100 client calls, each fork will get 10 calls. The order of the pid call stack is also always the same even not ordered by pid (caused by load when instantiating the forks).
Doing the same on a MacOS OSX (Catalina) the forked processes will not get called balanced - that mean: "pid A" might get called 23 or whatever times while e.g. "pid G" was never used.
Sample code (originally from: https://relaxdiego.com/2017/02/load-balancing-sockets.html)
#!/usr/bin/env ruby
# server.rb
require 'socket'
# Open a socket
socket = TCPServer.open('0.0.0.0', 9999)
puts "Server started ..."
# For keeping track of children pids
wpids = []
# Forward any relevant signals to the child processes.
[:INT, :QUIT].each do |signal|
Signal.trap(signal) {
wpids.each { |wpid| Process.kill(:KILL, wpid) }
}
end
5.times {
wpids << fork do
loop {
connection = socket.accept
connection.puts "Hello from #{ Process.pid }"
connection.close
}
end
}
Process.waitall
Run some netcat to the server on a second terminal:
for i in {1..20}; do nc -d localhost 9999; done
As said: if running on Linux each forked process will get 4 calls - doing same on MacOS OSX its a random usage per forked process.
Any solution or correction to make it work on MacOS OSX in a balanced manner also?
The problem is that the default socket backlog size is 5 on MacOS and 128 on Linux. You can change the backlog size by passing it as the second argument to TCPServer#listen:
socket.listen(128)
Or you can use the backlog size from the environment variable SOMAXCONN:
socket.listen(ENV.fetch('SOMAXCONN', 128).to_i)
Usually, Ohai plugin runs periodically to collect some host parameters and some of the plugins usually added to all nodes in the company. This could be sensitive for resources using and how Ohai handle that. So I have two questions here.
The first one is what will happen if I will put infinite loop accidentally? Does Ohai/ruby has some max heap size or any memory limits?
Second question would be about shell out in Ohai. Is it possible to reduce timeout? Do you know more protections just in case?
I use only special ruby timeout for now:
require 'timeout'
begin
status = Timeout::timeout(600) {
# all code here
}
rescue Timeout::Error
puts 'timeout'
end
The chef-client run won't start/succeed, if ohai hangs.. you should notice this in some kind of monitoring.
Regarding the timeout part: Searching the source code reveals this:
def shell_out(cmd, **options)
# unless specified by the caller timeout after 30 seconds
options[:timeout] ||= 30
so = Mixlib::ShellOut.new(cmd, options)
So you should be able to set the timeout as you like (2 seconds in this case):
so = shell_out("/bin/your-command", :timeout => 2)
Regarding the third sub-question
Do you know more protections just in case?
you are getting pretty broad. Try to solve the problems that occur, stop over-engineering.
Just for the sake of completeness, Chef does not guard against broken or malicious Ohai plugins. If you put sleep 1 while true in your Ohai plugin it will happily sit there forever.
Seems I have found solution to limit Ohai resources for Redhat Linux in terms of CPU, disk space usage, disk space I/O, long run timeout and heap size memory limit. So you will not affect other host's components. In ideal world, you write optimised and right code, but memory leak is global problem and could happen so I think protections are needed especially when you have loaded Ohai plugin to hunders or tousands production servers.
CPU -
If I'm right Ohai plugin gets lowest cpu priority (-19?). Please confirm this if you know. So Ohai plugin cannot affect your production app in terms of CPU.
Disk space -
Ohai plugin should write to node attributes
Protection for unexpected long run -
require 'timeout'
begin
status = Timeout::timeout(600) {
# Ohai plugin code is here
}
rescue Timeout::Error
puts 'timeout'
end
Protection for unexpected long run of shell_out:
so = shell_out("/bin/your-command", :timeout => 30)
Memory (RAM) heap size limit -
require "thread"
# This thread is memory watcher. It works separately and does exit if heap size reached.
# rss is checked including childs but excluding shared memory.
# This could be ok for Ohai plugin. I'm assuming memory is not shared.
# Exit - if heap size limit reached (10 000 KB) or any unexpected scenario happened.
Thread.start {
loop do
sleep 1
current_memory_rss = `ps ax -o pid,rss | grep -E "^[[:space:]]*#{$$}"`.strip.split.map(&:to_i)[1].to_i
if current_memory_rss != nil && current_memory_rss > 0 && $$ != nil && $$.to_i > 0
exit if current_memory_rss > 10_000
else
exit
end
end
}
# Your Ohai code begins here
# For testing, any code can be included to make memory growing constantly as infinite loop:
loop do
puts `ps ax -o pid,rss | grep -E "^[[:space:]]*#{$$}"`.strip.split.map(&:to_i)[1].to_s + ' KB'
end
Please let me know if you have better solutions, but it seems it works.
Disk I/O read heavy usage -
timeout should help here, but recommended to avoid commands like find and similar others
Hi I want Thor to start a server - Jekyll / Python / PHP etc then open the browser
However the starting is a blocking task.
Is there a way to create a child process in Thor; or spawn a new terminal window - couldnt see and google gave me no reasonable answers.
My Code
##
# Project Thor File
#
# #use thor list
##
class IanWarner < Thor
##
# Open Jekyll Server
#
# #use thor ian_warner:openServer
##
desc "openServer", "Start the Jekyll Server"
def openServer
system("clear")
say("\n\t")
say("Start Server\n\t")
system("jekyll --server 4000 --auto")
say("Open Site\n\t")
system("open http://localhost:4000")
say("\n")
end
end
It looks like you are messing things up. Thor is in general a powerful CLI wrapper. CLI itself is in general singlethreaded.
You have two options: either to create different Thor descendents and run them as different threads/processes, forcing open thread/process to wait until jekyll start is running (preferred,) or to hack with system("jekyll --server 4000 --auto &") (note an ampersand at the end.)
The latter will work, but you still are to control the server is started (it may take a significant amount of time.) The second ugly hack to achieve this is to rely on sleep:
say("Start Server\n\t")
system("jekyll --server 4000 --auto &")
say("Wait for Server\n\t")
system("sleep 3")
say("Open Site\n\t")
system("open http://localhost:4000")
Upd: it’s hard to imagine what do you want to yield. If you want to leave your jekyll server running after your script is finished:
desc "openServer", "Start the Jekyll Server"
def openServer
system "clear"
say "\n\t"
say "Starting Server…\n\t"
r, w = IO.pipe
# Jekyll will print it’s running status to STDERR
pid = Process.spawn("jekyll --server 4000 --auto", :err=>w)
w.close
say "Spawned with pid=#{pid}"
rr = ''
while (rr += r.sysread(1024)) do
break if rr.include?('WEBrick::HTTPServer#start')
end
Process.detach(pid) # !!! Leave the jekyll running
say "Open Site\n\t"
system "open http://localhost:4000"
end
If you want to shutdown the jekyll after the page is opened, you are to spawn the call to open as well and Process.waitpid for it.
I have a Sinatra::Base webservice which I want to start from a command line Ruby program, so I have this:
# command line program file
require 'mymodule/server'
puts "Running on 0.0.0.0:4567, debugging to STDOUT..."
MyModule::Server.run! bind: '0.0.0.0', port: 4567, environment: :production
This works as expected but it throws out:
$ myscript
Running on 0.0.0.0:4567, debugging to STDOUT...
== Sinatra/1.3.1 has taken the stage on 4567 for production with backup from Thin
>> Thin web server (v1.3.1 codename Triple Espresso)
>> Maximum connections set to 1024
>> Listening on 0.0.0.0:4567, CTRL+C to stop
127.0.0.1 - - [23/Dec/2011 18:44:55] "POST /images HTTP/1.1" 200 360 0.0133
...
And I want it to be silent, and let me output what I want. For example, if I start it not daemonized I want to just see some message from the command line program and the log output, something like:
$ myscript
Running on 0.0.0.0:4567, debugging to STDOUT...
127.0.0.1 - - [23/Dec/2011 18:44:55] "POST /images HTTP/1.1" 200 360 0.0133
...
Also would like to silently shutdown it, hiding:
== Sinatra has ended his set (crowd applauds)
One last question, is this the best option to start a sinatra app with thin from inside an application code(ruby script in this case)?
You can turn off Sinatra logging with
set :logging, false
http://www.sinatrarb.com/configuration.html
As far as whether or not this is the best way to start a sinatra app... You might want to look at the "foreman" gem, and the "Procfile" (which Heroku.com uses) as an example:
http://ddollar.github.com/foreman/
Due to unicorn_rails complaining about different gem versions we moved to running bundle exec unicorn_rails... in our bluepill files. This change solved that particular problem and things start and stop but when we try sudo bluepill status we now get
unicorn(pix: XXXXXX): unmonitored
Which looks like bluepill is not monitoring the unicorn processes now. It will restart the child processes if I stop them but won't restart the parent process.
I've searched around but can't find much about this issue and was hoping someone could shed some light on it. The bluepill config file is
app_dir = "/opt/local/share/httpd/apps/xyz"
Bluepill.application('xyz', :log_file => "#{app_dir}/current/log/bluepill.log") do |app|
app.process('unicorn') do |process|
process.pid_file = "#{app_dir}/shared/pids/unicorn.pid"
process.working_dir = "#{app_dir}/current"
process.stdout = process.stderr = "#{app_dir}/shared/log/unicorn.err.log"
process.start_command = "bundle exec unicorn_rails -D -c #{app_dir}/current/config/environments/production/unicorn.rb -E production"
process.stop_command = "kill -QUIT {{PID}}"
process.restart_command = "kill -USR2 {{PID}}"
process.start_grace_time = 8.seconds
process.stop_grace_time = 5.seconds
process.restart_grace_time = 13.seconds
process.monitor_children do |child_process|
child_process.stop_command = "kill -QUIT {{PID}}"
child_process.checks :mem_usage, :every => 10.seconds, :below => 200.megabytes, :times => [3,5]
child_process.checks :cpu_usage, :every => 10.seconds, :below => 50, :times => [3,5]
end
end
end
When you run bundle exec, it sets up an environment and forks the unicorn_rails process. Bluepill ends up monitoring the original bundle exec process instead of unicorn which is why you see unmonitored.
I set up my bundler environment directly in bluepill and then execute unicorn_rails directly:
Bluepill.application('xyz') do |app|
app.environment = `env -i BUNDLE_GEMFILE=#{app_dir}/Gemfile bundle exec env`.lines.inject({}) do |env_hash,l|
kv = l.chomp.split('=',2)
env_hash[kv[0] = kv[1]
env_hash
end
app.process('unicorn') do |process|
process.start_command = "unicorn_rails -D -c #{app_dir}/current/config/environments/production/unicorn.rb -E production"
end
end
(Note: I omitted part of the above config file for clarity. Your config file looks good, just try adding the app.environment stuff above and removing bundle exec from your start command.)
This sets up the environment in bluepill by capturing the bundler environment variables using backticks, parsing the returned string into a hash and assigning it to app.environment.
I know this question is old, but I've been facing this problem for weeks. My suspicions were aroused when I realised that 'bluepill quit', followed by reloading the pill while unicorn was running allowed bluepill to consider the process 'up'.
#blt04's answer didn't help. Today I came to a realization. My grace start time of 8 seconds was not enough, because I had preload_app true in my unicorn config... and my app (Rails) takes 12 seconds to load, not 8.
Raising the start time to 30 seconds (wildly in excess of what was needed) solved the problem. Bluepill just says 'starting' for 30 seconds, then goes to 'up' correctly. Unicorn starts and runs as normal.
You'll want your restart time to be longer than it takes for Rails to start too.