Performance of classic sinatra and modular sinatra? am I doing wrong? - ruby

Question

UPDATED: OK, problem solved.
The different way I started the server makes the different result .
# this gives 2800 req/s in a production server, server based on thin
$ bundle exec thin start -R config.ru -e production
# this gives 1600 req/s in the same server, server based on Rack( seems that)
$ bundle exec rackup config.ru -s thin
so the ways of starting sinatra:
wrong: $ ruby main.rb (based on rack?)
wrong: $ rackup config.ru (based on rack)
wrong: $ rackup config.ru -s thin ( event based on rack)
correct: $ thin start -R config.ru -e production
------------------- Original question --------------------
Today I am coding Sinatra for an API application, and found that:
Classic sinatra code can process :
1.1 1800 request/s, with thin as the server.
1.2 2000 request/s, with puma as the server.
Modular sinatra code can only process:
2.1 1100 requests/s, with thin as the server
2.2 800 requests/s , with puma as the server.
How to reproduce this:
classic sinatra
# test_classic_sinatra.rb
require 'sinatra'
get '/' do
'hihihi'
end
run :
siwei $ ruby test.rb
== Sinatra (v2.0.5) has taken the stage on 4567 for development with backup from Thin
Thin web server (v1.7.2 codename Bachmanity)
Maximum connections set to 1024
Listening on localhost:4567, CTRL+C to stop
test:
$ ab -n 1000 -c 100 http://localhost:4567/
and got result:
Concurrency Level: 100
Time taken for tests: 0.530 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 211000 bytes
HTML transferred: 6000 bytes
Requests per second: 1885.43 [#/sec] (mean)
Time per request: 53.038 [ms] (mean)
Time per request: 0.530 [ms] (mean, across all concurrent requests)
Transfer rate: 388.50 [Kbytes/sec] received
Modular sinatra:
# config.ru
require 'sinatra/base'
class App < Sinatra::Application
set :environment, :production
get '/' do
'hihihi'
end
end
run App
with thin as server
run:
$ rackup config.ru -s thin
Thin web server (v1.7.2 codename Bachmanity)
Maximum connections set to 1024
Listening on localhost:9292, CTRL+C to stop
test:
Concurrency Level: 100
Time taken for tests: 0.931 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 211000 bytes
HTML transferred: 6000 bytes
Requests per second: 1073.58 [#/sec] (mean)
Time per request: 93.146 [ms] (mean)
Time per request: 0.931 [ms] (mean, across all concurrent requests)
Transfer rate: 221.22 [Kbytes/sec] received
with puma as server
run:
siwei$ rackup config.ru
Puma starting in single mode...
* Version 3.11.4 (ruby 2.3.8-p459), codename: Love Song
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://localhost:9292
Use Ctrl-C to stop
test:
$ab -n 1000 -c 100 http://localhost:9292/
got result:
Concurrency Level: 100
Time taken for tests: 1.266 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 178000 bytes
HTML transferred: 6000 bytes
Requests per second: 789.62 [#/sec] (mean)
Time per request: 126.643 [ms] (mean)
Time per request: 1.266 [ms] (mean, across all concurrent requests)
Transfer rate: 137.26 [Kbytes/sec] received
Before deciding to use Sinatra, I read many posts about "sinatra, grape and rails api", and I ran test agaist these frameworks, and finally decide to use Sinatra.
But now , I found Modular Sinatra seems not so good as expected. Could someone give me a clue about how to use "Classic Sinatra" or "Modular Sinatra" ?
If I don't use Modular Sinatra, how to write code for big applications?
thanks a lot!

Answer 1

Your benchmark is incorrect.
According to the snippets you posted, in the first case you use Thin as a server and in the second it's Puma.
These servers implement absolutely different concurrency models: as far as I remember it is a single-threaded event loop for the former and multiple threads for the latter. As a result, Thin performs better for light non-blocking tasks while Puma beats it in the scenarios with relatively heavy computations or blocking ones.
Your dummy example just better fits to the Thin's model and this causes the difference... Modular Sinatra should be absolutely fine, just compare apples to apples :)

Answer 2

OK, I found the root cause, and with The Tin Man's suggestion, I post my answer here.
The problem is the "web server" , but not the "framework"
The different way I started the server makes the different result .
# this gives 2800 req/s in a production server, server based on thin
$ bundle exec thin start -R config.ru -e production
# this gives 1600 req/s in the same server, server based on Rack( seems that)
$ bundle exec rackup config.ru -s thin
so the ways of starting sinatra:
wrong: $ ruby main.rb (based on rack?)
wrong: $ rackup config.ru (based on rack)
wrong: $ rackup config.ru -s thin ( event based on rack)
correct: $ thin start -R config.ru -e production