My pumaconf.rb has this line:
stdout_redirect '/var/log/puma/puma.log', '/var/log/puma/puma-err.log', true
puma.log contains:
=== puma startup: 2016-09-07 23:33:01 +0000 === [4433] - Worker 0 (pid: 4436) booted, phase: 0
puma-err.log contains:
10.10.10.1 - - [08/Sep/2016:00:03:27 +0000] "GET /health HTTP/1.0" 200 - 0.0005
10.10.10.1 - - [08/Sep/2016:00:03:27 +0000] "GET /health HTTP/1.0" 200 - 0.0005
Any tips on what I may have incorrectly setup?
Edit:
A little more info on my side -- I set up a new test in a clone of the puma code, with the following config:
log_requests
stdout_redirect "t4-stdout", "t4-stderr", true
pidfile "t4-pid"
bind "tcp://0.0.0.0:10103"
rackup File.expand_path('../hello.ru', File.dirname(__FILE__))
daemonize false
And wrote a test around "200 OK should only be in stdout, not in stderr" and everything is passing.
I'm pretty stumped why spinning up a puma server from the base repo is acting so differently, but I will keep digging.
Related
input:
generate:
mapping: root = {"id":48554}
pipeline:
processors:
- http:
url: https://example.com/**use_id_here_from_root**
verb: GET
retries: 5
timeout: 10s
retry_period: 2s
I tried using {{.id}}, eq.${! json("id") } but both of these dont seem to be working.
The http processor url field supports interpolation functions. You should be on the right track with ${! json("id") }, but there might be something wrong with how your test setup.
This config worked for me:
input:
generate:
count: 1
interval: 0s
mapping: root = {"id":48554}
pipeline:
processors:
- http:
url: http://127.0.0.1:6666/${! json("id") }
verb: GET
retries: 5
timeout: 10s
retry_period: 2s
output:
stdout: {}
To test it, I ran nc in a loop:
> while true; do echo -e "HTTP/1.1 200 OK\r\n" | nc -l 127.0.0.1 6666; done
then I ran benthos:
> benthos -c test.yaml
and I got the following output from nc:
GET /48554 HTTP/1.1
Host: 127.0.0.1:6666
User-Agent: Go-http-client/1.1
Content-Length: 12
Content-Type: application/octet-stream
Accept-Encoding: gzip
{"id":48554}
Trying to deploy my rails application to elastic beanstalk but I'm getting 502 errors. I'm running React on Rails and I've tried to follow this -> Rails application deployed on Elastic Beanstalk with Puma fails - 502 errors on every request but no luck. Any ideas?
Here are the logs
/var/logs/nginx/error.log
2020/01/09 04:11:45 [warn] 3083#0: conflicting server name "localhost" on 0.0.0.0:80, ignored
2020/01/09 04:20:04 [crit] 3087#0: *108 connect() to unix:///var/run/puma/my_app.sock failed (2: No such file or directory) while connecting to upstream, client: 172.31.44.209, server: _, request: "GET / HTTP/1.1", upstream: "http://unix:///var/run/puma/my_app.sock:/", host: "staging5.q39956drtr.ap-southeast-1.elasticbeanstalk.com"
2020/01/09 04:25:23 [crit] 3087#0: *185 connect() to unix:///var/run/puma/my_app.sock failed (2: No such file or directory) while connecting to upstream, client: 172.31.44.209, server: _, request: "GET / HTTP/1.1", upstream: "http://unix:///var/run/puma/my_app.sock:/", host: "staging5.q39956drtr.ap-southeast-1.elasticbeanstalk.com"
2020/01/09 04:25:24 [crit] 3087#0: *185 connect() to unix:///var/run/puma/my_app.sock failed (2: No such file or directory) while connecting to upstream, client: 172.31.44.209, server: _, request: "GET /favicon.ico HTTP/1.1", upstream: "http://unix:///var/run/puma/my_app.sock:/favicon.ico", host: "staging5.q39956drtr.ap-southeast-1.elasticbeanstalk.com", referrer: "http://staging5.q39956drtr.ap-southeast-1.elasticbeanstalk.com/"
Puma
healthd 2775 0.0 3.4 683640 34356 ? Ssl 04:11 0:04 puma 2.11.1 (tcp://127.0.0.1:22221) [healthd]
root 7841 1.5 0.2 58788 2612 ? Ss 05:56 0:00 su -s /bin/bash -c bundle exec puma -C /opt/elasticbeanstalk/support/conf/pumaconf.rb webapp
webapp 7940 43.0 1.1 70180 11460 ? Rs 05:56 0:00 /opt/rubies/ruby-2.6.5/bin/ruby /opt/rubies/ruby-2.6.5/bin/bundle exec puma -C /opt/elasticbeanstalk/support/conf/pumaconf.rb
root 7957 0.0 0.2 110516 2136 pts/0 S+ 05:56 0:00 grep --color=auto puma
pumaconf.rb
max_threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }
min_threads_count = ENV.fetch("RAILS_MIN_THREADS") { max_threads_count }
threads min_threads_count, max_threads_count
port ENV.fetch("PORT") { 3000 }
#
environment ENV.fetch("RAILS_ENV") { "development" }
# pidfile ENV.fetch("PIDFILE") { "tmp/pids/server.pid" }
bind "unix:///var/run/puma/my_app.sock"
pidfile "/var/run/puma/my_app.sock"
plugin :tmp_restart
I have a logfile
123.123.123.123 - - [26/Apr/2000:00:23:48 -0400] "GET /pics/wpaper.gif HTTP/1.0" 200 6248 "http://www.jafsoft.com/asctortf/" "Mozilla/4.05 (Macintosh; I; PPC)"
123.123.123.123 - - [26/Apr/2000:00:23:47 -0400] "GET /asctortf/ HTTP/1.0" 200 8130 "http://search.netscape.com/Computers/Data_Formats/Document/Text/RTF" "Mozilla/4.05 (Macintosh; I; PPC)"
123.123.123.123 - - [26/Apr/2000:00:23:48 -0400] "GET /pics/5star2000.gif HTTP/1.0" 200 4005 "http://www.jafsoft.com/asctortf/" "Mozilla/4.05 (Macintosh; I; PPC)"
123.123.123.123 - - [26/Apr/2000:00:23:50 -0400] "GET /pics/5star.gif HTTP/1.0" 200 1031 "http://www.jafsoft.com/asctortf/" "Mozilla/4.05 (Macintosh; I; PPC)"
123.123.123.123 - - [26/Apr/2000:00:23:51 -0400] "GET /pics/a2hlogo.jpg HTTP/1.0" 200 4282 "http://www.jafsoft.com/asctortf/" "Mozilla/4.05 (Macintosh; I; PPC)"
123.123.123.123 - - [26/Apr/2000:00:23:51 -0400] "GET /cgi-bin/newcount?jafsof3&width=4&font=digital&noshow HTTP/1.0" 200 36 "http://www.jafsoft.com/asctortf/" "Mozilla/4.05 (Macintosh; I; PPC)"
I want to calculate the requests per minute and output something like this.
2012/01/01 00:00 2
2012/01/01 00:01 33
I was thinking of looping over the whole file and extracting out the timestamps to an array, using regex like this.
File.open("log.txt") do |f|
f.each_line do |line|
timestamps << line[/\[(\d{2})\/([a-zA-Z]{3})\/(\d{4}):(\d{2}):(\d{2}) (\d{2})\s(-\d{4})]/]
end
Then using that array to somehow calculate the requests per second, is there a better way I can do this using ruby? Without using CLI tools.
It's not the prettiest, but this is what you're going to want to do.
require 'time'
TIMESTAMP_REGEX = %r{\[(.*?)\]} # extract everything between the []
datetimes = log.scan(TIMESTAMP_REGEX).flatten.map { |log_time| DateTime.parse(log_time.sub(":", ' ')) } # get the results from the regex and make an array of DateTime objects
results = Hash.new(0)
datetimes.each do |datetime|
time = datetime.strftime('%Y/%m/%d %H:%M')
results[time] += 1
end
results.each do |k,v|
puts "#{k}: #{v} requests"
end
There are more optimal ways to do this — including a moderately-lengthy one-liner — but if you're looking for straightforwardness this is the way to go.
This question already has answers here:
Save modifications in place with awk
(7 answers)
Closed 8 years ago.
I have a task to receive such information from access log file (in common log format) using unix command line:
Count # of requests with status_code == 200
There is a piece of that log file (wich includes data for 1 day):
127.0.0.1 "1.1.1.1" - [02/Dec/2014:14:30:00 +0000] "POST /server/ad?uid=abc&type=INV HTTP/1.1" 200 3966 7
127.0.0.1 "1.1.1.1" - [02/Dec/2014:14:32:30 +0000] "POST /server/ad?uid=abc&type=INV HTTP/1.1" 200 3966 8
127.0.0.1 "1.1.1.1" - [02/Dec/2014:15:20:12 +0000] "POST /server/ad?uid=abc&type=INV HTTP/1.1" 200 3966 8
127.0.0.1 "1.1.1.1" - [02/Dec/2014:15:22:20 +0000] "POST /server/ad?uid=abc&type=INV HTTP/1.1" 200 3966 8
127.0.0.1 "1.1.1.1" - [02/Dec/2014:15:30:10 +0000] "POST /server/ad?uid=abc&type=INV HTTP/1.1" 200 3966 8
127.0.0.1 "1.1.1.1" - [02/Dec/2014:15:35:15 +0000] "POST /server/ad?uid=abc&type=INV HTTP/1.1" 200 3966 7
127.0.0.1 "1.1.1.1" - [02/Dec/2014:16:25:11 +0000] "POST /server/ad?uid=abc&type=INV HTTP/1.1" 200 3966 7
127.0.0.1 "1.1.1.1" - [02/Dec/2014:16:27:10 +0000] "POST /server/ad?uid=abc&type=INV HTTP/1.1" 200 3966 8
127.0.0.1 "1.1.1.1" - [02/Dec/2014:16:33:12 +0000] "POST /server/ad?uid=abc&type=INV HTTP/1.1" 200 3966 10
I use this:
$ awk -F[:\ ] '{count[$5]++}; $12 == 200 { hour[$5]++} END { for (i in hour) print i, count[i] }' acces_log.log
And receive this:
14 2
15 4
16 3
But there is a small tip: all results should be stored in file. I wonder, how can I do this from command line.
Regards
All Linux/Unix and DOS command lines understand the symbols <, <<, >, >> for redirection.
To redirect output to a file, use
awk '{....}' > outputFile
#------------^ -- redirection
This redirection will always create a new outputFile, even if one exists already.
To append (extra) data to a file use
awk '{ .... }' >> outputFile
#--------------^^ -- append
IHTH
I want to run a script (code.rb) every time the url /code is hit.
How do I run the script?
require 'sinatra'
get '/' do
#run the script
end
Either fork off another process:
system('ruby code.rb')
... or simply load the script into the current context:
load 'code.rb' # *not* require
You need to load the code to ensure that it is run each time; require will only load the code once on the first request, and then not again:
smagic:Desktop phrogz$ cat hi.rb
puts "hi"
smagic:Desktop phrogz$ cat test.rb
require 'sinatra'
get '/require' do
x = require_relative( 'hi.rb' )
"require sez #{x}"
end
get '/load' do
x = load( 'hi.rb' )
"load sez #{x}"
end
smagic:Desktop phrogz$ ruby test.rb
== Sinatra/1.1.2 has taken the stage on 4567 for development with backup from Thin
>> Thin web server (v1.2.7 codename No Hup)
>> Maximum connections set to 1024
>> Listening on 0.0.0.0:4567, CTRL+C to stop
hi
127.0.0.1 - - [16/Jan/2011 20:49:43] "GET /require HTTP/1.1" 200 16 0.0019
127.0.0.1 - - [16/Jan/2011 20:49:46] "GET /require HTTP/1.1" 200 17 0.0005
hi
127.0.0.1 - - [16/Jan/2011 20:49:52] "GET /load HTTP/1.1" 200 13 0.0009
hi
127.0.0.1 - - [16/Jan/2011 20:49:54] "GET /load HTTP/1.1" 200 13 0.0008
127.0.0.1 - - [16/Jan/2011 20:50:09] "GET /require HTTP/1.1" 200 17 0.0005
127.0.0.1 - - [16/Jan/2011 20:50:12] "GET /require HTTP/1.1" 200 17 0.0005
The output of hi comes before the request entry in the log; you can see that the requests to 'require' only output hi the first time, while requests to 'load' show hi each time. Here's the output of the server, where true indicates that the code in the file was actually run and false shows that it was not:
smagic:~ phrogz$ curl http://localhost:4567/require
require sez true
smagic:~ phrogz$ curl http://localhost:4567/require
require sez false
smagic:~ phrogz$ curl http://localhost:4567/load
load sez true
smagic:~ phrogz$ curl http://localhost:4567/load
load sez true
smagic:~ phrogz$ curl http://localhost:4567/require
require sez false
smagic:~ phrogz$ curl http://localhost:4567/require
require sez false