Disabling echo from webrick - ruby

How can I disable messages from webrick echoed on to the terminal? For the INFO messages that appear at the beginning, I was able to disable it by setting the Logger parameter so as:
s = WEBrick::HTTPServer.new(
Port: 3000,
BindAddress: "localhost",
Logger: WEBrick::Log.new("/dev/null"),
)
But I further want to disable the messages that look like:
localhost - - [17/Jun/2011:10:01:38
EDT] "GET .... HTTP/1.1" 200 0
http://localhost:3000/ -> .....
when a request is made from the web browser.

Following the link to the source and suggestion provied by Yet Another Geek, I was able to figure out a way. Set the AccessLog parameter to [nil, nil] [] (Changed following suggestion by Robert Watkins).
s = WEBrick::HTTPServer.new(
Port: 3000,
BindAddress: "localhost",
Logger: WEBrick::Log.new("/dev/null"),
AccessLog: [],
)

Related

Sinatra Ruby app one line log per request with custom info

I have one app that is a Sinatra app. I would like to have the log of each request on one line, Currently only 1 line per request is shown(the request itself). But I want to append custom info to it.
This is my current line:
[02/Feb/2018:11:52:32 +0000] "GET /feed/name.csv HTTP/1.0" 200 - 0.1620
2400:gh654:32:6532:0:0:a43e:404f, 0.0.0.0:2017 - - [05/Feb/2018:13:32:48 +0000] "HEAD / HTTP/1.0" 404 18 0.0676
2400:gh654:32:6532:0:0:a43e:414d, 0.0.0.0:2017 - - [05/Feb/2018:13:32:48 +0000] "HEAD / HTTP/1.0" 404 18 0.0041
I want to transform it in something like:
INFO 2018-03-02T14:03:36Z [main] [066843] [8a6e846f-45b7-433d-9d98-21c8c0766ad7] method=GET path= /some/path.csv format=csv action=action_name status=200 duration=459.97 view=154.94 db=172.96 user=user_id ip=user_id on=protocol_used agent=agent_used params=params_used
In the past to do this in rails applications I have used lograge but it seems it is not supported by sinatra apps.
Have you guys ever had to append custom info to the sinatra logs?
This is my configuration for logging:
configure do
enable :logging
file = File.new("#{RAILS_ROOT}/log/#{settings.environment}.log", 'a+')
file.sync = true
use Rack::CommonLogger, file
set :show_exceptions, true
set :logging, true
ActiveRecord::Base.logger = Logger.new("#{RAILS_ROOT}/log/#{settings.environment}.log")
end

CouchDB 3-node cluster (Windows) - multiple erlang errors

Im receiving multiple erlang errors in my CouchDB 2.1.1 cluster (3 nodes/Windows), see errors and node configuration below:
3 nodes (10.0.7.4 - 10.0.7.6), Azure application gateway is used as load balancer.
Why do these errors appear? system resources of the nodes are far from overload.
I would be thankful for any help - thanks in advance.
Errors:
rexi_server: from: couchdb#10.0.7.4(<0.14976.568>) mfa: fabric_rpc:changes/3 exit:timeout [{rexi,init_stream,1,[{file,"src/rexi.erl"},{line,256}]},{rexi,stream_last,2,[{file,"src/rexi.erl"},{line,224}]},{fabric_rpc,changes,4,[{file,"src/fabric_rpc.erl"},{line,86}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,139}]}]
rexi_server: from: couchdb#10.0.7.6(<13540.24597.655>) mfa: fabric_rpc:all_docs/3 exit:timeout [{rexi,init_stream,1,[{file,"src/rexi.erl"},{line,256}]},{rexi,stream2,3,[{file,"src/rexi.erl"},{line,204}]},{fabric_rpc,view_cb,2,[{file,"src/fabric_rpc.erl"},{line,308}]},{couch_mrview,finish_fold,2,[{file,"src/couch_mrview.erl"},{line,642}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,139}]}]
rexi_server: from: couchdb#10.0.7.6(<13540.5991.623>) mfa: fabric_rpc:all_docs/3 exit:timeout [{rexi,init_stream,1,[{file,"src/rexi.erl"},{line,256}]},{rexi,stream2,3,[{file,"src/rexi.erl"},{line,204}]},{fabric_rpc,view_cb,2,[{file,"src/fabric_rpc.erl"},{line,308}]},{couch_mrview,map_fold,3,[{file,"src/couch_mrview.erl"},{line,511}]},{couch_btree,stream_kv_node2,8,[{file,"src/couch_btree.erl"},{line,848}]},{couch_btree,fold,4,[{file,"src/couch_btree.erl"},{line,222}]},{couch_db,enum_docs,5,[{file,"src/couch_db.erl"},{line,1450}]},{couch_mrview,all_docs_fold,4,[{file,"src/couch_mrview.erl"},{line,425}]}]
req_err(3206982071) unknown_error : normal [<<"mochiweb_request:recv/3 L180">>,<<"mochiweb_request:stream_unchunked_body/4 L540">>,<<"mochiweb_request:recv_body/2 L214">>,<<"chttpd:body/1 L636">>,<<"chttpd:json_body/1 L649">>,<<"chttpd:json_body_obj/1 L657">>,<<"chttpd_db:db_req/2 L386">>,<<"chttpd:process_request/1 L295">>]
System running to use fully qualified hostnames ** ** Hostname localhost is illegal
COMPACTION-ERRORS
Supervisor couch_secondary_services had child compaction_daemon started with couch_compaction_daemon:start_link() at <0.18509.478> exit with reason {compaction_loop_died,{timeout,{gen_server,call,[couch_server,get_server]}}} in context child_terminated
CRASH REPORT Process couch_compaction_daemon (<0.18509.478>) with 0 neighbors exited with reason: {compaction_loop_died,{timeout,{gen_server,call,[couch_server,get_server]}}} at gen_server:terminate/7(line:826) <= proc_lib:init_p_do_apply/3(line:240); initial_call: {couch_compaction_daemon,init,['Argument__1']}, ancestors: [couch_secondary_services,couch_sup,<0.200.0>], messages: [], links: [<0.12665.492>], dictionary: [], trap_exit: true, status: running, heap_size: 987, stack_size: 27, reductions: 3173
gen_server couch_compaction_daemon terminated with reason: {compaction_loop_died,{timeout,{gen_server,call,[couch_server,get_server]}}} last msg: {'EXIT',<0.23195.476>,{timeout,{gen_server,call,[couch_server,get_server]}}} state: {state,<0.23195.476>,[]}
Error in process <0.16890.22> on node 'couchdb#10.0.7.4' with exit value: {{rexi_DOWN,{'couchdb#10.0.7.5',noproc}},[{mem3_rpc,rexi_call,2,[{file,"src/mem3_rpc.erl"},{line,269}]},{mem3_rep,calculate_start_seq,1,[{file,"src/mem3_rep.erl"},{line,194}]},{mem3_rep,repl,2,[{file,"src/mem3_rep.erl"},{line,175}]},{mem3_rep,go,1,[{file,"src/mem3_rep.erl"},{line,81}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,"src/mem3_sync.erl"},{line,208}]}]}
#vm.args
-name couchdb#10.0.7.4
-setcookie monster
-kernel error_logger silent
-sasl sasl_error_logger false
+K true
+A 16
+Bd -noinput
+Q 134217727`
local.ini
[fabric]
request_timeout = infinity
[couchdb]
max_dbs_open = 10000
os_process_timeout = 20000
uuid =
[chttpd]
port = 5984
bind_address = 0.0.0.0
[httpd]
socket_options = [{recbuf, 262144}, {sndbuf, 262144}, {nodelay, true}]
enable_cors = true
[couch_httpd_auth]
secret =
[daemons]
compaction_daemon={couch_compaction_daemon, start_link, []}
[compactions]
_default = [{db_fragmentation, "50%"}, {view_fragmentation, "50%"}, {from, "23:00"}, {to, "04:00"}]
[compaction_daemon]
check_interval = 300
min_file_size = 100000
[vendor]
name = COUCHCLUSTERNODE0X
[admins]
adminuser =
[cors]
methods = GET, PUT, POST, HEAD, DELETE
headers = accept, authorization, content-type, origin, referer
origins = *
credentials = true
[query_server_config]
os_process_limit = 2000
os_process_soft_limit = 1000

sinatra behind nginx resets the session

[Update at the bottom]
The problem is that after each reload of the page, regardless of what I put in the session, session gets 'reset'.
For example:
session[:test]="test"
puts session.inspect.to_s -> {"test"=>"test"}
(page reload)
puts session.inspect.to_s -> {"csrf"=>"b400efd6.....2362bd", "tracking"=>{"HTTP_USER_AGENT"=>"12a007....b", "HTTP_ACCEPT_LANGUAGE"=>"...."}}
In my nginx configuration I have:
proxy_pass http://localhost:9292;
In main.rb file I have:
configure do
(...)
set :sessions, key: '1234567',
path: '/',
expire_after: 86400,
secret: '7654321'
(...)
end
[Update]
It turns out that the response does not contain - set-cookie with the session id. Currently I am using CF->NGINX->Thin/Sinatra. Everything works fine without nginx.

Using Rack to run a React application is not working as it should

My idea is using Rack to load a web application separated in two parts. This is the structure:
my_app
|______api
| |______poros
|
|______fe
| |______build
| |______node_modules
| |______package.json
| |______public
| | |______ index.html
| | ...
| |______README.md
| |______src
| |______index.js
| ...
|______config.ru
The fe part (which stands for front-end) is a React application created with create-react-app. I've built it with
npm run build
and then its static optimized build is under my_app/fe/build directory.
This is my my_app/config.ru:
#\ -w -p 3000
require 'emeraldfw'
use Rack::Reloader, 0
use Rack::ContentLength
run EmeraldFW::Loader.new
And this is my EmeraldFW::Loader class, which is part of a gem installed and running fine.
module EmeraldFW
class Loader
def call(env)
path_info = (env['PATH_INFO'] == '/') ? '/index.html' : env['PATH_INFO']
extension = path_info.split('/').last.split('.').last
file = File.read("fe/build#{path_info}")
[200,{'Content-Type' => content_type[extension]},[file]]
end
def content_type
{
'html' => 'text/html',
'js' => 'text/javascript',
'css' => 'text/css',
'svg' => 'image/svg+xm',
'ico' => 'image/x-icon',
'map' => 'application/octet-stream'
}
end
end
end
As you may see, this is all quite simple. I capture the request with in my EmeraldFW::Loader class and transform its path_info it a bit. If the request is '/' I rewrite it to '/index.html' before doing anything else. In all cases I prepend fe/build to make it load from the static build of the React application.
When I run
rackup config.ru
and load the application at http://localhost:3000 the result is completely fine:
[2017-03-15 21:28:23] INFO WEBrick 1.3.1
[2017-03-15 21:28:23] INFO ruby 2.3.3 (2016-11-21) [x86_64-linux]
[2017-03-15 21:28:23] INFO WEBrick::HTTPServer#start: pid=11728 port=3000
::1 - - [15/Mar/2017:21:28:27 -0300] "GET / HTTP/1.1" 200 386 0.0088
::1 - - [15/Mar/2017:21:28:27 -0300] "GET /static/css/main.9a0fe4f1.css HTTP/1.1" 200 623 0.0105
::1 - - [15/Mar/2017:21:28:27 -0300] "GET /static/js/main.91328589.js HTTP/1.1" 200 153643 0.0086
::1 - - [15/Mar/2017:21:28:28 -0300] "GET /static/media/logo.5d5d9eef.svg HTTP/1.1" 200 2671 0.0036
::1 - - [15/Mar/2017:21:28:28 -0300] "GET /static/js/main.91328589.js.map HTTP/1.1" 200 1705922 0.1079
::1 - - [15/Mar/2017:21:28:28 -0300] "GET /static/css/main.9a0fe4f1.css.map HTTP/1.1" 200 105 0.0021
As you may see, all resources are loading correctly, with the correct mime types. But it happens that my default React logo, which should be spinning in my app frontpage, is not there! As if it wasn't loaded.
The big picture of all this is having this rack middleware loading the React front-end of the app and, at the same time, redirecting correctly the api requests made with fetch from the front-end to the poros (Plain Old Ruby Objects) which are the API part.
The concept is quite simple. But I just can't understand why this specific resource, the svg logo, is not loading.
It was all a matter of wrong mime type, caused by a typo.
It may be clearly seen in my question that I wrote:
def content_type
{
'html' => 'text/html',
'js' => 'text/javascript',
'css' => 'text/css',
'svg' => 'image/svg+xm', # <== Here is the error!!!
'ico' => 'image/x-icon',
'map' => 'application/octet-stream'
}
end
when the correct mime type for SVG files is 'image/svg-xml', with an 'l'...
This was making the browser ignore the file received and so it won't display it correctly.

Substring to hash key issue?

I have a log file and need to create a hash key for each URL in the record. Each line from the record has been placed into an array and I am looping through the array assigning hash keys.
I need to get from this:
"2010/08/23 15:25:35 [error]: (4: No such file or directory), clent: 80.154.42.54, server: localhost, request: "GET /logschecks/scripts/setup1.php HTTP/1.1", host: "www.example.com"
to this:
"/logschecks/scripts/setup1.php"
I have tried using match, scan and split but they have both failed to get me where I need to go.
My method currently looks like:
def pathHistogram (rowsInFile)
i = 0
urlHash = Hash.new
while i <= rowsInFile.length - 1
urlKey = rowsInFile[i].scan(/<"GET ">/).last.first
if urlHash.has_key?(urlKey) == true
#get the number of stars already in there and add one.
urlHash[urlKey] = urlHash[urlKey] + '*'
i = i + 1
else
urlHash[urlKey] = '*'
i = i + 1
end
end
end
I know that just scanning the "GET " won't complete the job but I was trying to baby-step through it. The match and split versions that I tried were fairly epic-fails, but I was likely using them incorrectly and they are long gone.
Running this script gives me an undefined method error on "first", though I have gotten other errors when I vary the way this is handled.
I should also say I am not married to using scan. If another method would work better, I would be more than happy to switch.
Any help would be greatly appreciated.
You state in a comment to the other answer the pattern is basically "GET ... HTTP, where you are interested in the ... part. That can be extracted very easily:
line = '2010/08/23 15:25:35 [error]: (4: No such file or directory), clent: 80.154.42.54, server: localhost, request: "GET /logschecks/scripts/setup1.php HTTP/1.1", host: "www.example.com"'
line[/"GET (.*?) HTTP/, 1]
# => "/logschecks/scripts/setup1.php"
Assuming each of your input lines contains /logschecks/...:
x = "2010/08/23 15:25:35 [error]: (4: No such file or directory), clent: 80.154.42.54, server: localhost, request: \"GET /logschecks/scripts/setup1.php HTTP/1.1\", host: \"www.example.com\""
x[%r(/logscheck[/\w\.]+)] # => "/logschecks/scripts/setup1.php"
Scanning HTTP logs isn't hard, but how you go about it will vary depending on the format. In the sample you're giving it's easier than a standard log because you have some landmarks you can look for:
Search for request: " using something like:
/request: "\S+ (\S+)/i
That pattern will skip over GET, POST, HEAD or whatever method was used for the request.
log_line[/request: "\S+ (\S+)/i, 1] # => "/logschecks/scripts/setup1.php"
You might want to know that if you're mining your logs. In that case...
Search for request: "[GET|POST|HEAD|...] using something like:
/request: "(\S+) (\S+)/i
You'd use it like:
method, url = log_line.match(/request: "(\S+) (\S+)/i).captures # => ["GET", "/logschecks/scripts/setup1.php"]
method # => "GET"
url # => "/logschecks/scripts/setup1.php"
You can also grab whatever is inside the double-quotes, then split it to get at the parts:
/request: "([^"]+)"/i
For instance:
log_line = %[2010/08/23 15:25:35 [error]: (4: No such file or directory), clent: 80.154.42.54, server: localhost, request: "GET /logschecks/scripts/setup1.php HTTP/1.1", host: "www.example.com"]
method, url, http_ver = log_line[/request: "([^"]+)"/i, 1].split # => ["GET", "/logschecks/scripts/setup1.php", "HTTP/1.1"]
method # => "GET"
url # => "/logschecks/scripts/setup1.php"
http_ver # => "HTTP/1.1"
Or use a bit more complex pattern, using some of the modern extensions and reduce the code:
log_line = %[2010/08/23 15:25:35 [error]: (4: No such file or directory), clent: 80.154.42.54, server: localhost, request: "GET /logschecks/scripts/setup1.php HTTP/1.1", host: "www.example.com"]
/request: "(?<method>\S+) (?<url>\S+) (?<http_ver>\S+)"/i =~ log_line
method # => "GET"
url # => "/logschecks/scripts/setup1.php"
http_ver # => "HTTP/1.1"

Resources