[Note: I "fixed" this problem by creating and using a new gemset. I'm still curious why the problem occurred but it is no longer blocking me.]
[I am aware that there is a similar issue at Deadlock in Ruby join(), but I have tried the timeout parameter suggested there and it does not help. I suspect there is a pry-specific problem not covered there.]
I am getting the error below when running the code below, but only when executed within a pry session. This code has not been changed and has been working fine for quite a while, and I have no idea why it's an issue just now. I am using pry version 0.11.3 on Ruby 2.5.1. Also, this code works fine when pasted into pry; it's not working in my wifi-wand application that launches pry in the context of one of its objects (gem install wifi-wand to install, https://github.com/keithrbennett/wifiwand is the project page).
domains = %w(google.com baidu.com)
puts "Calling dig on domains #{domains}..." if #verbose_mode
threads = domains.map do |domain|
Thread.new do
output = `dig +short #{domain}`
output.length > 0
end
end
threads.each(&:join)
[1] pry(#<WifiWand::CommandLineInterface>)> ci
Calling dig on domains ["google.com", "baidu.com"]...
fatal: No live threads left. Deadlock?
3 threads, 3 sleeps current:0x00007fbd13d0c5a0 main thread:0x00007fbd13d0c5a0
* #<Thread:0x00007fbd14069c20 sleep_forever>
rb_thread_t:0x00007fbd13d0c5a0 native:0x00007fff89c2b380 int:0
/Users/kbennett/work/wifi-wand/lib/wifi-wand/models/base_model.rb:89:in `join'
/Users/kbennett/work/wifi-wand/lib/wifi-wand/models/base_model.rb:89:in `each'
/Users/kbennett/work/wifi-wand/lib/wifi-wand/models/base_model.rb:89:in `block in connected_to_internet?'
/Users/kbennett/work/wifi-wand/lib/wifi-wand/models/base_model.rb:126:in `connected_to_internet?'
/Users/kbennett/work/wifi-wand/lib/wifi-wand/command_line_interface.rb:264:in `cmd_ci'
Strangely, the problem was fixed by uninstalling and reinstalling the awesome_print gem. I have no idea why.
Related
Summary: After adding logic to save user account data, my code seems to work fine and sometimes all my (many) tests pass. But sometimes they fail seemingly randomly, with /tmp test files not being deleted during testing.
In my hand-rolled Ruby/Sinatra "to do list" program, I added user accounts and can now save data to user files (.yml format) as well as tmp files for people who aren't logged in. Yay!
As far as I can tell, the code works fine. All tests pass...but only sometimes. Sometimes, the tests related to my new file processing methods fail. Here's a sample:
# Running:
....EF..........................
Finished in 3.930466s, 8.1415 runs/s, 53.1744 assertions/s.
1) Error:
ToDoTest#test_post_newtask:
Errno::EACCES: Permission denied # unlink_internal - tmp/1.yml
C:/Users/user/Dropbox/_Programming/Ruby/learning_projects/todo/test/test_todo.rb:404:in `delete'
C:/Users/user/Dropbox/_Programming/Ruby/learning_projects/todo/test/test_todo.rb:404:in `block (2 levels) in teardown'
C:/Users/user/Dropbox/_Programming/Ruby/learning_projects/todo/test/test_todo.rb:404:in `each'
C:/Users/user/Dropbox/_Programming/Ruby/learning_projects/todo/test/test_todo.rb:404:in `block in teardown'
2) Failure:
ToDoTest#test_get_deleted [C:/Users/user/Dropbox/_Programming/Ruby/learning_projects/todo/test/test_todo.rb:167]:
Expected false to be truthy.
32 runs, 209 assertions, 1 failures, 1 errors, 0 skips
rake aborted!
Command failed with status (1): [ruby -I"lib" -I"C:/Ruby23/lib/ruby/gems/2.3.0/gems/rake-10.4.2/lib" "C:/Ruby23/lib/ruby/gems/2.3.0/gems/rake-10.4.2/lib/rake/rake_test_loader.rb" "test/test_task.rb" "test/test_task_store.rb" "test/test_todo.rb" "test/test_todo_helpers.rb" "test/test_users.rb" ]
Tasks: TOP => default => test
(See full trace by running task with --trace)
This is only a sample, because sometimes many more tests fail or have errors. It's weirdly random. I noticed that my tests, which result in a lot /tmp files being made and deleted very rapidly, sometimes failed to delete some files, and as a result some would be left behind. If I reran my tests when there were some undeleted files in /tmp, there would be even more (again, random) errors.
One common error I saw, which I never saw before adding the new file processing commands, is this one: Errno::EACCES: Permission denied # unlink_internal. I looked this up on SO but there seems to be only (irrelevant-seeming) Rails stuff. This is a Sinatra program running on Windows. So could I replicate the tests in my Ubuntu VM? Yes I could. Precisely the same sort of error pattern.
Anyway, I suspected that system commands were not finishing before execution continued. But apparently not. I tried putting "sleep 2" after all my system commands, and I still got a random failing test and cruft left in /tmp. I also tried using threads, which I have never used before, like this:
delr = Thread.new do
File.delete(#store.path) # seems to help to add this here...
end
delr.join
But that didn't help.
One other thing...I'm teaching myself and this is probably not the way it's supposed to be done, but...all of my get methods are preceded by a check of my session[:id] variable to see if the user is logged in, and to see if the correct datafile is loaded. I don't know if that's relevant but it might be.
Any ideas on what the problem could be or how to fix it?
here is my code:
class App < Sinatra::Base
get "/" do
pid =fork do
end
Process.detach(pid)
end
end
App.start!
when i curl localhost:4567, the server auto exit and the output is:
127.0.0.1 - - [13/Aug/2016:23:45:18 CST] "GET / HTTP/1.1" 200 0
- -> /
[2016-08-13 23:45:18] INFO WEBrick::HTTPServer#start done.
== Sinatra has ended his set (crowd applauds)
my environment is:
Linux Mint 17.3 Rosa
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
Why does Sinatra quit after just one request? I want it to keep running until I deliberately kill it.
It's not Process.detach that causes Sinatra to quit, but the fact that the forked process ends and runs its at_exit hooks, which shut down the server (though I'm not quite sure how that can leak from the child process to the parent).
There's a couple of simple ways to prevent this.
I prefer this set-and-forget solution:
configure do
disable :traps
end
The downside is that Ctrl-C causes a not-so-graceful exit.
Alternatively, add this to the beginning of any fork block:
settings.running_server = nil
The downside is that you need to remember to add this to all forked code.
You might also find someone suggesting at_exit { Process.exit! } in all fork blocks, but I think it's less reliable because other at_exit handlers could interfere with it.
Unfortunately, I don't see any configuration option in Sinatra that would fix forking and still allow graceful handling of Ctrl-C.
I'm experiencing an unresponsive socket in with my Puma setup after random time. Up to this point I don't have a clue what's causing the issue. I was hoping somebody over here can help we with some answers or point me in the right direction. I'm having the following setup:
I'm using the official docker ruby-2.2.3-slim image together with the latest puma release 2.15.3, I've also installed Nginx as a reverse proxy. But I'm already sure Nginx isn't the problem over here because and I've tried to verify if the socket was working using this script. And the socket wasn't working, I got a timeout over there as well so I could ignore Nginx.
This is a testing environment so the server isn't experiencing any extreme load, I've also check memory consumption it has still several GB's of free space so that couldn't be the issue either.
What triggered me to look at the puma socket was the error message I got in my Nginx error logging:
upstream timed out (110: Connection timed out) while reading response header from upstream
Also I couldn't find anything in the logs of puma indicating what is going wrong, over here are my puma setup:
threads 0, 16
app_dir = ENV.fetch('APP_HOME')
environment ENV['RAILS_ENV']
daemonize
bind "unix://#{app_dir}/sockets/puma.sock"
stdout_redirect "#{app_dir}/log/puma.stdout.log", "#{app_dir}/log/puma.stderr.log", true
pidfile "#{app_dir}/pids/puma.pid"
state_path "#{app_dir}/pids/puma.state"
activate_control_app
on_worker_boot do
require 'active_record'
ActiveRecord::Base.connection.disconnect! rescue ActiveRecord::ConnectionNotEstablished
ActiveRecord::Base.establish_connection(YAML.load_file("#{app_dir}/config/database.yml")[ENV['RAILS_ENV']])
end
And this it the output in my puma state file:
---
pid: 43
config: !ruby/object:Puma::Configuration
cli_options:
conf:
options:
:min_threads: 0
:max_threads: 16
:quiet: false
:debug: false
:binds:
- unix:///APP/sockets/puma.sock
:workers: 1
:daemon: true
:mode: :http
:before_fork: []
:worker_timeout: 60
:worker_boot_timeout: 60
:worker_shutdown_timeout: 30
:environment: staging
:redirect_stdout: "/APP/log/puma.stdout.log"
:redirect_stderr: "/APP/log/puma.stderr.log"
:redirect_append: true
:pidfile: "/APP/pids/puma.pid"
:state: "/APP/pids/puma.state"
:control_url: unix:///tmp/puma-status-1449260516541-37
:config_file: config/puma.rb
:control_url_temp: "/tmp/puma-status-1449260516541-37"
:control_auth_token: cda8879717be7a645ea323d931b88d4b
:tag: APP
The application itself is a Rails app on the latest version 4.2.5, it's deployed on GCE (Google Container Engine).
If somebody could give me some pointer's on how to debug this any further would be very much appreciated. Because now I don't see any output anywhere which could help me any further.
EDIT
I replaced the unix socket with tcp connection to Puma with the same result, still hangs after x time
I'd start with:
How many requests get processed successfully per instance of puma?
Make sure you log the beginning and end of each request with the thread id of the thread executing it, what do you see?
Not knowing more about your application, I'd say it's likely the threads get stuck doing some long/blocking calls without timeouts or spinning on some computation until the whole thread pool gets depleted.
We'll see.
I finally found out why my application was behaving the way it was.
After trying to use a tcp connection and switching to Unicorn I start looking into other possible sources.
That's when I thought maybe my connection to Google Cloud SQL could be the problem. Once I read the faq of Cloud SQL, they mentioned that you have to tweak you Compute instances to ensure they keep open your DB connection. So I performed the next steps they recommend and that solved the problem for me, I added them just in case:
# Display the current tcp_keepalive_time value.
$ cat /proc/sys/net/ipv4/tcp_keepalive_time
# Set tcp_keepalive_time to 60 seconds and make it permanent across reboots.
$ echo 'net.ipv4.tcp_keepalive_time = 60' | sudo tee -a /etc/sysctl.conf
# Apply the change.
$ sudo /sbin/sysctl --load=/etc/sysctl.conf
# Display the tcp_keepalive_time value to verify the change was applied.
$ cat /proc/sys/net/ipv4/tcp_keepalive_time
I'm facing same problem described here: why-rails-fails-with-ruby-exe-has-encountered-a-problem-and-needs-to-close. that is ruby get crashed with following error:
ruby.exe has encountered a problem and needs to close. We are sorry for the inconvenience.
I'm asking this to add some details on it as that question was not asked by me I am not able to add information in it. So this is not duplicate.
Ruby crashes eventually no matter how I run it (in development or in production), but it seems to crash more often in production mode. Sometimes it will crash when I hold down the F5 key, but sometimes I have to hold and release it intermittently for a minute or two. It appears to be very dependent on timing, but I can usually make it crash in less than 60 seconds.
When I refresh the GET request is sent for 3 times as follows:
Started GET "/app/page" for 127.0.0.1 at 2011-02-23 10:57:35 +0530
Processing by AppController#page as HTML
Rendered pms/dashboard.html.erb within layouts/application (109.4ms)
Completed 200 OK in 141ms (Views: 140.6ms | ActiveRecord: 0.0ms)
Started GET "/app/page" for 127.0.0.1 at 2011-02-23 10:57:35 +0530
Processing by AppController#page as */*
Rendered pms/dashboard.html.erb within layouts/application (15.6ms)
Completed 200 OK in 187ms (Views: 187.5ms | ActiveRecord: 0.0ms)
Started GET "/app/page" for 127.0.0.1 at 2011-02-23 10:57:35 +0530
Processing by AppController#page as */*
Rendered pms/dashboard.html.erb within layouts/application (15.6ms)
Completed 200 OK in 219ms (Views: 218.7ms | ActiveRecord: 0.0ms)
And if I rapidly refresh the page get following error in one of or all the 3 request:
ERROR Errno:ECONNABORTED: An established connection was aborted by the software in your host machine:
c:/Ruby/lib/1.9.1/webrick/httpresponse.rb:323:in 'write'
c:/Ruby/lib/1.9.1/webrick/httprespose.rb:323:in '<<'
c:/Ruby/lib/1.9.1/webrick/httprespose.rb:323:in '_write_data'
c:/Ruby/lib/1.9.1/webrick/httprespose.rb:295:in 'send_body_string'
c:/Ruby/lib/1.9.1/webrick/httprespose.rb:186:in 'send_body'
c:/Ruby/lib/1.9.1/webrick/httprespose.rb:103:in 'send_response'
c:/Ruby/lib/1.9.1/webrick/httpserver.rb:86:in 'run'
ERROR Errno:ECONNABORTED: An established connection was aborted by the software in your host machine:
c:/Ruby/lib/1.9.1/webrick/httpserver.rb:56:in 'eof?'
c:/Ruby/lib/1.9.1/webrick/httpserver.rb:56:in 'run'
c:/Ruby/lib/1.9.1/webrick/server.rb:183:in 'block in start_thread'
INFO going to shutdown....
INFO WEBrick::HTTPServer#start done
When I started to get this error?
When I added images, css and javascript in my project I started to get this error. I also tried by removing one of this 3 and see who causes the error but every time my ruby.exe get crashed and i see the error:
ruby.exe has encountered a problem and needs to close. We are sorry for the inconvenience.
How can I solve this?
When I start server following is printed on console:
=>Booting WEBrick
=>Rails 3.0.3 application starting in development on http://0.0.0.0:3000
=>Call with -d to detach
=>Ctrl-C to shutdown server
[2011-02-23 10:59:22] INFO WEBrick 1.3.1
[2011-02-23 10:59:22] INFO ruby 1.9.2 (2010-08-18) [i386-mingw32]
[2011-02-23 10:59:22] INFO WEbrick::HTTPServer#start: pid:2448 port=3000
Output of ruby -v
ruby 1.9.2p0 (2010-08-18) [i386-mingw32]
... except the installer is now ALSO the crashing 1.9.2-p290
Workaround:
Add (or change)
config.log_level = :warn
in config/environments/development.rb
(not my solution - found it in another thread)
Running 1.9.2 on windows puts you in a REALLY small minority of users. If you don't have an explicit need for 1.9, I'd use 1.8.7. Specifically, I'd use the package from RailsInstaller: http://railsinstaller.org/
i got the same crashes on windows 7 with 1.9.2-p290. someone said to clear development.log file. i can't believe but after deleting my 12mb development.log everything works fine.
it is possible that a native (C) extension is failing and bringing the whole ruby down; in my case it was sql server adapter looping indefinitely/using whole heap; after we traced and fixed the bug in the adapter (https://github.com/rails-sqlserver/tiny_tds/pull/124) it does not fail again
I am using the ruby daemons gem to create a custom daemon for my rails project. The only problem is that when I try to start the daemons ruby lib/daemons/test_ctl start that it fails and will not start. The log file has this output.
# Logfile created on Wed Oct 22 16:14:23 +0000 2008 by /
*** below you find the most recent exception thrown, this will be likely (but not certainly) the exception that made the application exit abnormally \*\*\*
# MissingSourceFile: no such file to load -- utf8proc_native
*** below you find all exception objects found in memory, some of them may have been thrown in your application, others may just be in memory because they are standard exceptions ***
# NoMemoryError: failed to allocate memory>
# SystemStackError: stack level too deep>
# fatal: exception reentered>
# LoadError: no such file to load -- daemons>
# LoadError: no such file to load -- active_support>
# MissingSourceFile: no such file to load -- lib/string>
# MissingSourceFile: no such file to load -- utf8proc_native>
It even happens when I generate a daemon (from the rails plugin) and try to run it. Does anybody know how to fix this problem?
OK, I actually found the answer to this problem. I require two custom files in the config/environment.rb. I used relative path names and because the daemons are executed in the rails main directory it could not find these two files. after making them absolute path it fixed the problem.
I just spent 30 minutes trying to solve a similar error when trying to get daemons plugin working:
LoadError: no such file to load -- active_support
For some reason, it wasn't finding active_support lib, even though it was installed. (Perhaps due to me having frozen rails).
In my case, the solution to this was to use the absolute path for active_support in my
ctl file (eg lib/daemons/mailer_ctl).
I needed to change line 5 from:
require 'active_support'
to
require './vendor/rails/activesupport/lib/active_support.rb'