Environment
Rails 3.2.11
Ruby 1.9.1
utf8-cleaner gem
I am struggling with this issue for over a year now, and I am not able to reproduce it on my dev environment, which makes it rather difficult for me to understand why this is happening and how I can resolve. Here is the error notification I am getting (via email using ExceptionNotifier):
A ArgumentError occurred in home#index:
invalid byte sequence in UTF-8
.bundle/gems/ruby/1.9.1/gems/rack-1.4.5/lib/rack/utils.rb:104:in `normalize_params'
Apparently caused by Chinese spider:
HTTP_USER_AGENT : Mozilla/5.0 (compatible; EasouSpider; +http://www.easou.com/search/spider.html)
I have tried to a couple things, see my earlier question where I was attempting to catch the error.
I have also installer utf8-cleaner gem but that doesn't seem to resolve, unless I missed a step.
How can I reproduce the issue? Note the URL causing the problem is perfectly correct when I access it (?)
UPDATE 20140721 - use rack-utf8_sanitizer
Add gem 'rack-utf8_sanitizer' in Gemfile
Add config.middleware.insert 0, Rack::UTF8Sanitizer in application.rb
$ bundle install
That worked perfectly on DEV but FAILED on my Heroku PRODUCTION, with the following issue:
$ heroku run rake middleware --a test-app
Running `rake middleware` attached to terminal... up, run.4846
WARNING: Nokogiri was built against LibXML version 2.8.0, but has dynamically loaded 2.7.6
rake aborted!
uninitialized constant Rack::UTF8Sanitizer
/app/config/application.rb:71:in `<class:Application>'
I am still investigating why I should be getting this.
I managed to fix it (on a Rails 3.2.18 app) as described in this gist:
https://gist.github.com/joost/ca4eda8f31655cf6095a
Reproduce the issue caused by
HTTP_USER_AGENT : Mozilla/5.0 (compatible; EasouSpider; +http://www.easou.com/search/spider.html)
Create a ruby script
#!ruby
invalid = "data\xed\xe5\xed\xe0".force_encoding('ASCII-8BIT')
`curl localhost:3000 -d #{invalid}`
Add the rack-utf8_sanitizer gem to your Gemfile
This resolved for my Dev environment, but wouldn't work on Heroku. I updated my question accordingly.
UPDATE:
I added require "rack/utf8_sanitizer" in my application.rb file and that seems to resolve, the Heroku issue.
Related
I'm making a basic Sinatra app to play around with the Passbook gem.
My app runs fine on localhost, but when I try and run it on Heroku I get this error message:
/app/vendor/bundle/ruby/2.0.0/gems/passbook-0.2.1/lib/passbook/pkpass.rb:122:in `outputZip'
/app/vendor/bundle/ruby/2.0.0/gems/passbook-0.2.1/lib/passbook/pkpass.rb:60:in `stream'
/app/vendor/bundle/ruby/2.0.0/gems/sinatra-1.4.4/lib/sinatra/base.rb:1593:in `call'
NoMethodError - undefined method `write_buffer' for Zip::ZipOutputStream:Class:
This happens when I call the line passbook.stream.string in my app file.
I don't think Passbook is the problem here - it seems that the class method write_buffer exists in the version of Zip::ZipOutputStream on my local machine but isn't there on Heroku. Why is this? How can I get around it?
I've tried including both zip and rubyzip in my Gemfile, individually and at the same time, and neither of them solve the issue.
Whenever I see something like this, I always ask myself Do you have Dev/Prod parity?.
So:
What Ruby version do you use locally, and does it differ from what you're using on Heroku?
Do you specify a Ruby version in your Gemfile?
How do you run locally? Do you use bundle exec to ensure you run in the same way, with the same gems?
Finally figured it out.
When I ran ruby app.rb, things worked locally, but when I ran bundle exec ruby app.rb, I got the same error as I did on heroku.
I upgraded to the latest version of passbook (which was just updated today to handle the latest changes to rubyzip), but things still didn't work.
Turns out I needed to remove gem 'zip' from my Gemfile and just include gem 'rubyzip' - previously I had both.
I am a total noob to Ruby and Rails, so any help is appreciated about this.
I installed Ruby 2.0.0 using RVM. It got most of the way through, when I got this error message:
Error running 'env GEM_PATH=/Users/victoriamielke/.rvm/gems/ruby-2.0.0-p0:/Users/victoriamielke/.rvm/gems/ruby-2.0.0-p0#global:/Users/victoriamielke/.rvm/gems/ruby-2.0.0-p0:/Users/victoriamielke/.rvm/gems/ruby-2.0.0-p0#global GEM_HOME=/Users/victoriamielke/.rvm/gems/ruby-2.0.0-p0 /Users/victoriamielke/.rvm/rubies/ruby-2.0.0-p0/bin/ruby -d /Users/victoriamielke/.rvm/src/rubygems-2.0.3/setup.rb --verbose',
please read /Users/victoriamielke/.rvm/log/ruby-2.0.0-p0/rubygems.install.log
Installation of rubygems did not complete successfully.
I also opened up rubygems.install.log, and it said near the end:
ERROR: While executing gem ... (NoMethodError)
undefined method `fu_stream_blksize' for #<Gem::Commands::SetupCommand:0x007fcab29f5838>
What caused the failure of Rubygems to install?
Another problem am having is the error message ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2). I tried a Google search on this, which said I needed to find the mysql.sock file. Well, I have never found this file on my computer and could not find out in plain English how to add it or generate it.
Looks like the installation error is a recent known issue. If you're a beginner I recommend using Ruby 1.9.3, since 2.0.0 is brand new and you'll certainly find bugs in both the language implementation and libraries that haven't been tested on 2.0.0 yet.
The second issue is unrelated. It could be that you haven't started MySQL (did you run mysql.server start?), file permissions, MySQL configuration, or any number of other issues. You're not supposed to create /tmp/mysql.sock yourself, it's created automatically by MySQL. Please open a separate question about it.
I stood up a Rack server with Geminabox, running on my machine at http://localhost:9292. Now I was trying to upload the gem to the server (from a different tab on the terminal acting like a client), but when I type:
gem sources -a http://localhost:9292
I get the following error:
Error fetching http://localhost:9292:
bad response Not Found 404 (http://localhost:9292/specs.4.8.gz)
There is a trailing colon and I'm a bit lost, any help on that?
Thanks!
You need to upload at least one gem before Geminabox starts to serve the necessary files.
(I know this is an old question but I came here from google so others might as well)
Here's the stack: Sproutcore 1.0.1046. Ruby 1.9.1, in RVM. Thin 1.2.7. Thor 0.13.8. Rack 1.2.1. Eventmachine 0.12.10. Erubis 2.6.6.
When I start the sc-server on any application, my first request to this server produces this in the console log:
ArgumentError: invalid byte sequence in US-ASCII
...followed by this stack trace. (I've listed gems which appear in the stack trace above, but there's a complete gemset list in the same gist as the stack trace.)
Research on the error message points out that this is a common problem with Ruby 1.9, but the stack trace suggests that the problem is in one of the gems somewhere.
I have:
Upgraded my OS (Mac OS X 10.5 to 10.6) in order to get the latest gcc Apple provides.
Reinstalled RVM.
Reinstalled Ruby.
Reinstalled all the relevant gems.
And yet I still have this problem on one system, but not on another. (N.B. there are several devs working on this code, and I'm the only one seeing this problem. I'm 99% certain it's not our code.) I guess what I'm saying is that I've cleared and rebuilt a lot of gems to try to isolate or remove this glitch, and yet I still haven't gotten rid of it.
Where should I look next?
You have some special setting in your bash environment that is setting ruby to use US-ASCII , this happened to me trying to execute sc-server from a remote terminal... I'm not really sure what it is, but it doesn't use UTF-8 and that's when it runs into trouble.
You can probably also change Encoding.default_external
Thanks so much for the Encoding.default_external suggestion. I was having the same problem, despite correctly set magic comments and environment variables. In Rails 2.3.9 I added this before_filter in application_controller.rb, resolved the issue:
def set_encoding
Encoding.default_external = 'UTF-8'
end
I had the same issue with RSS Feeds that I was displaying in a Rails 2.3.8 app with Ruby 1.9.2. My issue wasn't solved by the application_controller.rb before_filter technique mentioned here. The fix was to put the following into "RAILS_ROOT/config/initializers/string_encodings.rb":
Encoding.default_external = 'UTF-8'
That worked for me site-wide, instead of on a controller level.
In my case the ArgumentError occurred during a Capistrano deploy call that envolves ruby's net-ssh (ruby 1.9.2p290, net-ssh 2.3.0). None of the solutions afore mentioned worked and none of the other reasons I've read about so far: i.e. "strange character in key file", etc..
Finally, I found a none ASCII-character in a comment(!) line in my ASCII encoded ~/.ssh/config file. BINGO!
Question 828421 asked similar question, but received only one real answer (update rubygems) and that attempt results in the same error.
Ruby version 1.9.1.p243 on Windows. Included Gem, version 1.3.5.
Never installed any gems before; never did any special config for this Ruby.
Ruby itself works, as does irb, and "gem" operates but can't do install (and maybe other ops).
Tried this (from a book):
gem install rspec
Got this:
ERROR: http://gems.rubyforge.org/ does
not appear to be a repository
ERROR: While executing gem ...
(Gem::RemoteFetcher::FetchError)
SocketError: getaddrinfo: The storage control blocks were destroyed.
(http://gems.rubyforge.org/yaml)
When I go to that URL (without "yaml") using MSIE7, I get a page titled "Gemcutter | awesome gem hosting" and have no problem wandering around that site. So I don't -think- it's a proxy problem (though this is all from inside corporate firewall/proxies/etc).
When I go to that URL -with- "yaml", it goes to "http://production.s3.rubygems.org/yaml" and shows what I assume is an update specification page, starting with this:
--- !ruby/object:Gem::SourceIndex
I didn't destroy any "storage control blocks". So what is preventing gem from installing a gem?
Web search shows MANY people having this same problem over a long span of time, but I have yet to see anyone say "It's because of THIS, so do THIS to fix it." Well, someone suggested updating "gem", but trying that gets same error.
Help please?
Your browser might be using system-wide proxy settings or some sort of automatic configuration. The gem command probably doesn't. I'm behind an university proxy and I can't install/update any gems normally, but can access everything with my browser. To install gems, I normally create a tunnel to my server so I can bypass the proxy server.
This might help you to configure the proxy settings for the gem command: How do I update Ruby Gems from behind a Proxy (ISA-NTLM)
Sample:
sudo gem install nifty-generators -p http://proxy:port
It works well.
If your proxy requires authentication, then use:
gem install --http-proxy http://USERNAME:PASS#HOST:PORT gem_name
This can "at times" mean that for some reason rubygems.org is down currently, and so your local "invisible proxy" is returning you a 404 or what not.
#Tomas Markauskas didn't work for me because i'm not behind a proxy. And I didn't find a solution anywhere, I tried disable my AV and firewall, didn't
This was the solution for me:
gem install rails -r -w -p
Hope this will help people with the same problem.
EDIT: I have Windows 7 64bit.