My Rails application regularly polls partners' ICS files and sometimes it fails for no reason whatsoever. When I do:
curl https://www.airbnb.es/calendar/ical/234892374.ics?s=23412342323
(params #'s faked here)
I get output matching the content of the ICS file. Just opening it in the browser works fine as well.
When I use:
Net::HTTP.get(URI(a.ics_link))
I get a "503 Service Temporarily Unavailable" response. I also tried the same with OpenURI with similar results.
Why is it that the server is treating requests from curl or a browser differently?
Is there some way to get Ruby to get around this?
It's an https issue... not sure why, but switch your url in Ruby to https and it should work.
Related
I noticed when running wget https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=foo and similar queries, I don't get the search results, but the google homepage.
There seems to be some redirect within the google page. Does anyone know a fix to wget so it would work?
You can use this curl commands to pull Google query results:
curl -sA "Chrome" -L 'http://www.google.com/search?hl=en&q=time' -o search.html
For using https URL:
curl -k -sA "Chrome" -L 'https://www.google.com/search?hl=en&q=time' -o ssearch.html
-A option sets a custom user-agent Chrome in request to Google.
#q=foo is your hint, as that's a fragment ID, which never gets sent to the server. I'm guessing you just took this URL from your browser URL-bar when using the live-search function. Since it is implemented with a lot of client-side magic, you cannot rely on it to work; try using Google with live search disabled instead. A URL pattern that seems to work looks like this: http://www.google.com/search?hl=en&q=foo.
However, I do notice that Google returns 403 Forbidden when called naïvely with wget, indicating that they don't want that. You can easily get past it by setting some other user-agent string, but do consider all the implications before doing so on a regular basis.
I've been trying to use mechanize to download mp3 files, but the server always returns a 404.
Looking at the headers my browser sends (checked on Chrome and FF), I noticed that the If-Range and Range headers are used to initiate a successful download, so I'm guessing the server is rejecting any request that doesn't specify them.
What is the right way to download files in this way, using ruby (Net::HTTP) or bash (curl or wget)?
404 is file not found. Are you sure your URL is correct? If it is correct then you should be able to use wget <full url and file name> to test it.
I was messing around and recklessly trying to implement SSL with Sinatra. I was using many scripts that I found on the internet without actually knowing what they did. I then realized I didn't actually need SSL and now sinatra is completely broken.
No matter what my app is, it always always prints "!! Invalid request" on the terminal whenever it receives a request from the browser. I've noticed that the http://localhost:4567 always switches to https://localhost:4567 in the browser URL.
I would like to have Sinatra act like it did when I originally installed it.
I'm new to this...
Uninstall & reinstall Sinatra and Ruby
I'm trying to implement an xmlrpc server/client per the CodeIgniter User Guide. I've taken the code as is and keep getting
Did not receive a '200 OK' response from remote server.
I'm running PHP 5.2.1 on the server. A quick google search leads to http://codeigniter.com/forums/viewthread/63287/ which does not help. I've also tried modifying my Xmlrpcs.php file under system/libraries without success.
So what IS CI actually sending back? Setup test using Firebug and CUrL might help.
I use a CI install with "ajax" controller which handles its own AJAX requests.
This is my first time asking a question, please be gentle!
I have a Rails application that handles content for a whole bunch of domains (over 100 so far). Each domain either points to where my app is hosted (Heroku, if you're interested), or the original place it was hosted. Every time a domain is ready, it needs to point to the heroku servers, so that my app can serve content for it.
To check to see if a domain has successfully been changed over from its original location to my application, I'm writing a script that looks for a special hidden tag I included in them. If it finds the tag, then the domain is pointing to my app. If not, it hasn't been changed, which I record.
The problem is that, at least for one domain so far, I'm getting a 404 OpenURI::HTTPError exception for my script. Which is strange, because I can visit the site just fine and I can even get it via curl. Does anyone know why a working site would get an error like this? Here's the important snippet:
require 'rubygems'
require 'open-uri'
require 'hpricot'
...
url = "http://www.#{domainname}.com"
doc = Hpricot(open(url)) #<---- Problem right here.
...
Thanks for all of your help!
Welcome to SO!
Here would be my debugging method:
See if you can replicate in irb with open-uri alone, no Hpricot:
$ irb -rubygems -ropen-uri
>> open('http://www.somedomain.com')
Look in your Heroku log to see if it even touches the server.
Look in your original server's log for the same.
Throw open something like Wireshark to see the HTTP transaction, and see if a 404 is indeed coming back.
Start with that, and come back with your results.