Unknown algorithm MD5 using net-http-digest_auth - ruby

Question

I'm trying to do some digest authorization to a server, then parse the resulting HTML with nokogiri. I'm using the net-http-digest_auth gem (https://github.com/drbrain/net-http-digest_auth) to do the url connection. All is fine up until I start the digest_auth code (line 20); it throws an 'unknown algorithm ""MD5"" error'..
The full error message from the console:
~/.rvm/gems/ruby-1.9.3-p194#rails32/gems/net-http-digest_auth-1.2.1/lib/net/http/digest_auth.rb:105:in 'auth_header': unknown algorithm ""MD5"" (Net::HTTP::DigestAuth::Error)
from ./server_connection.rb:20:in '<main>'
Line 20 is the auth line:
auth = digest_auth.auth_header uri, res['www-authenticate'], 'GET'
Here's my complete code (almost completely verbatim from the sample code used at the github link):
#!/usr/bin/env ruby
require 'uri'
require 'net/http'
require 'net/http/digest_auth'
digest_auth = Net::HTTP::DigestAuth.new
uri = URI.parse 'http://url/controlpage?name=_internal_variables_&asList=1&useJS=True'
uri.user = 'username'
uri.password = 'password'
h = Net::HTTP.new uri.host, uri.port
req = Net::HTTP::Get.new uri.request_uri
res = h.request req
# res is a 401 response with a WWW-Authenticate header
auth = digest_auth.auth_header uri, res['www-authenticate'], 'GET'
# create a new request with the Authorization header
req = Net::HTTP::Get.new uri.request_uri
req.add_field 'Authorization', auth
# re-issue request with Authorization
res = h.request req
if res.code == "200"
page = Nokogiri::HTML(res)
isDaylight = page.css('.controlTitle:contains("isDaylight") ~ .controlValue');
puts isDaylight.content
end
Updated this question to include the request headers via Chrome's dev tools:
GET /_getupdatedcontrols?name=_internal_variables_&asList=True&folderFilter=0&changeCount=479&serverState=idle HTTP/1.1
Host: url
Connection: keep-alive
Cache-Control: no-cache
Authorization: Digest username="username", realm="Indigo Control Server", nonce="71079e9f29f7210325ae451d0f423f07", uri="/_getupdatedcontrols?name=_internal_variables_&asList=True&folderFilter=0&changeCount=479&serverState=idle", algorithm=MD5, response="bc056cc472d35f7967973cb51c5b1a65", qop=auth, nc=00005649, cnonce="18dfcf3e4a7b809d"
X-Indigo-Web-Server-Version: 1
X-Prototype-Version: 1.6.0.3
X-Requested-With: XMLHttpRequest
Pragma: no-cache
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.17 Safari/536.11
Accept: text/javascript, text/html, application/xml, text/xml, */*
Referer: http://url/controlpage?name=_internal_variables_&asList=1&useJS=True
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

Answer 1

I ended up using the httpclient gem to accomplish the same thing.
The final code to do exactly what i was after:
#!/usr/bin/env ruby
require 'httpclient'
require 'nokogiri'
c = HTTPClient.new
c.debug_dev = STDOUT
c.set_auth("http://domain.com", "username", "password")
doc = Nokogiri::HTML(c.get_content("http://domain.com"))
isDaylight = "";
doc.css('.controlTitle:contains("isDaylight") ~ .controlValue').each do |var|
isDaylight = var.content
end
if (!isDaylight)
system("curl -X PUT --digest -u username:password -d isOn=1 http://domain.com")
else
system("curl -X PUT --digest -u username:password -d isOn=0 http://domain.com")
end
I hope this helps others that may be working with a home automation server and needing to easily do digest-based authentication.

Answer 2

Seth,
I ran into this same issue while working on a script in ruby. I am new to ruby but after a few google searches and some Charles Proxy showing me what was going on, I see that it is common for HTTP implementations to include quotes in the algorithm="MD5" portion of the Auth header, which is incorrect according to spec (it should be algorithm=MD5, with out quotes). Your updated header logs from Chrome devtools shows that your server response is honoring the spec, but the ruby library is NOT when it interprets that response string. This can be seen by
You server's 401 response included:
note the algorithm=MD5
Authorization: Digest username="username", realm="Indigo Control Server", nonce="71079e9f29f7210325ae451d0f423f07", uri="/_getupdatedcontrols?name=_internal_variables_&asList=True&folderFilter=0&changeCount=479&serverState=idle", algorithm=MD5, response="bc056cc472d35f7967973cb51c5b1a65", qop=auth, nc=00005649, cnonce="18dfcf3e4a7b809d"
But the console output of the initial request using this Ruby library shows:
note the algorithm=\"MD5\"
<- "GET /some/request HTTP/1.1\r\nAccept: */*\r\nUser-Agent: Ruby\r\nConnection: close\r\nHost: 10.1.0.15\r\n\r\n"
-> "HTTP/1.1 401 Unauthorized\r\n"
-> "Content-Length: 530\r\n"
-> "Server: SomeServer/5.0\r\n"
-> "Allow: GET, HEAD, POST, PUT\r\n"
-> "Date: Sun, 27 Jan 2013 00:29:23 GMT\r\n"
-> "Content-Type: text/html;charset=utf-8\r\n"
-> "Www-Authenticate: Digest realm=\"Some Realm\", nonce=\"5a8b8b46cfb84466431baf454eb9ddb9\", algorithm=\"MD5\", qop=\"auth\"\r\n"
For the script example in the original post, I would insert the following two lines:
www_auth_response = res['www-authenticate']
www_auth_response["algorithm=\"MD5\""] = "algorithm=MD5"
And Modify the third line:
auth = digest_auth.auth_header uri, www_auth_response, 'GET'
As follows:
...
res = h.request req
# res is a 401 response with a WWW-Authenticate header
www_auth_response = res['www-authenticate']
www_auth_response["algorithm=\"MD5\""] = "algorithm=MD5"
auth = digest_auth.auth_header uri, www_auth_response, 'GET'
# create a new request with the Authorization header
req = Net::HTTP::Get.new uri.request_uri
req.add_field 'Authorization', auth
...
The important thing that is going on here is that we are modifying the www-authenticate string that is coming back from your initial unauthorized 401 request (as interpreted by this ruby library). Sending the modified header string (www_auth_response) to the digest_auth.auth_header method produces no errors. At least that worked for me in my script!
I hope that helps!
Matt