Ruby How to send image over tcp (http) server to client - ruby

I am trying to create a simple http server but when i send an png file firefox tells me that there are problems with the image. (the image can not be display).
I changed the content type to octed-stream so i can download the file from the browser.
By comparing the original image and the downloaded image with the text editor I realized that the downloaded one missed some lines in the beginning.
# Provides TCPServer and TCPSocket classes
require 'socket'
server = TCPServer.new('localhost', 1234)
ary = Array.new
f = File.binread '/Users/serrulez/Documents/GIT/KOS_Simple_HTTP_Server/Shrek.png'
loop do
socket = server.accept
request = socket.gets
index1 = request.index(' ');
index2 = request.index(' ',index1+1);
index3 = request.index("\r\n");
index4 = request.length;
method = request[0,index1]
URI = request[index1+1,index2-index1-1]
version = request[index2+1,index3-index2-1]
CRLF = b2s((request[index3,2] == "\r\n"))
if (URI == "/Shrek" || URI == "/index.html")
socket.print "HTTP/1.1 200 OK\r\n" +
"Content-Type: image/png\r\n" +
"Connection: close\r\n\r\n"
ary = Array.new
f = File.binread '/Users/serrulez/Documents/GIT/KOS_Simple_HTTP_Server/Shrek.png'
#if I print 'f' here to the console i get the hole image, but downloaded from the browser
#the upper part ist cut
puts f
ary.push(f)
socket.write(ary) # <- update, forgot to copy this line
end
socket.close
end

I now get the whole image with the above code but my characters are escaped.
The original image starts with :
�PNG
IHDR�H
�_jsRGB���gAMA���a
[...]
and the image I receive with the browser starts with;
["\x89PNG\r\n\x1A\n\x00\x00\x00\rIHDR\x00\x00\x00\xF3\x00\x00\x01H\b\x02\x00\x00\x00\n\xEE_j\x0

I solved the problem by myself. The last piece to add was the encoding (see below)
if (URI == "/Shrek")
socket.print "HTTP/1.1 200 OK\r\n" +
"Content-Type: image/png; charset=utf-8\r\n" +
"Connection: close\r\n\r\n"

Related

Bad record MAC on TLS v1.3 when using early_data

I have a project to complete in Ruby involving TLS v.1.3. I want to optimize requests and thus use "early data". I'm using a package called tttls1.3 and the client works until I send early data to the server. What's even more wired is that a request with early data goes through and I get a response from the server but immediately after the reply (response message) an alert 20 (Bad Record MAC) is received. I went so far that I even go and recalculate the "client-finished" message which seemed suspicious but it looks correct.
What could be the problem? Is there a TCP or other issue I could check?
Here's an example:
require 'socket'
require 'tttls1.3'
settings2 = {
alpn: ['http/1.1'],
supported_groups: [TTTLS13::NamedGroup::SECP256R1],
cipher_suites: [TTTLS13::CipherSuite::TLS_AES_256_GCM_SHA384],
check_certificate_status: false,
}
settings1 = {
alpn: ['http/1.1'],
supported_groups: [TTTLS13::NamedGroup::SECP256R1],
cipher_suites: [TTTLS13::CipherSuite::TLS_AES_256_GCM_SHA384],
check_certificate_status: false,
process_new_session_ticket: lambda do |nst, rms, cs|
return if Time.now.to_i - nst.timestamp > nst.ticket_lifetime
settings2[:ticket] = nst.ticket
settings2[:resumption_master_secret] = rms
settings2[:psk_cipher_suite] = cs
settings2[:ticket_nonce] = nst.ticket_nonce
settings2[:ticket_age_add] = nst.ticket_age_add
settings2[:ticket_timestamp] = nst.timestamp
end
}
# REQUEST
socket = TCPSocket.new("ssltest.louis.info", 443)
client = TTTLS13::Client.new(socket, "ssltest.louis.info", settings1)
client.connect
client.write("GET / HTTP/1.1\r\n")
client.write("Host: ssltest.louis.info\r\n")
client.write("\r\n\r\n")
client.read
client.close
socket.close
sleep(1)
# RESUMPTION
socket = TCPSocket.new("ssltest.louis.info", 443)
client = TTTLS13::Client.new(socket, "ssltest.louis.info", settings2)
client.early_data("HEAD / HTTP/1.1\r\nHost: ssltest.louis.info\r\n\r\n\r\n")
client.connect
p client.read
p client.read
p client.read
p client.read
Original issue: https://github.com/thekuwayama/tttls1.3/issues/48
It turned out that the Connection: close header must be present in the request. It must be the remote server implementation specific.

Aws::S3::Errors::InvalidArgument (): on bucket.put_object

I'm trying to upload base64 encoded image to S3 bucket. My bucket list shows correctly after signing in. I'm trying to upload using the bucket#put_object
I don't see any hint as to what the invalid argument is.
My JSON body looks like..
""product": {
"product_photo": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAT8AAADQCAYAAABxw2ZIAAABQGlDQ1BJQ0MgUHJvZmlsZQAAKJFjYGASSCwoyGFhYGDIzSspCnJ3UoiIjFJgf8LAxsDMIAqEVonJxQWOAQE+QCUMMBoVfLvGwAiiL+uCzH ...."
and if I use the Carrierwave file storage, image saves fine.
How do I upload my base64 image using just the aws-sdk?
#Aws::S3::Bucket:0x00007fd60a2a8928
THat was the new bucket
UPLOADING NOW
Completed 500 Internal Server Error in 654ms (ActiveRecord: 17.3ms)
Aws::S3::Errors::InvalidArgument ():
app/controllers/products_controller.rb:139:in `supload'
def supload(params)
s3 = Aws::S3::Client.new(access_key_id: ENV['AWS_ACCESS_KEY_ID'].strip, secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'].strip) #call strip on the keys
resp = s3.list_buckets
puts resp
bucket = resp[:buckets][0]
s3Resource = Aws::S3::Resource.new
# reference an existing bucket by name
bucket = s3Resource.bucket('htm-product-photo')
puts bucket
puts "That was the new bucket"
data = Base64.decode64(params[:product_photo].to_s)
type = "image/png" #params[:contentType].to_s
extension = ".png" #params[:extension].to_s
name = ('a'..'z').to_a.shuffle[0..7].join # + ".#{extension}"
puts "UPLOADING NOW"
obj = bucket.put_object({body: data, key: name, content_type:type,acl:"public_read"})
puts obj.errors
puts "Tried to upload"
# https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Bucket.html#put_object-instance_method
url = obj.public_url().to_s
url
end
EDIT: With wire trace enabled per #D. SM
#<Aws::S3::Bucket:0x00007ffa87738708>
THat was the new bucket
UPLOADING NOW
opening connection to htm-product-photo.s3.amazonaws.com:443...
opened
starting SSL for htm-product-photo.s3.amazonaws.com:443...
SSL established, protocol: TLSv1.2, cipher: ECDHE-RSA-AES128-GCM-SHA256
<- "PUT /ekjlaqbo HTTP/1.1\r\nContent-Type: image/png\r\nAccept-Encoding: \r\nUser-Agent: aws-sdk-ruby3/3.104.3 ruby/2.6.2 x86_64-darwin18 aws-sdk-s3/1.76.0\r\nX-Amz-Acl: public_read\r\nExpect: 100-continue\r\nContent-Md5: ThpRJ9rI1gkEr5zO+D4zOA==\r\nHost: htm-product-photo.s3.amazonaws.com\r\nX-Amz-Date: 20200816T221225Z\r\nX-Amz-Content-Sha256: f39e6d48f796085a4c601e08ce91be80a06fdffd22b0b3426b9eab95098fc003\r\nAuthorization: AWS4-HMAC-SHA256 Credential=AKIAUSPGIPMTP6S6J47A/20200816/us-east-1/s3/aws4_request, SignedHeaders=content-md5;content-type;host;user-agent;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=d30f1a30c6065f8d808cc238c0650c2c11799e55c2e22003571e43c90e021936\r\nContent-Length: 87850\r\nAccept: */*\r\n\r\n"
-> "HTTP/1.1 400 Bad Request\r\n"
-> "x-amz-request-id: B4DF8652D31666ED\r\n"
-> "x-amz-id-2: 32VYR2WsieoAOP2mOb27faxyhO1koISnl8ZerKLpIRQVIJE9dtZeQ3isluBzV042F1kBV7WgIEQ=\r\n"
-> "Content-Type: application/xml\r\n"
-> "Transfer-Encoding: chunked\r\n"
-> "Date: Sun, 16 Aug 2020 22:12:26 GMT\r\n"
-> "Connection: close\r\n"
-> "Server: AmazonS3\r\n"
-> "\r\n"
-> "139\r\n"
reading 313 bytes...
-> "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InvalidArgument</Code><Message></Message><ArgumentName>x-amz-acl</ArgumentName><ArgumentValue>public_read</ArgumentValue><RequestId>B4DF8652D31666ED</RequestId><HostId>32VYR2WsieoAOP2mOb27faxyhO1koISnl8ZerKLpIRQVIJE9dtZeQ3isluBzV042F1kBV7WgIEQ=</HostId></Error>"
read 313 bytes
Enable wire trace to see what is being sent to S3 and what the responses are.
The allowed options for :acl according to documentation are private, public-read, public-read-write, authenticated-read, aws-exec-read, bucket-owner-read, bucket-owner-full-control.
So, you have to change public_read to public-read
obj = bucket.put_object({body: data, key: name, content_type:type,acl:"public-read"})

ESP8266 NodeMCU out of memory

NodeMCU info
> Lua 5.1.4
> SDK 2.2.1
> Memory Usage :
> Total : 3260490 bytes
> Used : 9287 bytes
> Remain: 3251203 bytes
Error I get when I try to send HTTP response with big json string response (json_response)
PANIC: unprotected error in call to Lua API (file.lua:5: out of memory)
Code:
-- a simple HTTP server
srv = net.createServer(net.TCP)
srv:listen(80, function(conn)
conn:on("receive", function(sck, payload)
sck:send("HTTP/1.0 200 OK\r\nContent-Type: text/html\r\n\r\n"..json_response)
end)
conn:on("sent", function(sck) sck:close() end)
end)
Yes, that won't work if you're trying to send a lot of data. You need to send this piece-by-piece. Our API documentation shows two approaches (you would find further references here on SO) the first being this:
srv = net.createServer(net.TCP)
function receiver(sck, data)
local response = {}
-- if you're sending back HTML over HTTP you'll want something like this instead
-- local response = {"HTTP/1.0 200 OK\r\nServer: NodeMCU on ESP8266\r\nContent-Type: text/html\r\n\r\n"}
response[#response + 1] = "lots of data"
response[#response + 1] = "even more data"
response[#response + 1] = "e.g. content read from a file"
-- sends and removes the first element from the 'response' table
local function send(localSocket)
if #response > 0 then
localSocket:send(table.remove(response, 1))
else
localSocket:close()
response = nil
end
end
-- triggers the send() function again once the first chunk of data was sent
sck:on("sent", send)
send(sck)
end
srv:listen(80, function(conn)
conn:on("receive", receiver)
end)

Connect to Microsoft Push Notification Service for Windows Phone 8 from Ruby

We are developing a WP8 app that requires push notifications.
To test it we have run the push notification POST request with CURL command line, making sure that it actually connects, authenticates with the client SSL certificate and sends the correct data. We know for a fact that this work as we are receiving pushes to the devices.
This is the CURL command we have been using for testing purposes:
curl --cert client_cert.pem -v -H "Content-Type:text/xml" -H "X-WindowsPhone-Target:Toast" -H "X-NotificationClass:2" -X POST -d "<?xml version='1.0' encoding='utf-8'?><wp:Notification xmlns:wp='WPNotification'><wp:Toast><wp:Text1>My title</wp:Text1><wp:Text2>My subtitle</wp:Text2></wp:Toast></wp:Notification>" https://db3.notify.live.net/unthrottledthirdparty/01.00/AAF9MBULkDV0Tpyj24I3bzE3AgAAAAADCQAAAAQUZm52OkE1OUZCRDkzM0MyREY1RkE
Of course our SSL cert is needed to actually use the URL, but I was hoping someone else has done this and can see what we are doing wrong.
Now, our problem is that we need to make this work with Ruby instead, something we have been unable to get to work so far.
We have tried using HTTParty with no luck, and also net/http directly without any luck.
Here is a very simple HTTParty test script I have used to test with:
require "httparty"
payload = "<?xml version='1.0' encoding='utf-8'?><wp:Notification xmlns:wp='WPNotification'><wp:Toast><wp:Text1>My title</wp:Text1><wp:Text2>My subtitle</wp:Text2></wp:Toast></wp:Notification>"
uri = "https://db3.notify.live.net/unthrottledthirdparty/01.00/AAF9MBULkDV0Tpyj24I3bzE3AgAAAAADCQAAAAQUZm52OkE1OUZCRDkzM0MyREY1RkE"
opts = {
body: payload,
headers: {
"Content-Type" => "text/xml",
"X-WindowsPhone-Target" => "Toast",
"X-NotificationClass" => "2"
},
debug_output: $stderr,
pem: File.read("/Users/kenny/Desktop/client_cert.pem"),
ca_file: File.read('/usr/local/opt/curl-ca-bundle/share/ca-bundle.crt')
}
resp = HTTParty.post uri, opts
puts resp.code
This seems to connect with SSL properly, but then the MS IIS server returns 403 to us for some reason we don't get.
Here is essentially the same thing I've tried using net/http:
require "net/http"
url = URI.parse "https://db3.notify.live.net/unthrottledthirdparty/01.00/AAF9MBULkDV0Tpyj24I3bzE3AgAAAAADCQAAAAQUZm52OkE1OUZCRDkzM0MyREY1RkE"
payload = "<?xml version='1.0' encoding='utf-8'?><wp:Notification xmlns:wp='WPNotification'><wp:Toast><wp:Text1>My title</wp:Text1><wp:Text2>My subtitle</wp:Text2></wp:Toast></wp:Notification>"
pem_path = "./client_cert.pem"
cert = File.read pem_path
http = Net::HTTP.new url.host, url.port
http.use_ssl = true
http.cert = OpenSSL::X509::Certificate.new cert
http.key = OpenSSL::PKey::RSA.new cert
http.ca_path = '/etc/ssl/certs' if File.exists?('/etc/ssl/certs') # Ubuntu
http.ca_file = '/usr/local/opt/curl-ca-bundle/share/ca-bundle.crt' if File.exists?('/usr/local/opt/curl-ca-bundle/share/ca-bundle.crt') # Mac OS X
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
r = Net::HTTP::Post.new url.path
r.body = payload
r.content_type = "text/xml"
r["X-WindowsPhone-Target"] = "toast"
r["X-NotificationClass"] = "2"
http.start do
resp = http.request r
puts resp.code, resp.body
end
Like the HTTParty version, this also returns 403..
I'm starting to get the feeling that this won't actually work with net/http, but I've also seen a few examples of code claiming to work, but I can't see any difference compared to what we have tested with here.
Does anyone know how to fix this? Is it possible? Should I use libcurl instead perhaps? Or even do a system call to curl? (I may have to do the last one as an interim solution if we can't get this to work soon).
Any input is greatly appreciated!
Thanks,
Kenny
Try using some tool like http://mitmproxy.org to compare requests from your code and curl.
For example curl in addition to specified headers does send User-Agent and Accept-headers, microsoft servers may be checking for these for some reason.
If this does not help - then it's ssl-related

Making a URL in a string usable by Ruby's Net::HTTP

Ruby's Net:HTTP needs to be given a full URL in order for it to connect to the server and get the file properly. By "full URL" I mean a URL including the http:// part and the trailing slash if it needs it. For instance, Net:HTTP won't connect to a URL looking like this: example.com, but will connect just fine to http://example.com/. Is there any way to make sure a URL is a full URL, and add the required parts if it isn't?
EDIT: Here is the code I am using:
parsed_url = URI.parse(url)
req = Net::HTTP::Get.new(parsed_url.path)
res = Net::HTTP.start(parsed_url.host, parsed_url.port) {|http|
http.request(req)
}
If this is only doing what the sample code shows, Open-URI would be an easier approach.
require 'open-uri'
res = open(url).read
This would do a simple check for http/https:
if !(url =~ /^https?:/i)
url = "http://" + url
end
This could be a more general one to handle multiple protocols (ftp, etc.)
if !(url =~ /^\w:/i)
url = "http://" + url
end
In order to make sure parsed_url.path gives you a proper value (it should be / when no specific path was provided), you could do something like this:
req = Net::HTTP::Get.new(parsed_url.path.empty? ? '/' : parsed_url.path)

Resources