I'm experimenting with the UK NHS 'GP Connect' REST API and trying to wrap some of the connection complexity in a more simple Ruby layer.
I've been successfully playing with the API via Postman and the responses are all correctly unzipped to JSON/FHIR. Taking the simplest of the requests from my Postman session and converting it to Ruby using the Faraday HTTP gem, I can make the request/response cycle work fine, but the response is still gzipped.
(I'm using Ruby 2.6.1 and Faraday 0.15.4)
(The API requires 'accept-encoding': 'gzip' to work at all.)
Googling the problem I'm having turns up only Faraday middleware which is intended to be used when not using Net::HTTP (the default adapter, and what I am using)
The whole thing is a single script at this stage so it's easy enough to show. (There are additionally some client certs and a locally installed CA cert which are all working fine and no issues)
***** = I've blanked it to protect the innocent, although the healthcare endpoint I'm using here is a dummy-test with fictional data anyway.
require 'faraday'
identifier = "identifier=https://fhir.nhs.uk/Id/nhs-number|*****"
baseurl = "https://systmoneukdemo1.tpp-uk.com"
headers = {
"authorization": '*****',
"content-type": 'application/json+fhir',
"accept-encoding": 'gzip',
"accept": 'application/json+fhir',
"ssp-from": '*****',
"ssp-interactionid": 'urn:nhs:names:services:gpconnect:fhir:rest:search:patient-1',
"ssp-to": '123456',
"ssp-traceid": '*****',
"cache-control": 'no-cache',
}
connection = Faraday.new(
url: baseurl,
ssl: {
ca_file: './certs/cacert.crt',
client_cert: OpenSSL::X509::Certificate.new(File.read("certs/summer-school-cert.crt")),
client_key: OpenSSL::PKey::RSA.new(File.read("certs/summer-school-cert.key")),
}
)
response = connection.get do |request|
request.url "/SystmOneMHS/NHSConnect/Z12345/STU3/1/Patient?#{identifier}"
request.headers = headers
end
puts response.body
The response when I run ruby script.rb is:
�U�o�0������- |Z��U�nӪjrV�A�T����;;dM�n�4-������{wϽ'
������)9f�W�Һ�b�-+�e��7���2pZ���dPr�5o���,m��\;
`*�:�Ά������r��1Aju�H��셸T�Ϙ��ލ|�?�
߶ih>jy����D����5&=��m��/G��Ñ�'�Ϣ#?��'Q6�~���y��^[.�u��-��#�ȡSǛ^�~�b?|A�#צ�%eu��:
�� ���h�M�4dm��j�WB�l
8�%��f��b��sf����C�>iS[
��C�q$�ɛ���ǣ��h�oF�?�jٙR
�7�gh�h�����w�`��d�:iq�P� ������Ya�z<����k��d5X1����J�WfV��}�y�#�w%��0���
��;à)�!g����(��L�WMm�<xg��L����p�I�������9��]#
��i�N3�nz��z�'�Mf�țz�ڪ�����]"�y�����r'
j����gru��_��u��C,��U�� /
ý"���H����V��0�kT�ĸ��.f��pUNՊI�ݎvs��Z���잹˂�}��g��6Q��v���p<
�-.6z
R�Ğ��6�V�� ��-Pzs�L��Ի�4�_��L��Z�b��g:��;�OF=g{�y�?Ly�+����kk���H�y���
������a��s}��O/�W�=��r��F��%a
however if I pipe that output into gzip:
ruby script.rb | gzip -d
{"resourceType":"Bundle","id":"4D2EAEC0-A8DB-11E9-8B9A-1BA7090BDAD3","meta":{"profile":["https://fhir.nhs.uk/STU3/StructureDefinition/GPConnect-Searchset-Bundle-1"]},"type":"searchset","entry":[{"fullUrl":"Patient/9222500000000000","resource":{"resourceType":"Patient","id":"9222500000000000","meta":{"profile":["https://fhir.nhs.uk/STU3/StructureDefinition/CareConnect-GPC-Patient-1"],"versionId":"220262D942C746E9D8023C82ABC8F500","lastUpdated":"2019-07-17T22:39:05+01:00"},"identifier":[{"system":"https://fhir.nhs.uk/Id/nhs-number","value":"6656336566","extension":[{"url":"https://fhir.nhs.uk/STU3/StructureDefinition/Extension-CareConnect-GPC-NHSNumberVerificationStatus-1","valueCodeableConcept":{
"coding":
[
{
"system":"https://fhir.nhs.uk/STU3/CodeSystem/CareConnect-NHSNumberVerificationStatus-1",
"code":"01",
"display":"Number present and verified"
}
]
}}]}],"name":[{"prefix":["Mr"],"given":["Michael"],"family":"Smith","use":"official"}],"address":[{"use":"home","type":"both","line":["6 Wren Street"],"city":"London","postalCode":"WC1X 0HD"}],"telecom":[{"system":"phone","use":"home","value":"01132252263"},{"system":"email","use":"home","value":"liang.lin#tpp-uk.com"}],"birthDate":"1960-01-19","gender":"male","active":true,"managingOrganization":{"reference":"Organization/bd00000000000000"},"generalPractitioner":[{"reference":"Practitioner/e576600000000000"}],"extension":[{"url":"https://fhir.nhs.uk/STU3/StructureDefinition/Extension-CareConnect-GPC-RegistrationDetails-1","extension":[{"url":"registrationPeriod","valuePeriod":{"start":"2017-01-01T00:00:00Z"}},{"url":"registrationType","valueCodeableConcept":{"coding":[{"system":"https://fhir.nhs.uk/STU3/CodeSystem/CareConnect-RegistrationType-1","code":"O","display":"Other"}]}},{"url":"preferredBranchSurgery","valueReference":{"reference":"Location/bd00000000000000_0100000000000000"}}]}]}}]}
gzip: stdin: unexpected end of file
I get un-gzipped FHIR JSON content, which is what I'm expecting. So everything is working right up until the ungzipping stage, which is not being carried out, and the raw ugzipped content is returning to me as the response.
Somehow it seems that Faraday isn't automatically un-gzipping the response body. I've searched in Faraday's documentation for a flag or other control that I perhaps should be setting, but I can't find a clear explanation of how this should be handled.
Clearly I could un-gzip this output directly, but I would like to know if there is a proper internal 'Faraday' way of doing it. If it's using the Faraday Gzip middleware, then great, but how?
If it helps with working out what's going on, I've reimplemented this in plain Net::HTTP and the results are exactly the same, the reponse body is not being un-gzipped. Both Faraday and Net::HTTP say they do the ungzipping automatically, but neither of them are.
Any thoughts or advice welcome.
Given you are using the faraday-middleware gem, you can just use the :gzip middleware in connection setup.
request.use :gzip
Using your code as example:
response = connection.get do |request|
request.url "/SystmOneMHS/NHSConnect/Z12345/STU3/1/Patient?#{identifier}"
request.use :gzip
request.headers = headers
end
Related
I'm currently stuck with a response time problem with the Faraday HTTP library. I'm using the following code to initialize a POST request to a server running on my local machine at 127.0.0.1. The server returns a valid JSON string. The code to initialize the Faraday object on the client side is as follows:
url = 'http://127.0.0.1'
conn = Faraday.new(:url => url) do |faraday|
faraday.request :url_encoded
faraday.response :json, :content_type => 'application/json'
faraday.adapter Faraday.default_adapter
end
Then I send a JSON string via a POST request to the server. The code for sending the request looks like this (text size up to 5000 char):
payload = {:language => 'en', :text => 'some text'}.to_json
response = conn.post do |req|
req.url '/api'
req.headers['Content-Type'] = 'application/json'
req.body = payload
end
The expected result is a JSON string of the following structure:
{
"level1.1" : [
{
"level2.1" : [
{
"value1" : "Text",
"value2" : "Text",
"value(...)" : "Text",
"value(n)" : "Text"
},
{...
}
],
<and so on... - of course in a valid JSON structure ending>
When I run the code without doing anything with the result it performs pretty well and finishes in a reasonable time (< 0.5s). But as soon as I try to access the response object the script becomes terrible slow.
Just by adding the line:
p response.body
The processing time goes up to > 8s.
I have checked the server response with Postman and it works absolutely fine without any visible problems. Response time in Postman is >0.5 s as well. The slow down appears only on the client side when I try to access the response object. Just inspecting the response object doesn't effect the processing time either. But as soon as I start "to do" something with the response it becomes terribly slow.
I'm running Ruby 2.5.3 and Faraday 0.15.4 / Middleware 0.12.2
Any idea what might cause this slowdown is greatly appreciated.
Krid
EDIT
Despite my comment that HTTParty solved this issue this was just due to a smaller payload. HTTParty and Faraday both perform badly on my POST request while a POST request from Postman runs very fast even with large payloads. I have no idea what causes this different runtime behavior when querying exactly the same server app with exactly the same payload.
As said any ideas that might point me in the right direction greatly appreciated.
Krid
...after searching for a solution for days I finally found the issue. When you have Python and Ruby code combined within a folder that is connected with a virtualenv Python (and activated Python env of course) Ruby apparently has problems resolving the local address space. It works but it takes some strange detours that make the whole thing terribly slow.
My solution: I moved the Ruby code that has to play together with the Python code via HTTP connections in a folder outside of the Python code folder.
I am trying to debug a request that I am making to a server, but I can't figure out what is wrong with it, as it seems to be just like the request that I am putting through on RESTClient.
I am initializing it like so:
request = Net::HTTP::Post.new(url.to_s)
request.add_field "HeaderKey", "HeaderValue"
request.body = requestBody
and then I am executing it like so:
Net::HTTP.start(url.host, url.port) do |http|
response = http.request(request)
end
The requestBody is a string that is encoded with Base64.encode64.
Is there a way to output the request to see exactly where it's going and with what contents? I've used Paros for checking my iOS connections and I can also output a description of requests from most platforms I've worked with, but I can't figure it out for Ruby.
I've found that HTTP Scoop works pretty well for grabbing network traffic (disclaimer - it's not free!)
I am communicating with API that requires DELETE request with JSON body. This works on console:
curl -XDELETE http://api.com/endpoint_path/rest_resource
-d '{"items":[{"type":"type1","item_id":"item1"}]}'
It seems that most gems for making HTTP requests don't support DELETE request with body (I tried RestClient and Curb). Is there a way to do it using some Ruby gem (preferably Curb) or Net::HTTP?
Here's one way using HTTParty:
HTTParty.delete("http://api.com/endpoint_path/rest_resource", {
:body => '{"items":[{"type":"type1","item_id":"item1"}]}'
})
it could be used her. This is ORM for api. https://github.com/remiprev/her
Example of usage:
RestResource.destroy_existing(id, body_params)
I also spent some time on this issue and #Casper's answer shed the light.
Seems to me that the key is to pass the body value as JSON string, which is not written in most of the documentations I found.
Here's another example using httpclient
require 'json'
body = { 'items': [{ 'type': 'type1', 'item_id': 'item1' }]}
HTTPClient.new.delete('http://api.com/endpoint_path/rest_resource', body.to_json)
I am writing mock Ruby servers to test components of API. I send a POST request with a body, and I'd like my mock server to return a body of that POST request. Currently i have this code:
require 'socket'
webserver = TCPServer.new('127.0.0.1', 7125)
loop do
session = webserver.accept
session.print "HTTP/1.1 200/OK\r\nContent-type:text/html\r\n\r\n"
request = session.gets
session.puts request
session.close
end
A POST request with body FOO returns a response with body that contains only POST / HTTP/1.1 How to fix it?
Writing your own HTTP server is really going to get you into trouble because the specification, while superficially simple, has a number of subtle nuances that can trip you up. In this case, you're reading one line with gets and ignoring the bulk of the submission. You're going to have to address that by reading in and properly decoding the posted data.
For something with a familiar interface you might start with Net::HTTP::Server instead.
I'm building some kind of proxy.
When I call some url in a rack application, I forward that request to an other url.
The request I forward is a POST with a file and some parameters.
I want to add more parameters.
But the file can be quite big. So I send it with Net::HTTP#body_stream instead of Net::HTTP#body.
I get my request as a Rack::Request object and I create my Net::HTTP object with that.
req = Net::HTTP::Post.new(request.path_info)
req.body_stream = request.body
req.content_type = request.content_type
req.content_length = request.content_length
http = Net::HTTP.new(#host, #port)
res = http.request(req)
I've tried several ways to add the proxy's parameters. But it seems nothing in Net::HTTP allows to add parameters to a body_stream request, only to a body one.
Is there a simpler way to proxy a rack request like that ? Or a clean way to add my parameters to my request ?
Well.. as i see it, this is a normal behaviour. I'll explain why. If you only have access to a Rack::Request,(i guess that) your middleware does not parse the response (you do not include something like ActionController::ParamsParser), so you don't have access to a hash of parameters, but to a StringIo. This StringIO corresponds to a stream like:
Content-Type: multipart/form-data; boundary=AaB03x
--AaB03x
Content-Disposition: form-data; name="param1"
value1
--AaB03x
Content-Disposition: form-data; name="files"; filename="file1.txt"
Content-Type: text/plain
... contents of file1.txt ...
--AaB03x--
What you are trying to do with the Net::HTTP class is to: (1). parse the request into a hash of parameters; (2). merge the parameters hash with your own parameters; (3). recreate the request. The problem is that Net::HTTP library can't do (1), since it is a client library, not a server one.
Therefore, you can not escape parsing some how your request before adding the new parameters.
Possible solutions:
Insert ActionController::ParamsParser before your middleware. After that, you may use the excellent rest-client lib to do something like:
RestClient.post ('http://your_server' + request.path_info), :params => params.merge(your_params)
You can attempt to make a wrapper on the StringIO object, and add, at the end of stream,your own parameters. However, this is not trivial nor advisable.
Might be one year too late, but I had the same issue verifying Paypal IPNs. I wanted to forward back the IPN request to Paypal for verification but needed to add :cmd => '_notify-validate'.
Instead of modifying the body stream, or body, I appended it as part of the URL path, like so:
reply_request = Net::HTTP::Post.new(url.path + '?cmd=_notify-validate')
It seems a bit of a hack, but I think it's worth it if you aren't going to use it for anything else.