How do I get a byte[] (image) from a webservice using micronaunt HttpClient - download

Question

I am porting a Grails 3.1 library for using some internal webservices to Grails 4.0. One of the services provides an image of a requested employee upon request. I am having difficulty implementing the (micronaut) HttpClient code to process the request - specifically to get a proper byte[] that is the returned image.
A simple curl command on the command line works with the service:
curl -D headers.txt -H 'Authorization:Basic <encodedKeyHere>' https:<serviceUrl> >> image.jpg
and the image is correct. The header.txt is:
HTTP/1.1 200
content-type: image/jpeg;charset=UTF-8
date: Tue, 27 Aug 2019 20:05:43 GMT
x-ratelimit-limit: 100
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 99
x-ratelimit-remaining: 99
X-RateLimit-Reset: 38089
x-ratelimit-reset: 15719
Content-Length: 11918
Connection: keep-alive
The old library uses the groovyx.net.http.HTTPBuilder and simply does:
http.request(Method.GET, ContentType.BINARY) {
uri.path = photoUrlPath
uri.query = queryString
headers.'Authorization' = "Basic $encoded".toString()
response.success = { resp, inputstream ->
log.info "response status: ${resp.statusLine}"
return ['status':resp.status, 'body':inputstream.getBytes()]
}
response.failure = { resp ->
return ['status':resp.status,
'error':resp.statusLine.reasonPhrase,
body:resp.getEntity().getContent().getText()]
}
}
so returning the bytes from an inputStream. This works.
I've tried several things using the micronaut HttpClient, both with the low level API and with the declarative API.
A simple example with the declarative API:
#Get(value='${photo.ws.pathurl}', produces = MediaType.IMAGE_JPEG)
HttpResponse<byte[]> getPhoto(#Header ('Authorization') String authValue,
#QueryValue("emplId") String emplId)
And than in the Service:
HttpResponse<byte[]> resp = photoClient.getPhoto(getBasicAuth(),emplId)
def status = resp.status() // code == 200 --> worked
def bodyStrOne = resp.getBody() // nope: get Optional.empty
// Tried different getBody(class) -> Can't figure out where the byte[]s are
// For example can do:
def buf = resp.getBody(io.netty.buffer.ByteBuf).value // Why need .value?
def bytes = buf.readableBytes() // Returns 11918 --> the expected value
byte[] ans = new byte[buf.readableBytes()]
buf.readBytes(ans) // Throws exception: io.netty.util.IllegalReferenceCountException: refCnt: 0
This "works" but the returned String looses some encoding that I can't reverse:
// Client - use HttpResponse<String>
#Get(value='${photo.ws.pathurl}', produces = MediaType.IMAGE_JPEG)
HttpResponse<String> getPhoto(#Header ('Authorization') String authValue,
#QueryValue("emplId") String emplId)
// Service
HttpResponse<String> respOne = photoClient.getPhoto(getBasicAuth(),emplId)
def status = respOne.status() // code == 200 --> worked
def bodyStrOne = respOne.getBody(String.class) // <-- RETURNS DATA..just NOT an Image..or encoded or something
String str = bodyStrOne.value // get the String data
// But these bytes aren't correct
byte[] ans = str.getBytes() // NOT an image..close but not.
// str.getBytes(StandardCharsets.UTF_8) or any other charset doesn't work
Everything I've tried with the ByteBuf classes throws the io.netty.util.IllegalReferenceCountException: refCnt: 0 exception.
Any direction/help would be greatly appreciated.
Running:
Grails 4.0
JDK 1.8.0_221
Groovy 2.4.7
Windows 10
IntellJ 2019.2

Answer 1

It must be Grails bug.
Add this line into logback.groovy:
logger("io.micronaut.http", TRACE)
Then you should see that the body was not empty but finally it ends with error Unable to convert response body to target type class [B. See the trace:
2019-09-11 11:19:16.235 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient : Status Code: 200 OK
2019-09-11 11:19:16.235 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient : Content-Type: image/jpeg
2019-09-11 11:19:16.235 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient : Content-Length: 11112
2019-09-11 11:19:16.237 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient : Accept-Ranges: bytes
2019-09-11 11:19:16.237 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient : Response Body
2019-09-11 11:19:16.237 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient : ----
2019-09-11 11:19:16.238 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient : ���� C
...
2019-09-11 11:19:16.241 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient : ----
2019-09-11 11:19:16.243 TRACE --- [ntLoopGroup-1-4] i.m.http.client.DefaultHttpClient : Unable to convert response body to target type class [B
But when you try the same in standalone Microunaut application (add <logger name="io.micronaut.http" level="trace"/> into logback.xml) the result is different:
09:02:48.583 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - Status Code: 200 OK
09:02:48.583 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - Content-Type: image/jpeg
09:02:48.589 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - content-length: 23195
09:02:48.590 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - Response Body
09:02:48.590 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - ----
09:02:48.612 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - ���� C���
...
09:02:48.620 [nioEventLoopGroup-1-5] TRACE i.m.http.client.DefaultHttpClient - ----
Micronaut trace has no error.
Here is an example of declarative HTTP client which downloads random image from https://picsum.photos web site:
import io.micronaut.http.HttpResponse
import io.micronaut.http.MediaType
import io.micronaut.http.annotation.Get
import io.micronaut.http.client.annotation.Client
#Client('https://picsum.photos')
interface LoremPicsumClient {
#Get(value = '{width}/{height}', produces = MediaType.IMAGE_JPEG)
HttpResponse<byte[]> getImage(Integer width, Integer height)
}
And Spock unit test for it:
import io.micronaut.http.HttpStatus
import io.micronaut.test.annotation.MicronautTest
import spock.lang.Specification
import javax.inject.Inject
import java.nio.file.Files
import java.nio.file.Paths
#MicronautTest
class LoremPicsumClientSpec extends Specification {
#Inject
LoremPicsumClient client
void 'image is downloaded'() {
given:
def output = Paths.get('test')
when:
def response = client.getImage(300, 200)
then:
response.status == HttpStatus.OK
response.getBody().isPresent()
when:
Files.write(output, response.getBody().get())
then:
Files.probeContentType(output) == 'image/jpeg'
}
}
In Micronaut the test passes and an image is saved into the test file. But in Grails the test fails because HttpClient is not able to convert the response bytes into byte array or better into anything else then String.

Answer 2

We are currently using this implementation:
#Client(value = "\${image-endpoint}")
interface ImageClient {
#Get("/img")
fun getImageForAddress(
#QueryValue("a") a: String
): CompletableFuture<ByteArray>
}
works fine for us.
When I use the HttpResponse I get an error as well, couldn't make it work with that.

Answer 3

Documentation propose to send bytes via input stream but I didn't manage to make it work. The most brittle thing that HttpClient should Consume bytes but Server should Produce.
#Get(value = "/write", produces = MediaType.TEXT_PLAIN)
HttpResponse<byte[]> write() {
byte[] bytes = "test".getBytes(StandardCharsets.UTF_8);
return HttpResonse.ok(bytes); //
}