Stream upload 'POST' in Spring WebClient - spring

Question

I am uploading streams of (raw bytes) data using HTTP posts using WebClient:
final byte[] rawData = IOUtils.toByteArray(sourceInputStream);
webClient.post()
.uri(uri)
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.bodyValue(rawData)
.exchange()...
I am concerned there is a potentially a lot of memory used given sometimes these objects can be quite big (~200Mb) so would like to read directly from the InputStream and upload as a stream.
I tried:
bodyValue(BodyInserters.fromResource(new InputStreamResource(inputStream)))
but got exception Content type 'application/octet-stream' not supported for bodyType=org.springframework.web.reactive.function.BodyInserters
So I then tried removing the header but the data is then corrupted.
Is there a way to stream the data without passing through the in memory 'buffer' rawData[]?
Thanks

Answer 1

Your first try was almost correct, however you need to use body(...) instead of bodyValue(...):
body(BodyInserters.fromResource(new InputStreamResource(inputStream)))
This is because bodyValue(...) wraps your resource inserter in a value inserter, which will then try to serialize the resource inserter itself and fail with the error you received.

Answer 2

I ended up keeping the rawData[] buffer and specifying contentLength in the request:
webClient.post()
.uri(uri)
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.contentLength(bytes.length)
.bodyValue(bytes)
.exchange()
.block(Duration.ofSeconds(30));
For very large files I did chunked uploads - for example 1Mb chunks of a file might look like:
POST: https://sharepoint.acme.com:443/sharepoint-site/_api/web/GetFileByServerRelativeUrl(#f)/StartUpload(uploadId=guid'680f93eb-8c6a-443d-87ad-1e1b324ea678')?#f='/sharepoint-site/folderA/subFolderB/etc/example.file'
POST: https://sharepoint.acme.com:443/sharepoint-site/_api/web/GetFileByServerRelativeUrl(#f)/ContinueUpload(uploadId=guid'680f93eb-8c6a-443d-87ad-1e1b324ea678',fileOffset=1048576)?#f='/sharepoint-site/folderA/subFolderB/etc/example.file'
POST: https://sharepoint.acme.com:443/sharepoint-site/_api/web/GetFileByServerRelativeUrl(#f)/ContinueUpload(uploadId=guid'680f93eb-8c6a-443d-87ad-1e1b324ea678',fileOffset=2097152)?#f='/sharepoint-site/folderA/subFolderB/etc/example.file'
POST: https://sharepoint.acme.com:443/sharepoint-site/_api/web/GetFileByServerRelativeUrl(#f)/ContinueUpload(uploadId=guid'680f93eb-8c6a-443d-87ad-1e1b324ea678',fileOffset=3145728)?#f='/sharepoint-site/folderA/subFolderB/etc/example.file'
POST: https://sharepoint.acme.com:443/sharepoint-site/_api/web/GetFileByServerRelativeUrl(#f)/ContinueUpload(uploadId=guid'680f93eb-8c6a-443d-87ad-1e1b324ea678',fileOffset=4194304)?#f='/sharepoint-site/folderA/subFolderB/etc/example.file'
POST: https://sharepoint.acme.com:443/sharepoint-site/_api/web/GetFileByServerRelativeUrl(#f)/ContinueUpload(uploadId=guid'680f93eb-8c6a-443d-87ad-1e1b324ea678',fileOffset=5242880)?#f='/sharepoint-site/folderA/subFolderB/etc/example.file'
POST: https://sharepoint.acme.com:443/sharepoint-site/_api/web/GetFileByServerRelativeUrl(#f)/FinishUpload(uploadId=guid'680f93eb-8c6a-443d-87ad-1e1b324ea678',fileOffset=6291456)?#f='/sharepoint-site/folderA/subFolderB/etc/example.file'
Driven by working on the raw bytes:
final UUID id = UUID.randomUUID();
int offset = 0;
while (offset < bytesTotal) {
final URI uri;
if (offset == 0) {
uri = createStartUri(id, path, filename);
} else if (offset < bytesTotal - CHUNK_SIZE) {
uri = createContinueUri(id, offset, path, filename);
} else {
uri = createFinishUri(id, offset, path, filename);
}
final byte[] bytes = ArrayUtils.subarray(fileDataBytes, offset, offset + CHUNK_SIZE);
webClient.post().uri(uri).contentType(MediaType.APPLICATION_OCTET_STREAM).contentLength(bytes.length)
.bodyValue(bytes).exchange().block(Duration.ofSeconds(30));
offset += CHUNK_SIZE;
}