I am trying to add some logging around each response by using a ServerHttpResponseDecorator. The responses are coming back fine via postman, however I’ve noticed I can no longer load my swagger UI (just get a blank white screen).
This does not work, in that the swagger-ui no longer loads.
class ResponseLoggingDecorator(
delegate: ServerHttpResponse
) : ServerHttpResponseDecorator(delegate) {
override fun writeWith(body: Publisher<out DataBuffer>): Mono<Void> {
val buffer = Mono.from(body)
return super.writeWith(buffer.doOnNext { dataBuffer: DataBuffer ->
})
}
}
But this does work (swagger UI loads), a bit obvious but shows something is wrong with the doOnNext?
class ResponseLoggingDecorator(
delegate: ServerHttpResponse
) : ServerHttpResponseDecorator(delegate) {
override fun writeWith(body: Publisher<out DataBuffer>): Mono<Void> {
return super.writeWith(body)
}
}
I'm not hugely knowledgeable on how it works, so I might be wrong, but it should be fine?
The issue was the toMono(), it should instead be toFlux()
Related
I'm building a simple REST API in Kotlin and front-end with React. I'm pretty new to Kotlin and Spring, but I've managed to get a simple service working with Postman. I'm able to return all objects in an SQL table using JpaRepository interface. When I make the call through Postman, my output is as expected (looks like a normal Json object).
However, when I make the call through a standard react fetch the body is produced as a ReadableStream {locked: false}. I did some research on readable streams and they appear to mostly be for images. I tried body.getReader() from https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Using_readable_streams, but that did not work either.
Here is my Kotlin code:
import org.springframework.web.bind.annotation.*
#RestController
#RequestMapping("/api/catalog")
class ArtworkController (
private val artworkService: ArtworkService
) {
#CrossOrigin
#GetMapping("")
#ResponseBody
fun getData(): MutableList<Artwork> {
// println(artworkService.getArt().javaClass.kotlin)
return artworkService.getArt()
}
}
#Service
class ArtworkService(
private val artworkRepository: ArtworkRepository
){
fun getArt(): MutableList<Artwork> {
return artworkRepository.findAll()
}
}
These are in separate files - just joined them here for brevity.
Any help would be greatly appreciated!
It turns out I was immediately returning a promise and had to set up an async function to handle the request.
const { isLoading, error, data } = useQuery("repoData",
async () => {
const data = await fetch("https://url_here")
.then(function(response) {
return response.json()
}
)
setData(data)
}
For context, this request is wrapped in react-query but works without it as well.
Problem
We're developing a Spring Boot service to upload data to different back end databases. The idea is that, in one multipart/form-data request a user will send a "model" (basically a file) and "modelMetadata" (which is JSON that defines an object of the same name in our code).
We got the below to work in the WebFlux annotated controller syntax, when the user sends the "modelMetadata" in the multipart form with the content-type of "application/json":
#PostMapping(consumes = [MediaType.MULTIPART_FORM_DATA_VALUE], produces = [MediaType.APPLICATION_JSON_VALUE])
fun saveModel(#RequestPart("modelMetadata") monoModelMetadata: Mono<ModelMetadata>,
#RequestPart("model") monoModel: Mono<FilePart>,
#RequestHeader headers: HttpHeaders) : Mono<ResponseEntity<ModelMetadata>> {
return modelService.saveModel(monoModelMetadata, monoModel, headers)
}
But we can't seem to figure out how to do the same thing in Webflux's functional router definition. Below are the relevant code snippets we have:
#Bean
fun modelRouter() = router {
accept(MediaType.MULTIPART_FORM_DATA).nest {
POST(ROOT, handler::saveModel)
}
}
fun saveModel(r: ServerRequest): Mono<ServerResponse> {
val headers = r.headers().asHttpHeaders()
val monoModelPart = r.multipartData().map { multiValueMap ->
it["model"] // What do we do with this List<Part!> to get a Mono<FilePart>
it["modelMetadata"] // What do we do with this List<Part!> to get a Mono<ModelMetadata>
}
From everything we've read, we should be able to replicate the same functionality found in the annotation controller syntax with the router functional syntax, but this particular aspect doesn't seem to be well documented. Our goal was to move over to use the new functional router syntax since this is a new application we're developing and there are some nice forward thinking features/benefits as described here.
What we've tried
Googling to the ends of the Earth for a relevant example
this is a similar question, but hasn't gained any traction and doesn't relate to our need to create an object from one piece of the multipart request data
this may be close to what we need for uploading the file component of our multipart request data, but doesn't handle the object creation from JSON
Tried looking at the #RequestPart annotation code to see how things are done on that side, there's a nice comment that seems to hint at how they are converting the parts to objects, but we weren't able to figure out where that code lives or any relevant example of how to use an HttpMessageConverter on the ``
the content of the part is passed through an {#link HttpMessageConverter} taking into consideration the 'Content-Type' header of the request part.
Any and all help would be appreciated! Even just some links for us to better understand Part/FilePart types and there role in multipart requests would be helpful!
I was able to come up with a solution to this issue using an autowired ObjectMapper. From the below solution I could turn the modelMetadata and modelPart into Monos to mirror the #RequestPart return types, but that seems ridiculous.
I was also able to solve this by creating a MappingJackson2HttpMessageConverter and turning the metadataDataBuffer into a MappingJacksonInputMessage, but this solution seemed better for our needs.
fun saveModel(r: ServerRequest): Mono<ServerResponse> {
val headers = r.headers().asHttpHeaders()
return r.multipartData().flatMap {
// We're only expecting one Part of each to come through...assuming we understand what these Parts are
if (it.getOrDefault("modelMetadata", listOf()).size == 1 && it.getOrDefault("model", listOf()).size == 1) {
val modelMetadataPart = it["modelMetadata"]!![0]
val modelPart = it["model"]!![0] as FilePart
modelMetadataPart
.content()
.map { metadataDataBuffer ->
// TODO: Only do this if the content is JSON?
objectMapper.readValue(metadataDataBuffer.asInputStream(), ModelMetadata::class.java)
}
.next() // We're only expecting one object to be serialized from the buffer
.flatMap { modelMetadata ->
// Function was updated to work without needing the Mono's of each type
// since we're mapping here
modelService.saveModel(modelMetadata, modelPart, headers)
}
}
else {
// Send bad request response message
}
}
Although this solution works, I feel like it's not as elegant as the one alluded to in the #RequestPart annotation comments. Thus I will accept this as the solution for now, but if someone has a better solution please let us know and I will accept it!
I am trying to make requests using WebClient in parallel, but I have no clue how to go about that,
because no matter what I do, the code is not waiting for requests to finish. If I execute just one request though (Commented fragment), everything works fine. Can someone help me with that?
#RequestMapping(method = [RequestMethod.POST], path = ["/upload/{batchId}"])
fun uploadFile(#RequestPart("file") file: Mono<FilePart>,
#PathVariable("batchId") batchId:String,
#RequestHeader("FILE-SIZE") fileSize:Int): Mono<ServiceResponse> {
val webClient = WebClient.create(commandEndpoint)
// return webClient.put().uri(seriesPath).retrieve().bodyToMono(String::class.java).map { ServiceResponse(it,0) }
return file.map{it.transferTo(Paths.get(storagePath,"excel"))}
.map{excelWorkbookToMetadata(WorkbookFactory.create(Paths.get(storagePath,"excel").toFile()))}
.flatMapMany{Flux.fromIterable(it)}
.flatMap {
it.transactionId = batchId
when (it) {
is SeriesMetadata -> webClient.put().uri(seriesPath,it.id)
.body(BodyInserters.fromObject(it))
.retrieve()
.onStatus({ it == HttpStatus.BAD_REQUEST },{
println("ERROR")
Mono.error(RuntimeException("blah")) }).toMono()
else -> Mono.error(NotImplementedError(""))
}
}
.collectList()
.map {ServiceResponse(batchId, it.size*2) }
}
So it seems, that collectList() filters out empty mono that are returned in case the body of the response is empty. The solution is basically, either to use Mono.defaultIfEmpty() method, or change retrieve() to exchange() which always returns something. At least that's what helped me.
I am writing tests for the method provide below.
`
class ScrapedRecipeCache #Autowired constructor(private val cache: RecipeScrapingCacheService,
private val recipeService: RecipeService) : ScrapedRecipeProvider {
override fun provide(request: ScrapingRequest): Flux<ScrapedRecipe> =
cache.retrieve(request.link)
.doOnNext { println(it) }
.flatMap { (link, _, recipeHash, error) ->
recipeService.findByHash(recipeHash)
.map { ScrapedRecipe(it, link, error)}
.switchIfEmpty(cache.remove(request.link).then(Mono.empty()))
}
.flux()
}
`
The test looks as follows:
private val recipeFetched = Recipe("Tortellini", RecipeDifficulty.EASY, 15.0)
val cacheContents = RecipeScrapingResource("www.google.com", ScrapingOrigin.JAMIE_OLIVER, recipeFetched.hash,
mutableListOf(
pl.goolash.core.Exception("aa", ErrorType.WARNING, LocalDateTime.MIN)
))
val request = ScrapingRequest("www.google.com", ScrapingOrigin.JAMIE_OLIVER, 4)
#BeforeEach
fun setUp() {
given(cache.retrieve("www.google.com")).willReturn(Mono.just(cacheContents))
given(recipeService.findByHash(recipeFetched.hash)).willReturn(Mono.just(recipeFetched))
}
#Test
#DisplayName("Then return data fetched from service and don't delete cache")
fun test() {
cacheFacade.provide(request)
.test()
.expectNext(ScrapedRecipe(recipeFetched, "www.google.com", cacheContents.error!!))
.expectComplete()
.verify()
BDDMockito.verify(cache, BDDMockito.never()).remove(request.link)
}
The test fails because cache.remove(request.link) is called. To my understanding (or from what I managed to gather from documentation) switchIfEmpty, should only be fired when recipeService.findByHash returns Mono.empty(). However the debugger shows that it returns mocked value of Mono.just(fetchedRecipe).
The interesting thing is that when I replace
.switchIfEmpty(cache.remove(request.link).then(Mono.empty()))
with
.switchIfEmpty(Mono.just(1).doOnNext{println("weeee")}.then(Mono.empty()))
Then weee is not printed hence it behaves as expected, that is switchIfEmpty is not fired.
Furthermore the tested issue runs properly in integration test and does not clear the cache.
Reactor version : 3.1.0-RC1
Other notable details: Spring Boot 2.0.0-M4, Mockito-core:2.10, junit 5, project is written in kotlin
The question is, does anybody see anything wrong with this? Because I have spent two days over and still have no clue why this behaves so bizzarely.
Finally I found out how to make this work.
In order to remedy it:
override fun provide(request: ScrapingRequest): Flux<ScrapedRecipe> =
cache.retrieve(request.link)
.flatMap { (link, _, recipeHash, error) ->
recipeService.findByHash(recipeHash)
.map { ScrapedRecipe(it, link, error) }
.switchIfEmpty(Mono.just(1)
.flatMap { cache.remove(request.link) }
.then(Mono.empty()))
}
.flux()
You can see how using flatMap to execute the asynch work does the job, even if this is not the neatest implementation, it revealed to me quite an interesting mechanism hidden here.
We are working with spring boot 2.0.0.BUILD_SNAPSHOT and spring boot webflux 5.0.0 and currently we cant transfer a flux to a client on request.
Currently I am creating the flux from an iterator:
public Flux<ItemIgnite> getAllFlux() {
Iterator<Cache.Entry<String, ItemIgnite>> iterator = this.getAllIterator();
return Flux.create(flux -> {
while(iterator.hasNext()) {
flux.next(iterator.next().getValue());
}
});
}
And on request I am simply doing:
#RequestMapping(value="/all", method=RequestMethod.GET, produces="application/json")
public Flux<ItemIgnite> getAllFlux() {
return this.provider.getAllFlux();
}
When I now locally call localhost:8080/all after 10 seconds I get a 503 status code. Also as at client when I request /all using the WebClient:
public Flux<ItemIgnite> getAllPoducts(){
WebClient webClient = WebClient.create("http://localhost:8080");
Flux<ItemIgnite> f = webClient.get().uri("/all").accept(MediaType.ALL).exchange().flatMapMany(cr -> cr.bodyToFlux(ItemIgnite.class));
f.subscribe(System.out::println);
return f;
}
Nothing happens. No data is transferred.
When I do the following instead:
public Flux<List<ItemIgnite>> getAllFluxMono() {
return Flux.just(this.getAllList());
}
and
#RequestMapping(value="/allMono", method=RequestMethod.GET, produces="application/json")
public Flux<List<ItemIgnite>> getAllFluxMono() {
return this.provider.getAllFluxMono();
}
It is working. I guess its because all data is already finished loading and just transferred to the client as it usually would transfer data without using a flux.
What do I have to change to get the flux streaming the data to the web client which requests those data?
EDIT
I have data inside an ignite cache. So my getAllIterator is loading the data from the ignite cache:
public Iterator<Cache.Entry<String, ItemIgnite>> getAllIterator() {
return this.igniteCache.iterator();
}
EDIT
adding flux.complete() like #Simon Baslé suggested:
public Flux<ItemIgnite> getAllFlux() {
Iterator<Cache.Entry<String, ItemIgnite>> iterator = this.getAllIterator();
return Flux.create(flux -> {
while(iterator.hasNext()) {
flux.next(iterator.next().getValue());
}
flux.complete(); // see here
});
}
Solves the 503 problem in the browser. But it does not solve the problem with the WebClient. There is still no data transferred.
EDIT 3
using publishOn with Schedulers.parallel():
public Flux<ItemIgnite> getAllFlux() {
Iterator<Cache.Entry<String, ItemIgnite>> iterator = this.getAllIterator();
return Flux.<ItemIgnite>create(flux -> {
while(iterator.hasNext()) {
flux.next(iterator.next().getValue());
}
flux.complete();
}).publishOn(Schedulers.parallel());
}
Does not change the result.
Here I post you what the WebClient receives:
value :[Item ID: null, Product Name: null, Product Group: null]
complete
So it seems like he is getting One item (out of over 35.000) and the values are null and he is finishing after.
One thing that jumps out is that you never call flux.complete() in your create.
But there's actually a factory operator that is tailored to transform an Iterable to a Flux, so you could just do Flux.fromIterable(this)
Edit: in case your Iterator is hiding complexity like a DB request (or any blocking I/O), be advised this spells trouble: anything blocking in a reactive chain, if not isolated on a dedicated execution context using publishOn, has the potential to block not only the entire chain but other reactive processes has well (as threads can and will be used by multiple reactive processes).
Neither create nor fromIterable do anything in particular to protect from blocking sources. I think you are facing that kind of issue, judging from the hang you get with the WebClient.
The problem was my Object ItemIgnite which I transfer. The system Flux seems not to be able to handle this. Because If I change my original code to the following:
public Flux<String> getAllFlux() {
Iterator<Cache.Entry<String, ItemIgnite>> iterator = this.getAllIterator();
return Flux.create(flux -> {
while(iterator.hasNext()) {
flux.next(iterator.next().getValue().toString());
}
});
}
Everything is working fine. Without publishOn and without flux.complete(). Maybe someone has an idea why this is working.