I have the following reactive repository:
#Repository
public interface FooCosmosRepository extends ReactiveCosmosRepository<Foo, String> {
}
I am using it as following:
#Override
public Mono<FooResponse> getFooDetails() {
FooResponse fooResponse = new FooResponse();
fooResponse.setCount(1000);
List<Foo> fooList = new ArrayList<>();
repository.findAll().collectList().flatMap(e ->{
//This is not invoked. findAll return Flux<T> in this case Flux<Foo>
for (Foo foo : e) {
fooList.add(foo);
}
return null;
});
fooResponse.setFooList(fooList);
return Mono.just(fooResponse);
}
FooResponse is defined as follows:
#NoArgsConstructor
#Data
#FieldDefaults(level = AccessLevel.PRIVATE)
public class FooResponse {
int rowCount;
List<Foo> fooList;
}
I cant block cause I get error
Iterating over a toIterable() / toStream() is blocking, which is not supported in thread reactor-http-nio-6
I cant return Flux<T> from the method also. I need to return Mono<FooResponse>. How can i query repository, actually get/collect response and add to the list?
Any ideas?
It's because you are coding imperatively instead of reactively. and you are breaking the chain which means that Reactor can't complete the assembly phase, and then execute during the subscription phase.
#Override
public Mono<FooResponse> getFooDetails() {
return repository.findAll()
.collectList()
.map(list -> {
FooResponse fooResponse = new FooResponse();
fooResponse.setCount(1000);
fooResponse.setList(list);
return fooResponse;
});
}
This is basic reactor and I recommend the following links:
Reactor Core Features
Flight of the flux
Related
I'm currently updating from Spring boot 2.2.x to 2.6.x + legacy code, it's a big jump so there were multiple changes. I'm now running into a problem with load balancing through an api-gateway. I'll apologize in advance for the wall of code to come. I will put the point of failure at the bottom.
When I send in an API request, I get the following error:
more than one 'primary' bean found among candidates: [zookeeperDiscoveryClientServiceInstanceListSupplier, serviceInstanceListSupplier, retryAwareDiscoveryClientServiceInstanceListSupplier]
it seems that the zookeeperDiscovery and retryAware suppliers are loaded through the default serviceInsatnceListSupplier, which has #Primary over it. I thought would take precedence over the other ones. I assume I must be doing something wrong due changes in the newer version, here are the relevant code in question:
#Configuration
#LoadBalancerClients(defaultConfiguration = ClientConfiguration.class)
public class WebClientConfiguration {
#Bean
#Qualifier("microserviceWebClient")
#ConditionalOnMissingBean(name = "microserviceWebClient")
public WebClient microserviceWebClient(#Qualifier("microserviceWebClientBuilder") WebClient.Builder builder) {
return builder.build();
}
#Bean
#Qualifier("microserviceWebClientBuilder")
#ConditionalOnMissingBean(name = "microserviceWebClientBuilder")
#LoadBalanced
public WebClient.Builder microserviceWebClientBuilder() {
return WebClient.builder();
}
#Bean
#Primary
public ReactorLoadBalancerExchangeFilterFunction reactorLoadBalancerExchangeFilterFunction(
ReactiveLoadBalancer.Factory<ServiceInstance> loadBalancerFactory) {
//the transformer is currently null, there wasn't a transformer before the upgrade
return new CustomExchangeFilterFunction(loadBalancerFactory, transformer);
}
}
There are also some Feign Client related configs here which I will omit, since it's not (or shouldn't be) playing a role in this problem:
public class ClientConfiguration {
/**
* The property key within the feign clients configuration context for the feign client name.
*/
public static final String FEIGN_CLIENT_NAME_PROPERTY = "feign.client.name";
public ClientConfiguration() {
}
//Creates a new BiPredicate for shouldClose. This will be used to determine if HTTP Connections should be automatically closed or not.
#Bean
#ConditionalOnMissingBean
public BiPredicate<Response, Type> shouldClose() {
return (Response response, Type type) -> {
if(type instanceof Class) {
Class<?> currentClass = (Class<?>) type;
return (null == AnnotationUtils.getAnnotation(currentClass, EnableResponseStream.class));
}
return true;
};
}
//Creates a Custom Decoder
#Bean
public Decoder createCustomDecoder(
ObjectFactory<HttpMessageConverters> converters, BiPredicate<Response, Type> shouldClose
) {
return new CustomDecoder(converters, shouldClose);
}
#Bean
#Qualifier("loadBalancerName")
public String loadBalancerName(PropertyResolver propertyResolver) {
String name = propertyResolver.getProperty(FEIGN_CLIENT_NAME_PROPERTY);
if(StringUtils.hasText(name)) {
// we are in a feign context
return name;
}
// we are in a LoadBalancerClientFactory context
name = propertyResolver.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
Assert.notNull(name, "Could not find a load balancer name within the configuration context!");
return name;
}
#Bean
public ReactorServiceInstanceLoadBalancer reactorServiceInstanceLoadBalancer(
BeanFactory beanFactory, #Qualifier("loadBalancerName") String loadBalancerName
) {
return new CustomRoundRobinLoadBalancer(
beanFactory.getBeanProvider(ServiceInstanceListSupplier.class),
loadBalancerName
);
}
#Bean
#Primary
public ServiceInstanceListSupplier serviceInstanceListSupplier(
#Qualifier(
"filter"
) Predicate<ServiceInstance> filter, DiscoveryClient discoveryClient, Environment environment, #Qualifier(
"loadBalancerName"
) String loadBalancerName
) {
// add service name to environment if necessary
if(environment.getProperty(LoadBalancerClientFactory.PROPERTY_NAME) == null) {
StandardEnvironment wrapped = new StandardEnvironment();
if(environment instanceof ConfigurableEnvironment) {
((ConfigurableEnvironment) environment).getPropertySources()
.forEach(s -> wrapped.getPropertySources().addLast(s));
}
Map<String, Object> additionalProperties = new HashMap<>();
additionalProperties.put(LoadBalancerClientFactory.PROPERTY_NAME, loadBalancerName);
wrapped.getPropertySources().addLast(new MapPropertySource(loadBalancerName, additionalProperties));
environment = wrapped;
}
return new FilteringInstanceListSupplier(filter, discoveryClient, environment);
}
}
There was a change in the ExchangeFilter constructor, but as far as I can tell, it accepts that empty transformer,I don't know if it's supposed to:
public class CustomExchangeFilterFunction extends ReactorLoadBalancerExchangeFilterFunction {
private static final ThreadLocal<ClientRequest> REQUEST_HOLDER = new ThreadLocal<>();
//I think it's wrong but I don't know what to do here
private static List<LoadBalancerClientRequestTransformer> transformersList;
private final Factory<ServiceInstance> loadBalancerFactory;
public CustomExchangeFilterFunction (Factory<ServiceInstance> loadBalancerFactory) {
this(loadBalancerFactory);
///according to docs, but I don't know where and if I need to use this
#Bean
public LoadBalancerClientRequestTransformer transformer() {
return new LoadBalancerClientRequestTransformer() {
#Override
public ClientRequest transformRequest(ClientRequest request, ServiceInstance instance) {
return ClientRequest.from(request)
.header(instance.getInstanceId())
.build();
}
};
}
public CustomExchangeFilterFunction (Factory<ServiceInstance> loadBalancerFactory, List<LoadBalancerClientRequestTransformer> transformersList) {
super(loadBalancerFactory, transformersList); //the changed constructor
this.loadBalancerFactory = loadBalancerFactory;;
}
#Override
public Mono<ClientResponse> filter(ClientRequest request, ExchangeFunction next) {
// put the current request into the thread context - ugly, but couldn't find a better way to access the request within
// the choose method without reimplementing nearly everything
REQUEST_HOLDER.set(request);
try {
return super.filter(request, next);
} finally {
REQUEST_HOLDER.remove();
}
}
//used to be an override, but the function has changed
//code execution doesn't even get this far yet
protected Mono<Response<ServiceInstance>> choose(String serviceId) {
ReactiveLoadBalancer<ServiceInstance> loadBalancer = loadBalancerFactory.getInstance(serviceId);
if(loadBalancer == null) {
return Mono.just(new EmptyResponse());
}
ClientRequest request = REQUEST_HOLDER.get();
// this might be null, if the underlying implementation changed and this method is no longer executed in the same
// thread
// as the filter method
Assert.notNull(request, "request must not be null, underlying implementation seems to have changed");
return choose(loadBalancer, filter);
}
protected Mono<Response<ServiceInstance>> choose(
ReactiveLoadBalancer<ServiceInstance> loadBalancer,
Predicate<ServiceInstance> filter
) {
return Mono.from(loadBalancer.choose(new DefaultRequest<>(filter)));
}
}
There were pretty big changes in the CustomExchangeFilterFunction, but the current execution doesn't even get there. It fails here, in .getIfAvailable(...):
public class CustomRoundRobinLoadBalancer implements ReactorServiceInstanceLoadBalancer {
private static final int DEFAULT_SEED_POSITION = 1000;
private final ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider;
private final String serviceId;
private final int seedPosition;
private final AtomicInteger position;
private final Map<String, AtomicInteger> positionsForVersions = new HashMap<>();
public CustomRoundRobinLoadBalancer (
ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider,
String serviceId
) {
this(serviceInstanceListSupplierProvider, serviceId, new Random().nextInt(DEFAULT_SEED_POSITION));
}
public CustomRoundRobinLoadBalancer (
ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider,
String serviceId,
int seedPosition
) {
Assert.notNull(serviceInstanceListSupplierProvider, "serviceInstanceListSupplierProvider must not be null");
Assert.notNull(serviceId, "serviceId must not be null");
this.serviceInstanceListSupplierProvider = serviceInstanceListSupplierProvider;
this.serviceId = serviceId;
this.seedPosition = seedPosition;
this.position = new AtomicInteger(seedPosition);
}
#Override
// we have no choice but to use the raw type Request here, because this method overrides another one with this signature
public Mono<Response<ServiceInstance>> choose(#SuppressWarnings("rawtypes") Request request) {
//fails here!
ServiceInstanceListSupplier supplier = serviceInstanceListSupplierProvider
.getIfAvailable(NoopServiceInstanceListSupplier::new);
return supplier.get().next().map((List<ServiceInstance> instances) -> getInstanceResponse(instances, request));
}
}
Edit: after some deeper stacktracing, it seems that it does go into the CustomFilterFunction and invokes the constructor with super(loadBalancerFactory, transformer)
I found the problem or a workaround. I was using #LoadBalancerClients because I thought it would just set the same config for all clients that way (even if I technically only have one atm). I changed it to ##LoadBalancerClient and it suddenly worked. I don't quite understand why this made a difference but it did!
I'm fairly new to Spring Batch and I would like to know if the following is possible. I am creating a batch job that requires some initial data queried from a database. So would be something like the following be possible. Below is an item reader the first step in the job execution. This initial step will query all FOOs:
#Bean
public JdbcCursorItemReader fooItemReader() {
return
new JdbcCursorItemReaderBuilder()
.dataSource(dataSource)
.sql(QueryConstants.ALL_FOOS)
.build();
}
#Bean(name = "step-one")
public Step stepOne() {
return stepBuilderFactory
.get("step-one")
.reader(fooItemReader())
...
.build();
}
The above cursor reader and step would ideally pull all of the "FOO"s from a a database. Given the above, I would like to know if it is possible to take the data from step-one and pass it to a step-two to select all "BAR"s for the given "FOO" from the list of "FOO"s from step-one. Something like below:
#Bean
public JdbcCursorItemReader barsForFooItemReader() {
//some how for each foo in the list of FOOs from step-one retrieve it's BARs
return
new JdbcCursorItemReaderBuilder()
.dataSource(dataSource)
.sql(QueryConstants.BARS_FOR_FOO)
.build();
}
#Bean(name = "step-two")
public Step stepOne() {
return stepBuilderFactory
.get("step-one")
.reader(barsForFooItemReader())
...
.build();
}
Is something like this possible with Spring Batch? Is Spring Batch "right" for something like this? Any help would be most appreciated!!
UPDATE:
I looked into the suggested posted from the comments and they are not exactly what I'm looking for and leaves some unanswered questions. So I'm going to to attempt to clarify what I am trying to accomplish with Spring Batch by adding an example (below) of how I would implement the above without Spring Batch and using JdbcTemplate.
#Repository
public class FooBarRepository {
private final NamedParameterJdbcTemplate namedParameterJdbcTemplate;
#Autowired
public FooRepository(NamedParameterJdbcTemplate namedParameterJdbcTemplate) {
this.namedParameterJdbcTemplate = namedParameterJdbcTemplate;
}
/**
* Get all FOOs from database
* #return
*/
public List<String> getAllFoos() {
String query = "SELECT * FROM FOO;";
try {
return namedParameterJdbcTemplate.query(query, (resultSet, i) -> resultSet.getString("FOO_NAME"))';'
}
catch (DataAccessException ex) {
//... log exception
return Collections.emptyList();
}
}
/**
* Get all Bars for a given foo name
* #param fooName
* #return
*/
public List<String> getBarsForFoo(String fooName) {
String query = "SELECT * FROM BAR CROSS JOIN FOO WHERE FOO.NAME = :fooName";
try {
return namedParameterJdbcTemplate.query(query, new MapSqlParameterSource().addValue("fooName", fooName), (resultSet, i) -> resultSet.getString("BarFoo"));
}
catch (DataAccessException ex) {
//... log exception
return Collections.emptyList();
}
}
}
Above is a simple dao repo. There's a query for retrieving all FOOs and another for retrieving all Bars for a given fooName. Below is how I would would use both methods.
#Component
public class FooBarProcessor {
private final FooRepository fooRepository;
#Autowired
public FooBarProcessor(FooRepository fooRepository) {
this.fooRepository = fooRepository;
}
public void processFooBars() {
List<String> foos = fooRepository.getAllFoos();
Map<String, List<String>> foobars = new HashMap<>();
foos.forEach(foo -> {
List<String> bars = fooRepository.getBarsForFoo(foo);
foobars.put(foo, bars);
});
}
}
In the above, I get a list of all the FOOs and then for each FOO I retrieve it's BARs. (I hope the above makes sense logically)
I want to accomplish something similar with Spring Batch as I expect the data set to be quite large.
In Spring Reactive Java how can I write an updateById() method using the Router and Handler?
For example, the Router has this code:
RouterFunctions.route(RequestPredicates.PUT("/employees/{id}").and(RequestPredicates.accept(MediaType.APPLICATION_JSON))
.and(RequestPredicates.contentType(MediaType.APPLICATION_JSON)),
employeeHandler::updateEmployeeById);
My question is how to write the employeeHandler::updateEmployeeById() keeping the ID as same but changing the other members of the Employee object?
public Mono<ServerResponse> updateEmployeeById(ServerRequest serverRequest) {
Mono<Employee> employeeMono = serverRequest.bodyToMono(Employee.class);
<And now what??>
return ServerResponse.ok().contentType(MediaType.APPLICATION_JSON).body(employeeMono, Employee.class);
}
The Employee class looks like this:
#Document
#Data
#AllArgsConstructor
#NoArgsConstructor
public class Employee {
#Id
int id;
double salary;
}
Thanks for any help.
First of all, you have to add ReactiveMongoRepository in your classpath. You can also read about it here.
#Repository
public interface EmployeeRepository extends ReactiveMongoRepository<Employee, Integer> {
Mono<Employee> findById(Integer id);
}
Then your updateEmployeeById method can have the following structure:
public Mono<ServerResponse> updateEmployeeById(ServerRequest serverRequest) {
return serverRequest
.bodyToMono(Employee.class)
.doOnSubscribe(e -> log.info("update employee request received"))
.flatMap(employee -> {
Integer id = Integer.parseInt(serverRequest.pathVariable("id"));
return employeeRepository
.findById(id)
.switchIfEmpty(Mono.error(new NotFoundException("employee with " + id + " has not been found")))
// what you need to do is to update already found entity with
// new values. Usually map() function is used for that purpose
// because map is about 'transformation' what is setting new
// values in our case
.map(foundEmployee -> {
foundEmployee.setSalary(employee.getSalary());
return foundEmployee;
});
})
.flatMap(employeeRepository::save)
.doOnError(error -> log.error("error while updating employee", error))
.doOnSuccess(e -> log.info("employee [{}] has been updated", e.getId()))
.flatMap(employee -> ServerResponse.ok().contentType(MediaType.APPLICATION_JSON).body(BodyInserters.fromValue(employee), Employee.class));
}
UPDATE:
Based on Prana's answer, I have updated the code above merging our solutions in one. Logging with a help of Slf4j was added. And switchIfEmpty() functions for the case when the entity was not found.
I would also suggest your reading about global exception handling which will make your API even better. A part of it I can provide here:
/**
* Returns routing function.
*
* #param errorAttributes errorAttributes
* #return routing function
*/
#Override
protected RouterFunction<ServerResponse> getRoutingFunction(ErrorAttributes errorAttributes) {
return RouterFunctions.route(RequestPredicates.all(), this::renderErrorResponse);
}
private HttpStatus getStatus(Throwable error) {
HttpStatus status;
if (error instanceof NotFoundException) {
status = NOT_FOUND;
} else if (error instanceof ValidationException) {
status = BAD_REQUEST;
} else {
status = INTERNAL_SERVER_ERROR;
}
return status;
}
/**
* Custom global error handler.
*
* #param request request
* #return response
*/
private Mono<ServerResponse> renderErrorResponse(ServerRequest request) {
Map<String, Object> errorPropertiesMap = getErrorAttributes(request, false);
Throwable error = getError(request);
HttpStatus errorStatus = getStatus(error);
return ServerResponse
.status(errorStatus)
.contentType(APPLICATION_JSON)
.body(BodyInserters.fromValue(errorPropertiesMap));
}
A slightly different version of the above worked without any exceptions:
public Mono<ServerResponse> updateEmployeeById(ServerRequest serverRequest) {
Mono<ServerResponse> notFound = ServerResponse.notFound().build();
Mono<Employee> employeeMono = serverRequest.bodyToMono(Employee.class);
Integer employeeId = Integer.parseInt(serverRequest.pathVariable("id"));
employeeMono = employeeMono.flatMap(employee -> employeeRepository.findById(employeeId)
.map(foundEmployee -> {
foundEmployee.setSalary(employee.getSalary());
return foundEmployee;
})
.flatMap(employeeRepository::save));
return ServerResponse.ok().contentType(MediaType.APPLICATION_JSON).body(employeeMono, Employee.class).switchIfEmpty(notFound);
}
Thanks to Stepan Tsybulski.
I can't manage to inject a property from application.yml to a spring data #Query.
The following results in an EL1008E error:
public interface MyRepository extends JpaRepository<MyEntity, Long> {
#Query("select e from MyEntity e where e.foo = :foo and e.env= ?#{env}")
MyEntity findByFoo(#Param("foo") String foo);
}
According to this blog it is possible to inject a property of spring's principal, which is not very different from what I would like to do.
Any hints on this?
I should really stop asking questions and answer them by myself shortly after ... That is not on purpose.
The mentioned blog has the solution included. Add this:
public class PropertyEvaluationContextExtension extends EvaluationContextExtensionSupport {
private final MyProps p;
public PropertyEvaluationContextExtension(final MyProps p) {
this.p= p;
}
#Override
public String getExtensionId() {
return "foo";
}
#Override
public MyProps getRootObject() {
return this.p;
}
}
#Configuration
public class PropertyConfig {
private final MyProps p;
public PropertyConfig(final MyProps p) {
this.p= p;
}
#Bean
EvaluationContextExtensionSupport propertyExtension() {
return new PropertyEvaluationContextExtension(p);
}
}
Now every property of MyProps is accessible via SpEL.
I generate a list of instances which implement interface CheckInterface
using reflection, but some of those instances need to use JPA repository
,as they are new by reflection so Spring will not inject repository for them.
So how can I inject repository into these instance
Or how can I delegate these class to spring as #Bean and iterate them?
#Component
#Log4j
public class ValidationRegister {
public static HashMap<String, CheckItem> itemHashMap = new HashMap<>();
static {
if (itemHashMap.size() == 0) {
Reflections reflections = new Reflections(CheckItem.class.getPackage().getName());
Set<Class<? extends CheckItem>> itemClasses =
reflections.getSubTypesOf(CheckItem.class);
for (Class<? extends CheckItem> checkItemClass : itemClasses) {
try {
itemHashMap.put(checkItemClass.getName(), checkItemClass.newInstance());
} catch (Exception e) {
log.error("ValidationRegister fail : ", e);
}
}
}
}
}
public class LPHYCZ extends CheckItem {
#Autowired
ClaimDataEntityRepository claimDataEntityRepository;
#Override
public boolean check(ClaimRequest request, List<String> errorList) {
ClaimDataEntity claimDataEntity = claimDataEntityRepository.findByClaimId(request.getClaimId());
if (claimDataEntity != null) {
return true;
}
return false;
}
}
When I get instance of LPHYCZ from ValidationRegister.itemHashMap its
claimDataEntityRepository will be null
Just make your Reflections reflections a Spring Bean and get it injected into 'ValidationRegister' then use it to initialize your HashMap your hashmap still can be static, if you want it to be shared by multiple ValidationRegister instances, but the injection won't happen on static fields.