I've noticed that org.springframework.data.mongodb.core.aggregation.AggregationOption class covers just a small subset of these options described within MongoDB Aggregation pipeline documentation: https://docs.mongodb.com/manual/reference/command/aggregate/#syntax
I need to set maxTimeMS option, but it is not available within org.springframework.data.mongodb.core.aggregation.AggregationOption:
public class AggregationOptions {
private static final String BATCH_SIZE = "batchSize";
private static final String CURSOR = "cursor";
private static final String EXPLAIN = "explain";
private static final String ALLOW_DISK_USE = "allowDiskUse";
private static final String COLLATION = "collation";
private static final String COMMENT = "comment";
...
However, the another class (of mongodb-driver) actually has such field maxTimeMS,
com.mongodb.AggregationOptions:
public class AggregationOptions {
private final Integer batchSize;
private final Boolean allowDiskUse;
private final OutputMode outputMode;
private final long maxTimeMS;
private final Boolean bypassDocumentValidation;
private final Collation collation;
...
Any idea/tricks how to set this maxTimeMS for Aggregation query using Spring Data MongoDB API? Or maybe do I need to build/write such aggregation by using native query?
Btw. I know that Spring Data MongoDB supports maxTimeMS for find operations, for example:
Query.query(mongoCriteria).with(pageable).maxTime(Duration.ofMinutes(4))
However I need to set aggregation query processing timeout on server side in order to prevent "never ending" queries that kill performance.
spring-data-mongodb:2.2.0.RELEASE
I was trying to set a time limit to my query and it took me some time to find the answer; so, I want to share my method with anyone who might be interested:
AggregationOptions aggregationOptions = new AggregationOptions.Builder().maxTime(Duration.ofSeconds(3)).build();
Aggregation aggregate = Aggregation.newAggregation(match).withOptions(aggregationOptions);
Related
We have model class saved in Redis as mentioned below:-
#Data
#NoArgsConstructor
#AllArgsConstructor
#RedisHash("book")
public class Book implements Serializable {
private static final long serialVersionUID = 2208852329346517265L;
#Id
private Integer bookID;
#Indexed
private String title;
#Indexed
private String authors;
private String averageRating;
private String isbn;
private String languageCode;
private String ratingsCount;
private BigDecimal price;
}
We have title and authors as our indexed property.
Now we wanted to search all the records from Redis by passing title and a list of authors using the spring crud repository as mentioned below.
public interface BookSpringRepository extends CrudRepository<Book, String> {
List<Book> findAllByTitleAndAuthors(String title, List<String> authors);
}
Service layer:-
#Override
public Optional<List<Book>> searchBooksByTitleAndAuthorNames(String title, List<String>
autherNames) {
return Optional.ofNullable(bookSpringRepository.findAllByTitleAndAuthors(title,
autherNames));
}
Here we are getting below exception
Unable to fetch data from Spring data Redis cache using List of Integer or
String.
Getting error while fetching - "Resolved
[org.springframework.core.convert.ConversionFailedException: Failed to convert from type
[java.lang.String] to type [byte] for value 'Ronak';
nested exception is java.lang.NumberFormatException: For input string: "Ronak"]."
We would not want to convert the list of string/integer to byte as it is a time-consuming process and as we tried took so much amount of time. Also when the results are retrieved we will again have to convert back to normal integer or string values.
The other option is to loop through the list and pass a single value at a time to the Redis crud repository and this time Redis crud repository is happy but that will be a loop call to Redis and network latency.
We cannot add ID attributes on authors' property as these can be duplicate records.
Does the spring crud repository support the LIKE query in search that way we can create a unique id having these authors' names and make put ID annotation on that new derived property to search the records using spring crud repository using LIKE or contains kind of query.
Any suggestions here are highly appreciated!!
Try to add serialization to your redis key and value. This might help :
https://medium.com/#betul5634/redis-serialization-with-spring-redis-data-lettuce-codec-1a1d2bc73d26
I am trying to use Java8 groupby and reduce feature to sum elements based on grouping but the code doesnt seem to be working for me.
Here is my data from the Dtolist that I am trying to map and extract in an excel sheet.
My requirement is that for every DTO object I want this kilometersNumber to be summed up based on groupingBy TruckId. So , the highlighted columns should be having summed up values for same TruckId(**Contract flotte)
This is my DTO:
'''public class CostsTruckWsDTO implements Serializable {
private static final long serialVersionUID = 1L;
private String truckId;
private LocalDate period;
private Bigdecimal KilometersNumber;
private String periodStartDate;
Here is the logic::
''costsTruckWsDTO.setGroupBy(TruckId.toUpperCase());
Map<String, List> mapCostsTruck = costsTruck.stream()
.collect(Collectors.groupingBy(d -> d.getGroupBy().toUpperCase()));
for (Map.Entry<String, List> entry : mapCostsTruck.entrySet()) {
BigDecimal totalKm = entry.getValue().stream().map(x -> x.getKilometersNumber()).reduce(BigDecimal.ZERO,
BigDecimal::add);
}
}''
But what am getting is value for kilometere is not geeting added as per the screenshot.
Can anybody pls help me what I am missing here!!!
Highly appreciate
You can do the operation of grouping together with the operation of summing using two collectors:
Map<String, BigDecimal> result = trucks.stream()
.collect(Collectors.groupingBy(CostsTruckWsDTO::getTruckId,
Collectors.mapping(CostsTruckWsDTO::getKilometersNumber, Collectors.reducing(BigDecimal.ZERO, BigDecimal::add))));
I am using Spring data for Elastic Search and am using the ReactiveCrudRepository for stuff like finding and deleting. I noticed that with attributes that are in root and are simple objects, the deletion works (deleteByAttributeName). However if I have nested objects then it does not work.
Here's my entities
Book
#Data
#TypeAlias("book")
#Document(indexName = "book")
public class EsBook{
#Field(type = FieldType.Long)
private Long id;
#Field(type = FieldType.Nested)
private EsStats stats;
#Field(type = FieldType.Date, format = DateFormat.date)
private LocalDate publishDate;
}
Stats
#Data
#Builder
#NoArgsConstructor
#AllArgsConstructor
#EqualsAndHashCode
public class EsStats{
#Field(type = FieldType.Double)
private Double averageRating;
#Field(type = FieldType.Integer)
private Double totalRatings;
#Field(type = FieldType.Keyword)
private String category; //this can be null
}
Here is what I have tried and is working and not working
I used ReactiveCrudRepository to delete documents in index. For all the regular fields on Book Level like id or with id and publishDate deletion works perfectly. As soon as I use embedded object like Stats, it stops working. I see the documents and the stats that I am sending match atleast visually but never finds or deletes them.
I tried to use EqualsAndHashcode in the Stats assuming maybe iternally somehow does not consider equal for some reason. I also tried changing double data type to int, because on looking at the elastic search document, I see that average review if whole number like 3 is save as 3 but when we send it from Java, i see in the debug 3 being shown as 3.0, so I was doubting if that is the case, but does not seem so. Even changing the datatype to int deletion does not work.
public interface ReactiveBookRepository extends ReactiveCrudRepository<EsBook, String> {
Mono<Void> deleteById(long id); //working
Mono<Void> deleteByIdAndPublishDate(long id, LocalDate publishDate); //Nor working
Mono<Void> deleteByIdAndStats(long id, LocalDate startDate);
}
Any help will be appreciated
Have you verified that your Elasticsearch index mapping matches your Spring Data annotations?
Verify that the index mapping defines the stats field as a nested field type.
If not, then try changing your Spring annotation to:
#Field(type = FieldType.Object)
private EsStats stats;
Considering the complexities involved in JPA we are planning to use Spring Data JDBC for our entities for its simplicity. Below is the sample structure and we have up to 6 child entities. We are able to successfully insert the data into various of these entities with proper foreign key mappings.
Challenge:- We have a workflow process outside of this application that periodically updates the "requestStatus" in the "Request" entity and this is the only field that gets updated after the Request is created. As with spring data JDBC, during the update it deletes all referenced entities and recreates(inserts) it again. This is kind of a heavy operation considering 6 child entities. Are there any workaround or suggestion in terms of how to handle these scenarios
#Table("Request")
public class Request {
private String requestId; // generated in the Before Save Listener .
private String requestStatus;
#Column("requestId")
private ChildEntity1 childEntity1;
public void addChildEntity1(ChildEntity1 childEntityobj) {
this.childEntity1 = childEntityobj;
}
}
#Table("Child_Entity1")
public class ChildEntity1 {
private String entity1Id; // Auto increment on DB
private String name;
private String SSN;
private String requestId;
#MappedCollection(column = "entity1Id", keyColumn = "entity2Id")
private ArrayList<ChildEntity2> childEntity2List = new ArrayList<ChildEntity2>();
#MappedCollection(column = "entity1Id", keyColumn = "entity3Id")
private ArrayList<ChildEntity3> childEntity3List = new ArrayList<ChildEntity3>();
public void addChildEntity2(ChildEntity2 childEntity2obj) {
childEntity2List.add(childEntity2obj);
}
public void addChildEntity3(ChildEntity3 childEntity3obj) {
childEntity3List.add(childEntity3obj);
}
}
#Table("Child_Entity2")
public class ChildEntity2 {
private String entity2Id; // Auto increment on DB
private String partyTypeCode;
private String requestId;
}
#Table(Child_Entity3)
public class ChildEntity3 {
private String entity3Id; // Auto increment on DB
private String PhoneCode;
private String requestId;
}
#Test
public void createandsaveRequest() {
Request newRequest = createRequest(); // using builder to build the object
newRequest.addChildEntity1(createChildEntity1());
newRequest.getChildEntity1().addChildEntity2(createChildEntity2());
newRequest.getChildEntity1().addChildEntity3(createChildEntity3());
requestRepository.save(newRequest);
}
The approach you describe in your comment:
Have a dedicated method performing exactly that update-statement is the right way to do this.
You should be aware though that this does ignore optimistic locking.
So there is a risk that the following might happen
Thread/Session 1: reads an aggregate.
Thread/Session 2: updates a single field as per your question.
Thread/Session 1: writes the aggregate, possibly with other changes, overwriting the change made by Session 2.
To avoid this or similar problems you need to
check that the version of the aggregate root is unchanged from when you loaded it, in order to guarantee that the method doesn't write conflicting changes.
increment the version in order to guarantee that nothing else overwrites the changes made in this method.
This might mean that you need two or more SQL statements which probably means you have to fallback even more to a full custom method where you implement this, probably using an injected JdbcTemplate.
I'm newbie in spring batch and I can't determinate what pattern for reader I would need to use. I need to create the class WSRequestClass and send it to SOAP web service.
public class WSRequestClass{
private String data1;
private String data2;
private String data3;
private String data4;
private List<ClassB> dataList;
}
To create WSRequestClass is necessary:
Read data1 and data2 from table A.
Read data3 and data4 from table B.
The List<ClassB> should be create from more complex flow. First I get data from query from table C, but the result of this query is a List<ClassA>. I need process each item of List<ClassA> and convert it to ClassB, where some attributes are calculated from ClassA. (Chunk pattern but without writer).
public class ClassA {
private Date date;
private BigDecimal amount1;
private BigDecimal amount2;
private String data;
//getters & setters
...
}
public class ClassB {
private Date date;
private BigDecimal amount1;
private BigDecimal amount2;
private BigDecimal amount3;
private BigDecimal amount4;
private String data1;
private String data2;
//getters & setters
...
}
I have found multiple examples for simples chunk pattern and tasklets, but none follows this structure. This job use java configuration and JdbcTemplate for queries. The development of the web service call it's done, my only issue is that I have to read from multiple tables and read efficiently the list, transform each item to ClassB and set to WsRequestClass.
Please guide me with the pattern to use, because common ItemReadernot work for me, and I don't know how implement the custom reader that allow me do what I want.
I think you're going about this wrong. There is a pattern in batch processing called the driving query pattern. In it, your reader reads essentially the keys for the objects. You then use processors to fill in the additional information. You can read more about this pattern in the Spring Batch documentation here: https://docs.spring.io/spring-batch/trunk/reference/html/patterns.html#drivingQueryBasedItemReaders