I'm using a spring ElasticsearchRepository to query Elasticsearch:
#Repository
public interface MyDocumentRepository extends ElasticsearchRepository<MyDocument, String>{
}
I can successfully run a search query and retrieve a list of elasticsearch results mapped in MyDocument bean
private final MyDocumentRepository myDocumentRepository;
...
Pageable pageable = PageRequest.of(0, 10);
QueryBuilder query = QueryBuilders.boolQuery().must(queryStringQuery("my query"));
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(query)
.withPageable(pageable)
.withHighlightFields(
new HighlightBuilder.Field("field1"),
new HighlightBuilder.Field("field2"))
.build();
Iterable<MyDocument> = myDocumentRepository.search(searchQuery);
Although the query sent to elasticsearch and response are correct (I can see the highlight block in the debug log) and results are also appropriate, I do not know how to get the highlight information in my java code.
I would like not to use ElasticsearchTemplate and a ResultExtractor to get the highlight information.
Thanks in advance!
Returning additional information in an SearchHit is a feature implemented in the current master branch which is available in the snaphot build. It will be released with the next release (4.0)
This is not the SearchHit class from Elasticsearch but a new introduced class in Spring Data Elasticsearch.
With the current version (3.2.x) this is only possible using a custom result mapper (which will not be available anymore in 4.0)
Edit:
As for the configuration in the upcoming version 4, check https://docs.spring.io/spring-data/elasticsearch/docs/current-SNAPSHOT/reference/html/#elasticsearch.clients.rest
With this configuration you then can inject an ElasticsearchOperations instance into your code, this bean is defined in the AbstractElasticsearchConfiguration class. The necessary code fragments:
#Autowired ElasticsearchOperations operations;
...
SearchHits<MyDocument> searchHits = operations.search(searchQuery,
MyDocument.class,
IndexCoordinates.of("index-name");
As for more information about the returned data check https://docs.spring.io/spring-data/elasticsearch/docs/current-SNAPSHOT/reference/html/#elasticsearch.operations.searchresulttypes
We know there are quite some deprecations and some breaking changes in version 4.0, but we we'll have a cleaner API and we will have the possibility to return all this search result metadata, this was not possible in 3.2
Acutally if you get the ES response you will have a SearchHit Object. This SearchHit Object just represents the HTTP Response of the ES result.
If your result hits a highlighted field, the SearchHit provides a Map of Map<String, HighlightField> where you can iterate and filter your highlighted text and attribute.
Related
I am using REST Data with Panache for JAX RESTful Web Service generation by extending PanacheEntityResource. In Spring land there is query builder mechanism that allows the Spring Data repository to generate an SQL query based on the name and return type of the custom method signature.
I'm trying to achieve the same thing using Panache, so far unsuccessfully.
#ResourceProperties(path = "tasks", paged = false)
public interface TaskResource extends PanacheEntityResource<Task, UUID> {
List<Task> findByOrganizationId(#QueryParam("organizationId") UUID organizationId);
}
I want to pass the organazation ID as a query parameter, such that my request will be http://localhost:8080/tasks?organizationId=1e7e669d-2935-4d6f-8b23-7a2497b0f5b0, and my response will return a list of Tasks whose organization ID matches the one provided. Is there support for this functionality?
That functionality is currently not supported.
See https://quarkus.io/guides/rest-data-panache and https://quarkus.io/guides/spring-data-rest for more details
Following is my WorkroomDTO:
#NotNull
private Instant createdOn;
#JsonInclude(JsonInclude.Include.NON_NULL)
private Instant changedOn;
As you can see i am using Java 8 Instant class.
In the elasticsearch Index server i store the following as JSON:
"createdOn": {
"nano": 877000000,
"epochSecond": 1579861613
},
"changedOn": {
"nano": 920000000,
"epochSecond": 1579861613
},
The problem is when i query the elasticsearch server to get me the workroom
return elasticsearchOperations.queryForPage(new NativeSearchQueryBuilder().withQuery(mainQuery)
.withPageable(elasticUtils.interceptPageable(searchDto.getPageable(), "name"))
.build(),
WorkroomDTO.class);
, i make a mapping of these fields to my WorkroomDTO i get the following exception:
com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `java.time.Instant` (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
at [Source: (String)"{"createdOn":{"nano":68000000,"epochSecond":1580127683}
FYI:
I have created a configuration file where is register explicitly the JavaTimeModule to the Object Mapper
#Configuration
public class JacksonConfiguration {
#Value("${application.serialization.include-type-key:true}")
private boolean includeTypeKey = true;
#Bean
public ObjectMapper objectMapper() {
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new JavaTimeModule());
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
mapper.addHandler(new MissingTypeIdHandler());
if (includeTypeKey) {
mapper.setMixInResolver(new TypeKeyMixInResolver());
}
return mapper;
}
}
Need help!
Where does this data come from? Is it written by your application?
It seems that the used Jackson mapper does not have the jackson-datatype-jsr310 module registered.
On reading the data tries to find a constructor of Instant that can be used to create an Instant object. But Instant does not have a default constructor and the Mapper should use the Instant.ofEpochSecond(long, long) method. This page pretty declares the problem and shows how the Jackson Mapper is configured.
Storing an instant in this way, as an object with two properties, is not the right way for storing dates in Elasticsearch. You should read the Elasticsearch documentation about how Elastcisearch handles date/time fields. When storing the instant as an object like this, you loose the ability to use Elasticsearch queries with criteria based on a date/time.
Which version of Spring Data Elasticsearch do you use? Because of problems like this, from the upcoming version 4.0 on, Spring Data Elasticsearch will not use the Jackson mapper anymore for entity mapping. The MappingElasticsearchConverter supports the use of the Elasticsearch date format and the java.time classes.
Well if I'm not completely wrong, your mapping fails due to the wrong format. The json you get looks like this:
"createdOn": {
"nano": 877000000,
"epochSecond": 1579861613
},
"changedOn": {
"nano": 920000000,
"epochSecond": 1579861613
},
That means you have 2 objects 'createdOn' and 'changedOn' with two properties (nano, epochSecond), while you try to map it to one object containing two properties named 'createdOn' and 'changedOn'. You need to modify that, you have e.g. a class called Entry, with two properties (nano, epochSeconds) and then a class with two properties (createdOn, changedOn) of type Entry
We have a service that simply returns the json document on a GET request. Since we do not have the POJO for the response "model", it appears we won't be able to use the auto response fields generation "goodness".
One option for us is to create the Pojos (quite large, about 50 attributes) and a corresponding controller that uses the pojos. This is awkward as we now have to maintain the model and corresponding controller just so we can auto generate the model.
Any ideas on how we can still leverage some auto generation of the response fields would be greatly appreciated.
Here's the controller I'm referring to:
#RestController
#RequestMapping("/api")
public class ProductController {
#Autowired
ProductService productService;
#RequestMapping(value = { "/products/{ids}" }, method = { RequestMethod.GET },
produces = "application/json", headers={"accept=application/json"})
#Timed
#ExceptionMetered
#LogExecutionTime
public String getProductDetails(#PathVariable("id") String id) {
return productService.getProductDetails(id);
}
At the moment I see no way of leveraging the auto generation without putting additional effort into it. Spring Auto REST Docs works by inspecting POJOs with a Jackson visitor (static introspection without runtime information) and there is currently no way of deriving the JSON fields from a string (would be dynamic at runtime). Thus, I only see two options:
The approach that you already described: Creating the corresponding POJO and using it.
Using Spring REST Docs for the corresponding test and manually document each field in the test. Might be the better option here if you do not want to alter the production code.
I have a Spring boot application using an AWS DynamoDb table which contains a list of items as such:
#DynamoDBTable(tableName = MemberDbo.TABLENAME)
public class MemberDbo {
public static final String TABLENAME = "Member";
#NonNull
#DynamoDBHashKey
#DynamoDBAutoGeneratedKey
protected String id;
// some more parameters
#DynamoDBAttribute
private List<String> membergroupIds;
}
I would like to find all members belonging to one specific groupId. In best case I would like to use CrudRepository like this:
#EnableScan
public interface MemberRepository extends CrudRepository<MemberDbo, String> {
List<MemberDbo> findByMembergroupIdsContaining(String membergroupIds); // actually I want to filter by ONE groupId
}
Unfortunately the query above is not working (java.lang.String cannot be cast to java.util.List)
Any suggestions how to build a correct query with CrudRepository?
Any suggestions how to create a query with Amazon SDK or some other Springboot-compliant methods?
Alternatively can I create a dynamoDb index somehow and filter by that index?
Or do I need to create and maintain a new table programmatically containing the mapping between membergroupIds and members (which results in a lot of overhead in code and costs)?
A solution for CrudRepository is preferred since I may use Paging in future versions and CrudRepository easily supports paging.
If I have understood correctly this looks very easy. You using DynamoDBMapper for model persistence.
You have a member object, which contains a list of membergroupids, and all you want to do is retrieve this from the database. If so, using DynamoDBMapper you would do something like this:
AmazonDynamoDB dynamoDBClient = new AmazonDynamoDBClient();
DynamoDBMapper mapper = new DynamoDBMapper(dynamoDBClient);
MemberDbo member = mapper.load(MemberDbo.class, hashKey, rangeKey);
member.getMembergroupIds();
Where you need to replace hashKey and rangeKey. You can omit rangeKey if you don't have one.
DynamoDBMapper also supports paging out of the box.
DynamoDBMapper is an excellent model persistence tool, it has strong features, its simple to use and because its written by AWS, it has seamless integration with DynamoDB. Its creators have also clearly been influenced by spring. In short, I would use DynamoDBMapper for model persistence and Spring Boot for model-controller stuff.
I have following code in my Spring RestController
Page<Message> messages = messagesRepository.findAll(new PageRequest(page, size, new Sort(Sort.Direction.DESC, "id")));
return messages.getContent().stream().filter(Message::isPublished).collect(Collectors.toList());
I would like to apply the filter on my collection first (only get the published messages) and then apply a PageRequest on it. How can I do that? Currently it is the other way around which does not return the expected result.
Just create a query method that will return only published messages. Something like this:
public interface MessageRepository extends JpaRepository<Message, Long> {
Page<Message> findByPublishedIsTrue(Pageable pageable);
}
Supported query keywords.