I have some custom data(let's call Camera) in my ElasticSearch, the data showed in Kibana is like
And I tried to delete data by Query according to the accepted answer in this article ElasticSearch Delete by Query, my code is like
String query = "{\"Name\":\"test Added into Es\"}";
DeleteByQuery delete = new DeleteByQuery.Builder(query).addIndex(this._IndexName).addType(this._TypeName).build();
JestResult deleteResult = this._JestClient.execute(delete);
And the result is 404 Not Found.
Its obvious that there exist one Camera data in ElasticSearch which Name match the query, so I believe the 404 is caused by other reason.
Did I do anything wrong? Should I change the query string?
The query needs to be a real query, not a partial document
Try with this instead
String query = "{\"query\": { \"match\": {\"Name\":\"test Added into Es\"}}}";
Related
I'm running an updateByQuery operation in ElasticSearch using Spring Data ElasticSearch (Spring Boot parent v2.6.1, Elastic Search 7.15.2).
In my ES index I have stored 2 documents.
When I give a non-existent document in the search, it doesn't give me any error, because of which I'm not able to distinguish whether the update actually ran or not.
Updates for a document that exists work fine. I'd like to figure a way such that if no rows are edited, I can log it.
What should I look at? What should I change so that I can get some message to understand if there was an update?
Here's my code snippet:
UpdateByQueryRequest request = new UpdateByQueryRequest('index');
Map<String, Object> data = new HashMap<>();
data.put("marks", "30");
data.put("name", "timmy");
data.put("roll_number", "10");
request.setScript(
new Script(
ScriptType.INLINE, "painless",
"if (ctx._source.name == params.name && ctx._source.roll_number == params.roll_number) {ctx._source.marks=params.marks;}",
data));
BulkByScrollResponse resp = globalClient.updateByQuery(request, RequestOptions.DEFAULT);
log.info("response: {}",resp.getStatus());
I've added the response status as well. What I find weird is that in case of both existent and non-existent document, the updated parameter count is 2, same as the number of documents I have in my index.
response in case of non-existent record:
response: BulkIndexByScrollResponse[sliceId=null,updated=2,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s]
response in case of existing record:
response: BulkIndexByScrollResponse[sliceId=null,updated=2,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s]
This is pure Elasticsearch code, nothing from Spring Data Elasticsearch .
Where do you specify the query? I don't see any in your code. That means that all documents will be updated - 2 in your case.
want to delete time based indices on elastic search older than some specific (let's say 30) days using date math expression in JAVA.
Trying to implement following approach in our spring boot application using Transaport Client but getting index not exist exception.
https://www.elastic.co/guide/en/elasticsearch/reference/current/date-math-index-names.html
When the same URI encoded index name (which is passed to DELETE INDEX API) is used to search index using query GET uri_encoded_index_name on kibana, It shows that index exists.
Is there something I am missing out ?
Is there any better approach to do that without using curator and wilcard characters?
code snippet:
String indexName = "<" + indexNameStaticPart + "{now/d-30d{MMddYYYY}}>";
String encodedIndexName = UriEncoder.encode( indexName ).replace( "/", "%2F" );
AcknowledgedResponse response = client.admin().indices().delete( new DeleteIndexRequest( encodedIndexName ) ).actionGet();
encodedIndexName : %3Cstring__string_string__%7Bnow%2Fd-30d%7BMMddyyyy%7D%7D%3E
kibana:
GET encodedIndexName
DELETE encodedIndexName
Try it without URI encoding..
https://discuss.elastic.co/t/delete-time-based-indices-on-elastic-search-older-than-30-days-using-date-math-expression-in-java-gives-indexnotexists-exception/171365
I am new in writing aggregate queries in Mongo DB + Spring
Scenario: We are storing birthDate(Jjava.uti.Date) in mongo db which got stored as ISO date. Now we are trying to look for the records which are matching with the dayOfMonth and Month only. So that we can corresponding object from the list.
I had gone through few solutions and here is the way I am trying but this is giving me a null set of records.
Aggregation agg = Aggregation.newAggregation(
Aggregation.project().andExpression("dayOfMonth(birthDate)").as("day").andExpression("month(birthDate)")
.as("month"),
Aggregation.group("day", "month"));
AggregationResults<Employee> groupResults = mongoTemplate.aggregate(agg, Employee.class, Employee.class);
I also tried applying a a query with the help of Criteria but this is also giving me a Employee object which all null content.
Aggregation agg = Aggregation.newAggregation(Aggregation.match(Criteria.where("birthDate").lte(new Date())), Aggregation.project().andExpression("dayOfMonth(birthDate)").as("day").andExpression("month(birthDate)")
.as("month"),
Aggregation.group("day", "month"));
AggregationResults<Employee> groupResults = mongoTemplate.aggregate(agg, Employee.class, Employee.class);
I must missing some important thing which is giving me these null data.
Additional Info: Employee object has only birthDate(Date) and email(String) in it
Please try to specify the fields to be included in the $project stage.
project("birthDate", "...").andExpression("...
The _id field is, by default, included in the output documents. To include any other fields from the input documents in the output documents, you must explicitly specify the inclusion in $project.
see: MongoDBReference - $project (aggregation)
I've created DATAMONGO-2200 to add an option to project directly onto the fields of a given domain type via something like project(Employee.class).
I am trying to use elastic search for full text search and Spring data for integrating elastic search with my application.
For example,
There are 6 fields to be indexed.
1)firstName
2)lastName
3)title
4)location
5)industry
6)email
http://localhost:9200/test/_mapping/
I can see these fields in the mapping.
Now, I would like to make a search against these fields with a search input.
For example, When I search "mike 123", it has to search against all these 6 fields.
In Spring data repository,
The below method works to search only in firstName.
Collection<Object> findByFirstNameLike(String searchInput)
But, I would like to search against all the fields.
I tried,
Collection<Object> findByFirstNameLikeOrLastNameLikeOrTitleLikeOrLocationLikeOrIndustryLikeOrEmailLike(String searchInput,String searchInput1,String searchInput2,String searchInput3,)
Here, even the input string is same, i need to pass the same input as 6 params. Also the method name looks bigger with multiple fields.
Is there anyway to make it simple with #Query or ....
Like,
Collection<Object> findByInput(String inputString)
Also, boosting should be given for one of the field.
For example,
When i search for "mike mat", if there is any match in the firstName, that should be the first one in the result even there are exact match in the other fields.
Thanks
Lets suppose your search term is in the variable query, you can use the method search in ElasticsearchRepository.
repo.search(queryStringQuery(query))
to use queryStringQuery use the following import
import static org.elasticsearch.index.query.QueryBuilders.queryStringQuery;
I found the way to achieve this and posting here. Hope, this would help.
QueryBuilder queryBuilder = boolQuery().should(
queryString("Mike Mat").analyzeWildcard(true)
.field("firstName", 2.0f).field("lastName").field("title")
.field("location").field("industry").field("email"));
Thanks
Not a spring-data elasticsearch expert. But I see two directions you can go. The first would be to use the #Query option. That way you can create your own query. The second would be to use the example in the Filter builder section:
http://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#elasticsearch.misc.filter
Within elasticearch you would want to use the multi_match query:
http://www.elastic.co/guide/en/elasticsearch/reference/1.5/query-dsl-multi-match-query.html
In java such a query would look like this:
QueryBuilder qb = multiMatchQuery(
"kimchy elasticsearch",
"user", "message"
);
Example coming from: http://www.elastic.co/guide/en/elasticsearch/client/java-api/current/query-dsl-queries.html#multimatch
We can write our own custom query as below.
we can specific index, routing value (this is used if alias is used)
SearchQuery searchQuery = new NativeSearchQueryBuilder().withIndices(INDEX)
.withRoute(yourQueryBuilderHelper.getRouteValue())
.withQuery(yourQueryBuilderHelper.buildQuery(yourSearchFilterRequestObject))
.withFilter(yourQueryBuilderHelper.buildFilter(yourSearchFilterRequestObject)).withTypes(TYPE)
.withSort(yourQueryBuilderHelper.buildSortCriteria(yourSearchFilterRequestObject))
.withPageable(yourQueryBuilderHelper.buildPaginationCriteria(yourSearchFilterRequestObject)).build();
FacetedPage<Ticket> searchResults = elasticsearchTemplate.queryForPage(searchQuery, YourDocumentEntity.class);
Its good to use your own queryBuilder helper which can seperate your elasticSearchService from queryBuilder responsibility.
Hope this helps
Thanks
QueryBuilder class is helpful to query from spring Dao to elastic search:
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.QueryBuilder;
QueryBuilder qb = QueryBuilders.boolQuery()
.must(QueryBuilders.termQuery("state", "KA"));
.must(QueryBuilders.termQuery("content", "test4"))
.mustNot(QueryBuilders.termQuery("content", "test2"))
.should(termQuery("content", "test3"));
.should(termQuery("content", "test3"));
Try like this, you can even set importance of the field
QueryBuilder queryBuilder = QueryBuilders.multiMatchQuery(query)
.field("name", 2.0f)
.field("email")
.field("title")
.field("jobDescription", 3.0f)
.type(MultiMatchQueryBuilder.Type.PHRASE_PREFIX);
Another way is using Query String query
Query searchQuery = new StringQuery(
"{\"query\":{\"query_string\":{\"query\":\""+ your-query-here + "\"}}}\"");
SearchHits<Product> products = elasticsearchOperations.search(
searchQuery,
Product.class,
IndexCoordinates.of(PRODUCT_INDEX_NAME));
This will search all the field from your document of specified index
I am using elastic search multi get for reading documents after bulk update. Its returning some document sources as null.
MultiGetRequestBuilder builder = client.prepareMultiGet();
builder.setRefresh(true);
builder.add(indexName, type, idsList);
MultiGetResponse multiResponse = builder.execute().actionGet();
for (MultiGetItemResponse response : multiResponse.getResponses())
{
String customerJson = response.getResponse().getSourceAsString();
System.out.println("customerJson::" + customerJson);
}
Any issues in my code? Thanks in advance.
When you say "some return sources as null", I assume the get response is marking them as not existing..?
If that's the case, then maybe :
some indexation request in the bulk are failing dur to mapping/random error.
You need to refresh your index between the indexation and the multiget (i.e : your docs are not available for search yet)
transportClient.admin().indices().prepareRefresh(index).execute();
good luck
EDIT : You answered your own question in the comment, but for readability's sake : when using get or multiget, if a routing key was used when indexing, it must be specified again during the get, else, a wrong shard is determined using default routing and the get fails.