How to search for a part of a word with spring data elasticSearch - spring

In my Spring Data Elasticsearch application I'd like to implement autocomplete functionality when user types a few chars and application will show him all possible variants with query*.
Right now I can't find a way how to properly implement it with Spring Data Elasticsearch.
For example I tried the following:
Criteria c = new Criteria("name").startsWith(query);
return elasticsearchTemplate.queryForPage(new CriteriaQuery(c, pageRequest), ESDecision.class);
It works for a single word query but in case of two or more words it returns error:
"Cannot constructQuery '*"security windows"'. Use expression or multiple clauses instead."
How to properly implement it in this case?

I have same requirement , I have implemented same . Querystring will work for you .
If you have two token like "security windows" than you have to pass "*security* *windows*" than Querystring will return all possible data available . If you have one token like "security" than you have to pass "*security*" .
One more explaination for this scenario ,check this answer -
https://stackoverflow.com/a/43278852/2357869
String aQueryString = "security windows" ;
String aQueryWithPartialSerach = null;
List<ESDecision> aESDecisions = null;
// Enabling partial sarch
if (aQueryString.contains(" ")) {
List<String> aTokenList = Arrays.asList(aQueryString.split(" "));
aQueryWithPartialSerach = String.join(" ", aTokenList.stream().map(p -> "*" + p + "*").collect(Collectors.toList()));
} else {
aQueryWithPartialSerach = "*" + aQueryString + "*";
}
NativeSearchQueryBuilder aNativeSearchQueryBuilder = new NativeSearchQueryBuilder();
aNativeSearchQueryBuilder.withIndices(indexName).withTypes(type).withPageable(new PageRequest(0, iPageRequestCount));
final BoolQueryBuilder aQuery = new BoolQueryBuilder();
aQuery.must(QueryBuilders.queryStringQuery(aQueryWithPartialSerach).defaultField("name"));
NativeSearchQuery nativeSearchQuery = aNativeSearchQueryBuilder.withQuery(aQuery).build();
aESDecisions = elasticsearchTemplate.queryForList(nativeSearchQuery, ESDecision.class);
return aESDecisions;
Imports need to be done :-
import org.springframework.data.elasticsearch.core.query.NativeSearchQuery;
import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder;
import org.springframework.data.elasticsearch.core.query.SearchQuery;
import org.springframework.data.domain.PageRequest;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;

Related

Convert Elastic Search Results to POJO

I have a project using the spring-data-elasticsearch library. I've got my system returning results, but I was wondering how to get my results in the form of my domain POJO class.
I'm not seeing too much documentation on how to accomplish this, but I don't know what the right question I should be Googling for.
Currently, my code looks like this, and in my tests, it retrieves the right results, but not as a POJO.
QueryBuilder matchQuery = QueryBuilders.queryStringQuery(searchTerm).defaultOperator(QueryStringQueryBuilder.Operator.AND);
Client client = elasticsearchTemplate.getClient();
SearchRequestBuilder request = client
.prepareSearch("mediaitem")
.setSearchType(SearchType.QUERY_THEN_FETCH)
.setQuery(matchQuery)
.setFrom(0)
.setSize(100)
.addFields("title", "description", "department");
System.out.println("SEARCH QUERY: " + request.toString());
SearchResponse response = request.execute().actionGet();
SearchHits searchHits = response.getHits();
SearchHit[] hits = searchHits.getHits();
Any help is greatly appreciated.
One option is to use jackson-databind to map JSON from the search hits to POJOs.
For example:
ObjectMapper objectMapper = new ObjectMapper();
SearchHit[] hits = searchHits.getHits();
Arrays.stream(hits).forEach(hit -> {
String source = hit.getSourceAsString();
MediaItem mediaItem = objectMapper.readValue(source, MediaItem.class);
// Use media item...
});

Spring Data Solr - Multiple FilterQueries separated by OR

I'm trying to implement a filter search using spring data solr. I've following filters types and all have a set of filters.
A
aa in (1,2,3)
ab between (2016-08-02 TO 2016-08-10)
B
ba in (2,3,4)
bb between (550 TO 1000)
The Solr query which I want to achieve using Spring data solr is:
q=*:*&fq=(type:A AND aa:(1,2,3) AND ab:[2016-08-02 TO 2016-08-10]) OR (type:B AND ba:(2,3,4) AND bb:[550 TO 1000])
I'm not sure how to group a number of clauses of a type of filter and then have an OR operator.
Thanks in advance.
The trick is to flag the second Criteria via setPartIsOr(true) with an OR-ish nature. This method returns void, so it cannot be chained.
First aCriteria and bCriteria are defined as described. Then bCriteria is flagged as OR-ish. Then both are added to a SimpleFilterQuery. That in turn can be added to the actual Query. That is left that out in the sample.
The DefaultQueryParser in the end is used only to generate a String that can be used in the assertion to check that the query is generated as desired.
import org.junit.jupiter.api.Test;
import org.springframework.data.solr.core.DefaultQueryParser;
import org.springframework.data.solr.core.query.Criteria;
import org.springframework.data.solr.core.query.FilterQuery;
import org.springframework.data.solr.core.query.SimpleFilterQuery;
import static org.junit.jupiter.api.Assertions.assertEquals;
public class CriteriaTest {
#Test
public void generateQuery() {
Criteria aCriteria =
new Criteria("type").is("A")
.connect().and("aa").in(1,2,3)
.connect().and("ab").between("2016-08-02", "2016-08-10");
Criteria bCriteria =
new Criteria("type").is("B")
.connect().and("ba").in(2,3,4)
.connect().and("bb").between("550", "1000");
bCriteria.setPartIsOr(true); // that is the magic
FilterQuery filterQuery = new SimpleFilterQuery();
filterQuery.addCriteria(aCriteria);
filterQuery.addCriteria(bCriteria);
// verify the generated query string
DefaultQueryParser dqp = new DefaultQueryParser(null);
String actualQuery = dqp.getQueryString(filterQuery, null);
String expectedQuery =
"(type:A AND aa:(1 2 3) AND ab:[2016\\-08\\-02 TO 2016\\-08\\-10]) OR "
+ "((type:B AND ba:(2 3 4) AND bb:[550 TO 1000]))";
System.out.println(actualQuery);
assertEquals(expectedQuery, actualQuery);
}
}

How to merge DataServiceQuery expression with AddQueryOption

OData linq doesn't support contains or any kind of "in" check. To solve this problem guys proposed following solution:
OData "where ID in list" query
The problem is that I want to mix this solution with regular Where clause. If I just add AddQueryOption I will clean all previous filters added by Where. I have tried to extract initial $filter generated by Where and append to it "in" clause and then call AddQueryOption - without any success. As far as I see most of the classes that translate expression are internal. So I could not find any way to inject custom code. Any ideas how to do this?
Generally I want something like this:
provider.Collection.Where(data => data.Name ="aaa" and categories.Contains(data.CategoryId))
Code that unfortunatelly not solve this:
private DataServiceQuery<TElement> AddContains(DataServiceQuery<TElement> query, string [] categories)
{
var filterParams = categories.Select(id => string.Format("(CategoryId eq '{0}')", id));
var filter = string.Join(" or ", filterParams);
//filter = originalFilter + " and " + string.Join(" or ", filterParams);
// also doesn't work
return query.AddQueryOption("$filter", filter);
}
var query = provider.Collection.Where(data => data.Name ="aaa") as DataServiceQuery<TElement>;
var results = AddContains(query, new string[] {"1","2"}).Execute().ToList();

How to update multiple fields using java api elasticsearch script

I am trying to update multiple value in index using Java Api through Elastic Search Script. But not able to update fields.
Sample code :-
1:
UpdateResponse response = request.setScript("ctx._source").setScriptParams(scriptParams).execute().actionGet();
2:
UpdateResponse response = request.setScript("ctx._source.").setScriptParams(scriptParams).execute().actionGet();
if I mentioned .(dot) in ("ctx._source.") getting illegalArgument Exception and if i do not use dot, not getting any exception but values not getting updated in Index.
Can any one tell me the solutions to resolve this.
First of all, your script (ctx._source) doesn't do anything, as one of the commenters already pointed out. If you want to update, say, field "a", then you would need a script like:
ctx._source.a = "foobar"
This would assign the string "foobar" to field "a". You can do more than simple assignment, though. Check out the docs for more details and examples:
http://www.elasticsearch.org/guide/reference/api/update/
Updating multiple fields with one script is also possible. You can use semicolons to separate different MVEL instructions. E.g.:
ctx._source.a = "foo"; ctx._source.b = "bar"
In Elastic search have an Update Java API. Look at the following code
client.prepareUpdate("index","typw","1153")
.addScriptParam("assignee", assign)
.addScriptParam("newobject", responsearray)
.setScript("ctx._source.assignee=assignee;ctx._source.responsearray=newobject ").execute().actionGet();
Here, assign variable contains object value and response array variable contains list of data.
You can do the same using spring java client using the following code. I am also listing the dependencies used in the code.
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.index.query.QueryBuilder;
import org.springframework.data.elasticsearch.core.query.UpdateQuery;
import org.springframework.data.elasticsearch.core.query.UpdateQueryBuilder;
private UpdateQuery updateExistingDocument(String Id) {
// Add updatedDateTime, CreatedDateTime, CreateBy, UpdatedBy field in existing documents in Elastic Search Engine
UpdateRequest updateRequest = new UpdateRequest().doc("UpdatedDateTime", new Date(), "CreatedDateTime", new Date(), "CreatedBy", "admin", "UpdatedBy", "admin");
// Create updateQuery
UpdateQuery updateQuery = new UpdateQueryBuilder().withId(Id).withClass(ElasticSearchDocument.class).build();
updateQuery.setUpdateRequest(updateRequest);
// Execute update
elasticsearchTemplate.update(updateQuery);
}
XContentType contentType =
org.elasticsearch.client.Requests.INDEX_CONTENT_TYPE;
public XContentBuilder getBuilder(User assign){
try {
XContentBuilder builder = XContentFactory.contentBuilder(contentType);
builder.startObject();
Map<String,?> assignMap=objectMap.convertValue(assign, Map.class);
builder.field("assignee",assignMap);
return builder;
} catch (IOException e) {
log.error("custom field index",e);
}
IndexRequest indexRequest = new IndexRequest();
indexRequest.source(getBuilder(assign));
UpdateQuery updateQuery = new UpdateQueryBuilder()
.withType(<IndexType>)
.withIndexName(<IndexName>)
.withId(String.valueOf(id))
.withClass(<IndexClass>)
.withIndexRequest(indexRequest)
.build();

Custom search in Dynamics CRM 4.0

I have a two related questions.
First:
I'm looking to do a full text search against a custom entity in Dynamics CRM 4.0. Has anyone done this before or know how to do it?
I know that I can build QueryExpressions with the web service and sdk but can I do a full text search with boolean type syntax using this method? As far as I can tell that won't do the trick.
Second:
Does anyone else feel limited with the searching abilities provided with Dynamics CRM 4.0? I know there are some 3rd pary search products out there but I haven't found one I like yet. Any suggestions would be appreciated.
Searching and filtering via the CRM SDK does take some time to get used to. In order to simulate full text search, you need to use nested FilterExpressions as your QueryExpression.Criteria. SDK page for nested filters The hardest part is figuring out how to build the parent child relationships. There's so much boolean logic going on that it's easy to get lost.
I had a requirement to build a "search engine" for one of our custom entities. Using this method for a complex search string ("one AND two OR three") with multiple searchable attributes was ugly. If you're interested though, I can dig it up. While it's not really supported, if you can access the database directly, I would suggest using SQL's full text search capabilities.
--
ok, here you go. I don't think you'll be able to copy paste this and fulfill your needs. my customer was only doing two to three key word searches and they were happy with the results from this. You can see what a pain it is to just do this in a simple search scenario. I basically puked out code until it was 'working'.
private FilterExpression BuildFilterV2(string[] words, string[] seachAttributes)
{
FilterExpression filter = new FilterExpression();
List<FilterExpression> allchildfilters = new List<FilterExpression>();
List<string> andbucket = new List<string>();
List<string> orBucket = new List<string>();
// clean up commas, quotes, etc
words = ScrubWords(words);
int index = 0;
while (index < words.Length)
{
// if current word is 'and' then add the next wrod to the ad bucket
if (words[index].ToLower() == "and")
{
andbucket.Add(words[index + 1]);
index += 2;
}
else
{
if (andbucket.Count > 0)
{
List<FilterExpression> filters = new List<FilterExpression>();
foreach (string s in andbucket)
{
filters.Add(BuildSingleWordFilter(s, seachAttributes));
}
// send existing and bucket to condition builder
FilterExpression childFilter = new FilterExpression();
childFilter.FilterOperator = LogicalOperator.And;
childFilter.Filters = filters.ToArray();
// add to child filter list
allchildfilters.Add(childFilter);
//new 'and' bucket
andbucket = new List<string>();
}
if (index + 1 < words.Length && words[index + 1].ToLower() == "and")
{
andbucket.Add(words[index]);
if (index + 2 <= words.Length)
{
andbucket.Add(words[index + 2]);
}
index += 3;
}
else
{
orBucket.Add(words[index]);
index++;
}
}
}
if (andbucket.Count > 0)
{
List<FilterExpression> filters = new List<FilterExpression>();
foreach (string s in andbucket)
{
filters.Add(BuildSingleWordFilter(s, seachAttributes));
}
// send existing and bucket to condition builder
FilterExpression childFilter = new FilterExpression();
childFilter.FilterOperator = LogicalOperator.And;
childFilter.Filters = filters.ToArray();
// add to child filter list
allchildfilters.Add(childFilter);
//new 'and' bucket
andbucket = new List<string>();
}
if (orBucket.Count > 0)
{
filter.Conditions = BuildConditions(orBucket.ToArray(), seachAttributes);
}
filter.FilterOperator = LogicalOperator.Or;
filter.Filters = allchildfilters.ToArray();
return filter;
}
private FilterExpression BuildSingleWordFilter(string word, string[] seachAttributes)
{
List<ConditionExpression> conditions = new List<ConditionExpression>();
foreach (string attr in seachAttributes)
{
ConditionExpression expr = new ConditionExpression();
expr.AttributeName = attr;
expr.Operator = ConditionOperator.Like;
expr.Values = new string[] { "%" + word + "%" };
conditions.Add(expr);
}
FilterExpression filter = new FilterExpression();
filter.FilterOperator = LogicalOperator.Or;
filter.Conditions = conditions.ToArray();
return filter;
}
private ConditionExpression[] BuildConditions(string[] words, string[] seachAttributes)
{
List<ConditionExpression> conditions = new List<ConditionExpression>();
foreach (string s in words)
{
foreach (string attr in seachAttributes)
{
ConditionExpression expr = new ConditionExpression();
expr.AttributeName = attr;
expr.Operator = ConditionOperator.Like;
expr.Values = new string[] { "%" + s + "%" };
conditions.Add(expr);
}
}
return conditions.ToArray();
}
Hm, that's a pretty interesting scenario...
You could certainly do a 'Like' query, and 'or' together the colums/attribute conditions you want included in the search. This seems to be how CRM does queries from the box above entity lists (and they're plenty fast). It looks like the CRM database has a full-text index, although exactly which columns are used to populate it is a bit foggy to me after a brief peek.
And remember LinqtoCRM for CRM query love (I started the project, sorry about the shameless plug).
Second - I can recommend "Global Search" by Akvelon which provides ability to search in all Custom Entities and attributes and Out of Box entities and attributes. Also they are using FTS for search in the attached documents contents. You can find more details in their official site: http://www.akvelon.com/Products/Dynamics%20CRM%20global%20Search/default.aspx
I would suggest utilizing the Dynamics CRM filtered views provided for you in the database. Then you can utilize all the power of native SQL to do any LIKE's or other logic you need. Plus, the filtered views are security trimmed, so you won't have to worry about users accessing records they do not have permission to.

Resources