Actually I'm a newcomer to ElasticSearch and got stuck with just a simple NEST query.
Here is my class to store data in ElasticSearch:
public class MyClass
{
public Guid Id { get; set; }
public string Name { get; set; }
public string Language { get; set; }
}
I need to get documents by the Language (e.g. Language == "eng")
I use the NEST 6.x
Here is the SerchDescriptor
searchDescriptor = new SearchDescriptor<MyClass>()
.Index(indexName)
.Query(q => q.Term("Language", "eng"));
it produces the request:
{
"query": {
"term": {
"Language": {
"value": "eng"
}
}
}
}
but the right request is
{
"query": {
"term": {
"Language": "eng"
}
}
}
How can I get the right request via NEST?
Both forms are valid; the former is the long form of term query that accepts other properties such as boost and _name.
NEST typically serializes request types to the long form, and deserializes from the long form.
Related
I work with Liferay 7.2 and I need to make an Elasticsearch query that finds als DLFileEntries that have the Document Type "XY". Currently I need to do this in Postman.
I am already able to find all DLFileEntry:
{
"query": {
"bool": {
"must": [
{
"match": {
"entryClassName": "com.liferay.document.library.kernel.model.DLFileEntry"
}
}
]
}
}
}
But I need to find only these DLFileEntry that have Document Type "XY".
How can I do this?
You can simply add another match to the field fileEntryTypeId, where its value must be equal to the created Document Type id. You can find this id on table dlfileentrytype on column fileentrytypeid. Considering the id equals 37105, the query would be like this
{
"query": {
"bool": {
"must": [
{
"match": {
"entryClassName": "com.liferay.document.library.kernel.model.DLFileEntry"
}
},
{
"match": {
"fileEntryTypeId": "37105"
}
}
]
}
}
}
edit: Responding to your comment about how to search the DLFileEntry by its DLFileEntryType name, there is no direct way to do this as the DLFileEntryType is not indexed on Elastic Search by default. It would also probably need sub queries to achieve this and Elastic Search doesn't support sub queries.
With that in mind, the easiest approach I can think of is to customize the way DLFileEntry is indexed on Elastic Search, adding the field fileEntryTypeName. For that, you only need to implement a ModelDocumentContributor for DLFileEntry and add the fileEntryTypeName field to the document.
Basically, you just need to create a class like this:
package com.test.liferay.override;
import com.liferay.document.library.kernel.model.DLFileEntry;
import com.liferay.portal.kernel.exception.PortalException;
import com.liferay.portal.kernel.search.Document;
import com.liferay.portal.search.spi.model.index.contributor.ModelDocumentContributor;
import org.osgi.service.component.annotations.Component;
#Component(
immediate = true,
property = "indexer.class.name=com.liferay.document.library.kernel.model.DLFileEntry",
service = ModelDocumentContributor.class
)
public class DLFileEntryModelDocumentContributor
implements ModelDocumentContributor<DLFileEntry> {
#Override
public void contribute(Document document, DLFileEntry dlFileEntry) {
try {
document.addText(
"fileEntryTypeName", dlFileEntry.getDLFileEntryType().getName());
} catch (PortalException e) {
// handle error
}
}
}
As the DLFileEntryType name is localized, you should probably index it as a localized value:
package com.test.liferay.override;
import com.liferay.document.library.kernel.model.DLFileEntry;
import com.liferay.portal.kernel.exception.PortalException;
import com.liferay.portal.kernel.search.Document;
import com.liferay.portal.kernel.search.Field;
import com.liferay.portal.kernel.util.LocaleUtil;
import com.liferay.portal.search.spi.model.index.contributor.ModelDocumentContributor;
import org.osgi.service.component.annotations.Component;
import java.util.Locale;
#Component(
immediate = true,
property = "indexer.class.name=com.liferay.document.library.kernel.model.DLFileEntry",
service = ModelDocumentContributor.class
)
public class DLFileEntryModelDocumentContributor
implements ModelDocumentContributor<DLFileEntry> {
#Override
public void contribute(Document document, DLFileEntry dlFileEntry) {
try {
Locale siteDefaultLocale = LocaleUtil.getSiteDefault();
String localizedName = dlFileEntry
.getDLFileEntryType().getName(siteDefaultLocale);
String localizedField = Field.getLocalizedName(
siteDefaultLocale, "fileEntryTypeName");
document.addText(localizedField, localizedName);
} catch (PortalException e) {
// handle error
}
}
}
Now your query will be something like this:
{
"query": {
"bool": {
"must": [
{
"match": {
"entryClassName": "com.liferay.document.library.kernel.model.DLFileEntry"
}
},
{
"match": {
"fileEntryTypeName_en_US": "XY"
}
}
]
}
}
}
The name fileEntryTypeName_en_US depends on your site default locale. For example, if it is pt_BR, the name would be fileEntryTypeName_pt_BR.
Obs.: The fileEntryType name field is not unique, as it is localized, so you might find files with the same fileEntryType name but different fileEntryType.
I have a document structured as follows, more or less:
post {
_id
title
isPublished
}
user {
_id
username
name
[posts]
}
I know I can query fields like postConnection and userConnection with the aggregate subfield in order to query a count of all objects. But how do I get the total count of all posts by a given user?
I was able to come up with this:
{
postsConnection(where: {isPublished: true}){
groupBy{
author{
key
connection{
aggregate{
count
}
}
}
}
}
}
But this returns (expectedly) something like this:
{
"data": {
"postsConnection": {
"groupBy": {
"author": [
{
"key": "5c9136976238de2cc029b5d3",
"connection": {
"aggregate": {
"count": 5
}
}
},
{
"key": "5c99d5d5fcf70010b75c07d5",
"connection": {
"aggregate": {
"count": 3
}
}
}
]
}
}
}
}
As you can see, it returns post counts for all authors in an array. What I need is to be able to return the count for only one specific user and not by _id (which is what the key field seems to map to) but by another unique field I have in the users collection, i.e. username.
Is that possible?
Need to pass in a parameter to either the query or the field to return specific data
I am trying to load data from an Sql Server into ElasticSearch. I am using Logstash with the jdbc plugin and the elastic-search plugin. I am loading my data in ElasticSearch but can not figure out how to set my index. I am using a template index to try this. Below is what I am using but whenever I search I do not get any results.
logstash.config
# contents of logstash\bin\logstash.config
input {
jdbc {
jdbc_driver_library => ".\Microsoft JDBC Driver 6.2 for SQL Server\sqljdbc_6.2\enu\mssql-jdbc-6.2.1.jre8.jar"
jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
jdbc_connection_string => "jdbc:sqlserver://mydbserver;databaseName=mydb;"
jdbc_user => "******"
jdbc_password => "******"
schedule => "* * * * *"
parameters => { "classification" => "EMPLOYEE" }
statement => "SELECT Cost_Center, CC_Acct_1, CC_Acct_2, CC_Acct_3 from dbo.Cost_Center where CC_Classification = :classification"
}
}
output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "allocation-testweb"
template => "index_template.json"
}
#stdout { codec => rubydebug }
}
index_template.json
{
"template": "allocation-*",
"order":1,
"settings": {
"number_of_replicas": 0,
"number_of_shards": 1,
"analysis": {
"analyzer": {
"substring_analyzer": {
"tokenizer": "ngram_tokenizer",
"filter": ["lowercase"]
}
},
"tokenizer": {
"ngram_tokenizer": {
"type": "edge_ngram",
"min_gram": 2,
"max_gram": 10,
"token_chars": ["letter","digit"]
}
}
}
},
"mappings":{
"costcenter": {
"properties": {
"cc_acct_1": {
"type": "string",
"analyzer": "substring_analyzer"
},
"cc_acct_2": {
"type": "string",
"analyzer": "substring_analyzer"
}
}
}
}
I have created a similar index in code while doing some initial research. Is my index_template incorrect or is there another way I should be doing this?
Update:
I had the mismatched index names between my 2 files. I'm now able to search using Postman and curl. However when I try to get data using a NEST client I can never get data back. Below is the code snippet for the query.
var searchResult = client.Search<CostCenter>(s => s
.Size(1000)
.Index("allocation_testweb")
.MatchAll());
This previously worked with the same data loaded from a file. CostCenter is simply an object with members called Cost_Center, CC_Acct_1, CC_Acct_2, and CC_Acct_3. I'm sure again I am over complicating the issue and missing something obvious.
UPDATE II:
I have made the changes suggested by #RussCam below and still do not get any results back. Below is my updated code.
var node = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(node);
//.InferMappingFor<CostCenter>(m => m.IndexName("allocation_testweb"));
var client = new ElasticClient(settings);
var searchResult = client.Search<CostCenter>(s => s
.Type("costCenter")
.Size(1000)
.Index("allocation_testweb")
.MatchAll());
I commented out the InferMappingFor<> since it was not providing a result.
Mapping image requested by #RussCam. I've also included my costcenter class (I have tried naming all variations of costcenter).
public class costcenter
{
public string cost_center { get; set; }
public string cc_acct_1 { get; set; }
public string cc_acct_2 { get; set; }
public string cc_acct_3 { get; set; }
}
I'm trying to issue a query which includes sorting
from Hibernate Search 5.7.1.Final
to ElasticSearch 2.4.2.
When I'm using curl I get the results:
curl -XPOST 'localhost:9200/com.example.app.model.review/_search?pretty' -d '
{
"query": { "match" : { "authors.name" : "Puczel" } },
"sort": { "title": { "order": "asc" } }
}'
But when I issue the query from code:
protected static Session session;
public static void prepareSession()
{
SessionFactory sessionFactory = new Configuration().configure()
.buildSessionFactory();
session = sessionFactory.openSession();
}
...
protected static void testJSONQueryWithSort()
{
FullTextSession fullTextSession = Search.getFullTextSession(session);
QueryDescriptor query = ElasticsearchQueries.fromJson(
"{ 'query': { 'match' : { 'authors.name' : 'Puczel' } }, 'sort': { 'title': { 'order': 'asc' } } }");
List<?> result = fullTextSession.createFullTextQuery(query, Review.class).list();
System.out.println("\n\nSearch results for 'author.name:Puczel':");
for(Object object : result)
{
Review review = (Review) object;
System.out.println(review.toString());
}
}
I get an Exception:
"[filtered] query does not support [sort]"
I understand where it comes from, because the query
that Hibernate Search issues is different than my curl query
- specifying the type is realised differently:
{
"query":
{
"filtered":
{
"query":
{
"match":{"authors.name":"Puczel"}
},
"sort":{"title":{"order":"asc"}},
"filter":{"type":{"value":"com.example.app.model.Review"}}
}
}
}
But I don't know how to change it.
I tried using the sort example from Hibernate documentation:
https://docs.jboss.org/hibernate/search/5.7/reference/en-US/html_single/#__a_id_elasticsearch_query_sorting_a_sorting
But the example is not full. I don't know:
which imports to use (there are multiple matching),
what are the types of the undeclared variables, like s,
how to initalise the variable luceneQuery.
I will appreciate any remarks on this.
Yes, as mentioned in the javadoc of org.hibernate.search.elasticsearch.ElasticsearchQueries.fromJson(String):
Note that only the 'query' attribute is supported.
So you must use the Hibernate Search API to perform sorts.
which imports to use (there are multiple matching),
Sort is the one from Lucene (org.apache.lucene), List is from java.util, and all the other imports should be from Hibernate Search (org.hibernate.search).
what are the types of the undeclared variables, like s
s is a FullTextSession retrieved through org.hibernate.search.Search.getFullTextSession(Session). It will also work with a FullTextEntityManager retrieved through org.hibernate.search.jpa.Search.getFullTextEntityManager(EntityManager).
how to initalise the variable luceneQuery
You'll have to use the query builder (qb):
Query luceneQuery = qb.keyword().onField("authors.name").matching("Puczel").createQuery();
If you intend to use the Hibernate Search API, and you're not comfortable with it yet, I'd recommend reading the general documentation first (not just the Elasticsearch part, which only mentions Elasticsearch specifics): https://docs.jboss.org/hibernate/search/5.7/reference/en-US/html_single/#search-query
ES newbie here, sorry for the dumb question.
I have been trying to create a elasticsearch query for a products index. I'm able to query it, but it never returns as I expect.
I'm probably using the query builder in a wrong way, have tried all sorts of queries builders and never got to make it work as I expected.
My Product class (simpler for the sake of the question):
public class Product {
private String sku;
private Boolean soldOut;
private Boolean freeShipping;
private Store store;
private Category category;
private Set<ProductUrl> urls;
private Set<ProductName> names;
}
Category nas name and id which I use for aggregations
The boolean field are used for filters.
ProductName and ProductUrl both have a String locale and String name or String url accordingly
I am currently building my query with the following logic
private SearchQuery buildSearchQuery(String searchTerm, List<Long> categories, Pageable pageable) {
NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
if (searchTerm != null) {
builder.withQuery(
new MultiMatchQueryBuilder(searchTerm, "names.name", "urls.url", "descriptions.description", "sku")
.operator(Operator.AND)
.type(MultiMatchQueryBuilder.Type.MOST_FIELDS)
);
}
builder.withPageable(pageable);
return builder.build();
}
The problem is that lots of products are not being matched, for example:
query "andro" does not return "android" products.
What am I missing? Is this way of building the query right?
UPDATE
Adding the names part of my product mapping:
{
"mappings": {
"product": {
"properties": {
"names": {
"properties": {
"id": {
"type": "long"
},
"name": {
"type": "string"
},
"locale": {
"type": "string"
}
}
}
}
}
}
}