I going to do a query that related to two _index ( conn, http). In sql it can be do like following, may I know how DSL query like in elasticSearch ? Please assist me, I really out of idea.
SELECT * FROM elastic INNER JOIN conn ON conn.uid=http.uid
WHERE elastic.date BETWEEN '2016-08-20 00:00:00' and '2016-08-22 13:41:09'
AND conn.service='http' ;
elasticsearch doesn't have support for joins like SQL.
They discuss the handling of relationships in the definitive guide: https://www.elastic.co/guide/en/elasticsearch/guide/current/relations.html
Related
(Note: all code examples are extremely simple. I know there are other ways to do such simple queries. The problem I am demonstrating, however, is a bigger deal for more complex queries).
There is a known issue with Spring JPA Repositories and paginated queries that I'm really hoping there is a good solution for. In JPQL, it is possible to use JOIN FETCH to specify that I want to eagerly fetch a related entity, rather than doing it lazily. This avoids the N+1 problem, among other things. JOIN FETCH requires that the owner of the association is included in the select clause. Here is a very simple example of the type of query I'm talking about:
#Query("""
SELECT p
FROM Person p
JOIN FETCH p.address
""")
Page<Person> getPeopleAndAddresses(Pageable page);
The problem with this kind of query is the pagination piece. When returning a Page, Spring will do the query I wrote but then also do a count query to get the total possible records. Spring appears to take my query exactly as written, and just replace SELECT p with SELECT COUNT(p). Doing this means that the owner of the JOIN FETCH association is no longer present in the SELECT clause, which then results in the JPQL giving an error.
The only way I know how to resolve this is to construct the query with separate query and countQuery values, like this:
#Query(query = """
SELECT p
FROM Person p
JOIN FETCH p.address
""", countQuery = """
SELECT COUNT(p)
FROM Person p
""")
Page<Person> getPeopleAndAddresses(Pageable page);
This resolves the JPQL JOIN FETCH error, because the count query no longer contains a JOIN FETCH clause. However, for complex queries with sophisticated JOINs and WHERE clauses, this will lead to excessive code duplication as I will have to write all that logic in two places.
This seems like the kind of issue where there really should be a better solution available. I'm exploring various alternatives, including Specifications (mixed feelings), Blaze Persistence, and others. However, I'm wondering if there is some way in Spring itself to resolve this issue so that the first code example would work without an error?
can anyone help me to translate below SQL query to ELasticSearch DSL form for document selections?
SELECT * FROM table1 WHERE source_id IN (select source_id FROM table2)
This JOIN semantics does not translate to Elasticsearch. If your index had parent/child mapping - you might be able to achieve what you want to do. Another option is to do a two-pass query.
This blog post gives an idea of how relations are modeled in Elasticsearch - Managing Relations in Elasticsearch
I have created an elasticsearch index against a news table on sqlserver using logstash via the JDBC Driver. This all looks good in Elasticsearch.
Using Index Server, the type of query that gets built for that takes the following form:
SELECT News.*, fulltextsearch.rank FROM News INNER JOIN CONTAINSTABLE(News, ( Headline, BodyText ), 'ISABOUT("car track race" WEIGHT(0.65), car NEAR track NEAR race)') fulltextsearch ON News.NewsID = fulltextsearch.[Key] WHERE DateSubmitted <= '01/11/2017' ORDER BY fulltextsearch.rank DESC
Is there any kind of query that I can do in Elasticsearch to give a similar/same outcome as the above.
No, elasticsearch (version 5.3) do not support JOIN like this. See https://www.elastic.co/guide/en/elasticsearch/reference/current/joining-queries.html.
SELECT u.Address,c.locality,jaccard_similarity(u.Address, c.locality) as score
FROM users u
left join communites c on jaccard_similarity(u.Address, c.locality) >=0.65
How do I achieve the same functionality in Elasticsearch?
There is no way to do that natively in elasticsearch.
You'll have to either use nested objects or do application joins ( doing 2 queries in your application).
The documentation will explain the strategies : https://www.elastic.co/guide/en/elasticsearch/guide/current/relations.html
Can you nest queries logically in ElasticSearch, so the output of one query is the input to another query.
Another, way to ask is how can I chain or pipe queries together?
This should be analogous to the IN operator or subqueries in SQL
i.e.:-
select au_lname, au_fname, title from
(select au_lname, au_fname, au_id from pubs.dbo.authors
where state = 'CA')
or
SELECT Name
FROM AdventureWorks2008R2.Production.Product
WHERE ListPrice =
(SELECT ListPrice
FROM AdventureWorks2008R2.Production.Product
WHERE Name = 'Chainring Bolts' );
Elasticsearch doesn't support subqueries; you would need to perform your first query, then construct a second query using the results of the first query as an input.
this is not supported in elastic-search you must normalize your data and have all field you need in one setting
That is totally correct, you must programm a subquery in your favorite programming language. An example can be found here:
https://sebastianviereck.de/elasticsearch-subquery-scoring-optimization/