I am learning and follow tutorial at https://www.udemy.com/course/elasticsearch-complete-guide/ .
I am using elasticsearch-7.12.0, Kibana 7.12.0, Logstash 7.12.0 on Windows 10 x64, I have document
{
"index": {
"_id": 1
}
}
{
"name": "Wine - Maipo Valle Cabernet",
"price": 152,
"in_stock": 38,
"sold": 47,
"tags": [
"Alcohol",
"Wine"
],
"description": "Aliquam augue quam, sollicitudin vitae, consectetuer eget, rutrum at, lorem. Integer tincidunt ante vel ipsum. Praesent blandit lacinia erat. Vestibulum sed magna at nunc commodo placerat. Praesent blandit. Nam nulla. Integer pede justo, lacinia eget, tincidunt eget, tempus vel, pede. Morbi porttitor lorem id ligula.",
"is_active": true,
"created": "2004\/05\/13"
}
{
"index": {
"_id": 2
}
}
{
"name": "Tart Shells - Savory",
"price": 99,
"in_stock": 10,
"sold": 430,
"tags": [],
"description": "Pellentesque at nulla. Suspendisse potenti. Cras in purus eu magna vulputate luctus. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Vivamus vestibulum sagittis sapien. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Etiam vel augue. Vestibulum rutrum rutrum neque. Aenean auctor gravida sem.",
"is_active": true,
"created": "2007\/10\/14"
}
Full file: https://gist.github.com/donhuvy/759d48d2ec96f70531d77d173cd9657e
I try
PUT _bulk
{
"index": {
"_id": 1
}
}
{
"name": "Wine - Maipo Valle Cabernet",
"price": 152,
"in_stock": 38,
"sold": 47,
"tags": [
"Alcohol",
"Wine"
],
"description": "Aliquam augue quam, sollicitudin vitae, consectetuer eget, rutrum at, lorem. Integer tincidunt ante vel ipsum. Praesent blandit lacinia erat. Vestibulum sed magna at nunc commodo placerat. Praesent blandit. Nam nulla. Integer pede justo, lacinia eget, tincidunt eget, tempus vel, pede. Morbi porttitor lorem id ligula.",
"is_active": true,
"created": "2004\/05\/13"
}
{
"index": {
"_id": 2
}
}
{
"name": "Tart Shells - Savory",
"price": 99,
"in_stock": 10,
"sold": 430,
"tags": [],
"description": "Pellentesque at nulla. Suspendisse potenti. Cras in purus eu magna vulputate luctus. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Vivamus vestibulum sagittis sapien. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Etiam vel augue. Vestibulum rutrum rutrum neque. Aenean auctor gravida sem.",
"is_active": true,
"created": "2007\/10\/14"
}
but not success.
My error
How to import the above document with specific format to Elasticsearch?
I did
PUT /vy
POST /vy/_bulk
{"index":{"_id":1}}
{"purchased_at":"2016-07-10T16:52:43Z","lines":[{"product_id":6,"amount":71.32,"quantity":1},{"product_id":3,"amount":58.96,"quantity":3},{"product_id":1,"amount":29.8,"quantity":3}],"total_amount":160.08,"salesman":{"id":11,"name":"Matthus Mitkcov"},"sales_channel":"store","status":"processed"}
{"index":{"_id":2}}
{"purchased_at":"2016-03-08T16:18:37Z","lines":[{"product_id":7,"amount":31.44,"quantity":2},{"product_id":4,"amount":74.82,"quantity":2}],"total_amount":106.26,"salesman":{"id":79,"name":"Ulberto Woodruff"},"sales_channel":"store","status":"completed"}
{"index":{"_id":3}}
{"purchased_at":"2016-01-04T02:46:54Z","lines":[{"product_id":9,"amount":38.89,"quantity":1}],"total_amount":38.89,"salesman":{"id":68,"name":"Thea Chidgey"},"sales_channel":"store","status":"completed"}
{"index":{"_id":4}}
{"purchased_at":"2016-04-14T11:44:29Z","lines":[{"product_id":4,"amount":26.18,"quantity":2},{"product_id":8,"amount":76.45,"quantity":1},{"product_id":1,"amount":65.59,"quantity":3}],"total_amount":168.22,"salesman":{"id":81,"name":"Melany Cerith"},"sales_channel":"store","status":"cancelled"}
{"index":{"_id":5}}
{"purchased_at":"2016-02-18T17:11:21Z","lines":[{"product_id":2,"amount":56.06,"quantity":3},{"product_id":5,"amount":79.75,"quantity":3}],"total_amount":135.81,"salesman":{"id":23,"name":"Flynn Dome"},"sales_channel":"web","status":"completed"}
{"index":{"_id":6}}
{"purchased_at":"2016-03-26T07:28:22Z","lines":[{"product_id":7,"amount":86.88,"quantity":1},{"product_id":5,"amount":27.34,"quantity":3},{"product_id":5,"amount":88.52,"quantity":2}],"total_amount":202.74,"salesman":{"id":16,"name":"Mirilla Penkman"},"sales_channel":"web","status":"completed"}
{"index":{"_id":7}}
{"purchased_at":"2016-05-06T18:37:33Z","lines":[{"product_id":3,"amount":25.27,"quantity":2}],"total_amount":25.27,"salesman":{"id":5,"name":"Modestia McCreadie"},"sales_channel":"app","status":"confirmed"}
{"index":{"_id":8}}
{"purchased_at":"2016-11-26T11:24:36Z","lines":[{"product_id":4,"amount":51.26,"quantity":1},{"product_id":8,"amount":72.41,"quantity":1}],"total_amount":123.67,"salesman":{"id":55,"name":"Grange Smiths"},"sales_channel":"phone","status":"cancelled"}
{"index":{"_id":9}}
{"purchased_at":"2016-12-26T08:29:47Z","lines":[{"product_id":2,"amount":98.87,"quantity":3}],"total_amount":98.87,"salesman":{"id":5,"name":"Carlyn Stegers"},"sales_channel":"web","status":"cancelled"}
{"index":{"_id":10}}
{"purchased_at":"2016-12-21T12:47:43Z","lines":[{"product_id":7,"amount":21.97,"quantity":3},{"product_id":10,"amount":19.26,"quantity":3}],"total_amount":41.23,"salesman":{"id":83,"name":"Kennie Gilks"},"sales_channel":"app","status":"cancelled"}
{"index":{"_id":11}}
{"purchased_at":"2016-04-02T04:42:31Z","lines":[{"product_id":6,"amount":86.89,"quantity":1},{"product_id":4,"amount":18.04,"quantity":3},{"product_id":10,"amount":18.61,"quantity":3}],"total_amount":123.54,"salesman":{"id":39,"name":"Ashely Rawnsley"},"sales_channel":"phone","status":"cancelled"}
{"index":{"_id":12}}
{"purchased_at":"2016-09-08T16:24:12Z","lines":[{"product_id":1,"amount":11.75,"quantity":1},{"product_id":5,"amount":77.33,"quantity":2},{"product_id":2,"amount":86.76,"quantity":1}],"total_amount":175.84,"salesman":{"id":40,"name":"Raynor Dennis"},"sales_channel":"web","status":"pending"}
{"index":{"_id":13}}
{"purchased_at":"2016-01-12T06:20:27Z","lines":[{"product_id":6,"amount":93.69,"quantity":2}],"total_amount":93.69,"salesman":{"id":62,"name":"Byran Gallihaulk"},"sales_channel":"app","status":"processed"}
{"index":{"_id":14}}
{"purchased_at":"2016-01-31T13:06:58Z","lines":[{"product_id":10,"amount":80.33,"quantity":2}],"total_amount":80.33,"salesman":{"id":43,"name":"Eveleen Ickovici"},"sales_channel":"store","status":"processed"}
{"index":{"_id":15}}
{"purchased_at":"2016-02-20T23:02:31Z","lines":[{"product_id":5,"amount":20.51,"quantity":1},{"product_id":10,"amount":93.35,"quantity":1}],"total_amount":113.86,"salesman":{"id":78,"name":"Erda Boyet"},"sales_channel":"store","status":"confirmed"}
{"index":{"_id":16}}
{"purchased_at":"2016-08-17T09:52:50Z","lines":[{"product_id":9,"amount":61.95,"quantity":1}],"total_amount":61.95,"salesman":{"id":91,"name":"Gherardo MattiCCI"},"sales_channel":"web","status":"confirmed"}
{"index":{"_id":17}}
{"purchased_at":"2016-06-05T18:59:23Z","lines":[{"product_id":8,"amount":32.46,"quantity":3},{"product_id":6,"amount":13.7,"quantity":2},{"product_id":8,"amount":28.74,"quantity":2}],"total_amount":74.9,"salesman":{"id":82,"name":"Shanon Eilhertsen"},"sales_channel":"phone","status":"confirmed"}
{"index":{"_id":18}}
{"purchased_at":"2016-07-26T12:10:15Z","lines":[{"product_id":3,"amount":23.16,"quantity":1},{"product_id":4,"amount":18.94,"quantity":1},{"product_id":10,"amount":38.65,"quantity":2}],"total_amount":80.75,"salesman":{"id":58,"name":"Thekla De Beauchemp"},"sales_channel":"phone","status":"processed"}
{"index":{"_id":19}}
{"purchased_at":"2016-05-08T19:54:44Z","lines":[{"product_id":9,"amount":29.8,"quantity":2},{"product_id":6,"amount":97.61,"quantity":3}],"total_amount":127.41,"salesman":{"id":96,"name":"Susannah Grigoryev"},"sales_channel":"phone","status":"cancelled"}
{"index":{"_id":20}}
{"purchased_at":"2016-06-19T21:34:51Z","lines":[{"product_id":2,"amount":23.3,"quantity":2},{"product_id":5,"amount":30.44,"quantity":2},{"product_id":6,"amount":22.15,"quantity":1}],"total_amount":75.89,"salesman":{"id":66,"name":"Janeta Crutchfield"},"sales_channel":"phone","status":"completed"}
{"index":{"_id":21}}
{"purchased_at":"2016-12-09T17:24:57Z","lines":[{"product_id":1,"amount":90.25,"quantity":1}],"total_amount":90.25,"salesman":{"id":20,"name":"Saxon Whitebread"},"sales_channel":"web","status":"cancelled"}
{"index":{"_id":22}}
{"purchased_at":"2016-09-18T12:26:52Z","lines":[{"product_id":7,"amount":48.03,"quantity":3},{"product_id":1,"amount":18.25,"quantity":1}],"total_amount":66.28,"salesman":{"id":96,"name":"Annissa Cleever"},"sales_channel":"web","status":"completed"}
{"index":{"_id":23}}
{"purchased_at":"2016-09-28T04:29:57Z","lines":[{"product_id":1,"amount":27.74,"quantity":3},{"product_id":8,"amount":82.88,"quantity":1}],"total_amount":110.62,"salesman":{"id":15,"name":"Catherina Tombleson"},"sales_channel":"phone","status":"pending"}
{"index":{"_id":24}}
{"purchased_at":"2016-08-05T23:08:23Z","lines":[{"product_id":5,"amount":90.7,"quantity":1}],"total_amount":90.7,"salesman":{"id":4,"name":"Saloma Raith"},"sales_channel":"app","status":"cancelled"}
{"index":{"_id":25}}
{"purchased_at":"2016-09-02T21:14:18Z","lines":[{"product_id":10,"amount":59.98,"quantity":1},{"product_id":4,"amount":34.72,"quantity":1}],"total_amount":94.7,"salesman":{"id":85,"name":"Raffaello Larrad"},"sales_channel":"web","status":"cancelled"}
{"index":{"_id":26}}
{"purchased_at":"2016-11-12T00:57:06Z","lines":[{"product_id":4,"amount":39.05,"quantity":3},{"product_id":8,"amount":39.29,"quantity":3}],"total_amount":78.34,"salesman":{"id":40,"name":"Maryanne O'Neill"},"sales_channel":"store","status":"confirmed"}
{"index":{"_id":27}}
{"purchased_at":"2016-12-30T07:37:52Z","lines":[{"product_id":7,"amount":12.41,"quantity":3},{"product_id":5,"amount":55.88,"quantity":3},{"product_id":6,"amount":33.89,"quantity":1}],"total_amount":102.18,"salesman":{"id":25,"name":"Myles Sedgemore"},"sales_channel":"web","status":"cancelled"}
{"purchased_at":"2016-10-03T15:57:59Z","lines":[{"product_id":9,"amount":12.15,"quantity":3},{"product_id":4,"amount":54.07,"quantity":1},{"product_id":10,"amount":83.01,"quantity":3}],"total_amount":149.23,"salesman":{"id":58,"name":"Leodora Mithon"},"sales_channel":"phone","status":"pending"}
GET /vy/_search
{
"query": {
"match_all": {}
}
}
In my Elasticsearch index, I have a book type that has many pages. Using nested documents, I have successfully mapped the relationship and I can index documents like this:
PUT /my_index/book/1
{
"title": "Harry Potter and the Chamber of Secrets",
"pages": [
{
"id": 1,
"text": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam a metus est. Duis ut est et mi feugiat bibendum feugiat eu tortor. Pellentesque accumsan, eros nec commodo euismod, odio dolor lobortis diam, in pulvinar lacus turpis sed justo. Ut placerat ut nulla sed blandit. Aenean vel turpis erat. Phasellus vehicula laoreet ex, nec dapibus leo tempus vitae. Nulla gravida efficitur metus, in euismod justo placerat sit amet. Maecenas tristique est mauris, sagittis scelerisque turpis suscipit vel. Nullam ultricies sapien sit amet neque aliquam hendrerit sed non nibh."
},
{
"id": 2,
"text": "Pellentesque facilisis turpis in diam maximus luctus. Mauris leo diam, pellentesque a malesuada vitae, scelerisque at ipsum. Fusce tincidunt neque dui. Nullam ac ex luctus, convallis leo eget, feugiat augue. Cras condimentum, purus eu scelerisque sodales, diam est commodo lectus, at finibus orci turpis nec lectus. Mauris in lectus ut diam finibus pellentesque quis tincidunt urna. Curabitur tristique luctus metus at interdum. Curabitur imperdiet ex vel enim pretium, a convallis velit tempor. Nullam odio eros, tincidunt ut consectetur non, scelerisque eget urna. Fusce placerat dui et odio tempus rutrum. Integer non dui eu ante interdum volutpat. Mauris quis ante sed lacus euismod mattis."
}
]
}
I want to query and obtain all pages from a book which contain a certain word in the text field. I can do this like this:
POST /my_index/book/_search
{
"_source": false,
"query": {
"nested": {
"path": "pages",
"query": {
"query_string": {
"query": "Mundo",
"fields": ["pages.text"]
}
},
"inner_hits": {}
}
}
}
My problem however, is that I cannot find a way to specify which book I want to search in. Since I already know which book I'll be searching all pages from. I was trying to do this with a ids query, but I cannot find a way to use it with the nested query together.
How can I specify the book I want to search in (pages from this book), and make the response return a list of all the id values of each page? This id value is a reference to the page's page number in the book.
You can combine two conditions under Bool query like so:
{
"_source": false,
"query": {
"bool": {
"must": [
{
"term": {
"title": "Harry Potter and the Chamber of Secrets"
}
},
{
"nested": {
"path": "pages",
"query": {
"query_string": {
"query": "Mundo",
"fields": [
"pages.text"
]
}
},
"inner_hits": {
"_source": [
"pages.id"
]
}
}
}
]
}
}
}
As can be seen, I have encapsulate your nested query under bool-must (AND condition) and added term which defines the title to filter by.
Second thing, as you described you want only the pages-ids. So, under inner_hits I added "_source": ["id"] which gives you the ids only (it just the same trick
you did when you set the source off at the head of the query.
Pay attention, that, default size for inner_hits is 10, so you may also want to add "size": 1000 under inner_hits.
I will have another data model for your problem.
I will create two types:
A type book
A type Page
You will be able to create a parent/child relationship (The book being the parent and the page the child).
You will be able to make simple query on the page type then retrieve easily the parent of the page.