How to write ES search queries for complex documents structure? - elasticsearch

I have the following documents and structure:
Document 1:
{
'id': '1',
'data': {
'parents': [],
'people': []
}
Document 2:
{
'id': '2',
'data': {
'parents': ["one", "two"],
'people': [{'relationship': 'boss', 'ids': ['1']}, {'relationship': 'friends', 'ids': ['1']}]
}
Document 3:
{
'id': '3',
'data': {
'parents': ["one", "two"],
'people': [{'relationship': 'boss', 'ids': ['1', '2']}, {'relationship': 'friends', 'ids': ['1', '2']}]
}
}
I want to delete a document given an id and then delete any relationships with document in other documents.
So given id = '1', document 1 is deleted, and document 2 and 3's relationship entires that include '1' are updated to not include '1' anymore; so after document 1 is deleted, we have the following:
Document 2:
{
'id': '2',
'data': {
'parents': ["one", "two"],
'people': []
}
Document 3:
{
'id': '3',
'data': {
'parents': ["one", "two"],
'people': [{'relationship': 'boss', 'ids': ['2']}, {'relationship': 'friends', 'ids': ['2']}]
}
}
I am struggling writing the ES queries because the document's structure is a complex one;
What I have tried so far are the following:
{ 'query': { 'match': { 'id.keyword': '1' } } }
{ 'terms': {data.people.ids.keyword: ['1'], 'minimum_should_match': 1 } }
but they do not work, nor do they throw an error.

Related

Elasticsearch how to get objects from a nested structure

Here is the structure of my document:
{
'parent_list': [
{
'child_list': [
{
'name': 'A',
'score': 0.86
},
{
'name': 'B',
'score': 0.92
},
{
'name': 'C',
'score': 0.77
}
]
},
{
'child_list': [
{
'name': 'B',
'score': 0.41
},
{
'name': 'D',
'score': 0.66
}
]
},
]
}
What I want is to get the object with maximum score property from each child_list, i.e. From the document above i would like to get something like this:
[
{
'name': 'B',
'score': 0.92
},
{
'name': 'D',
'score': 0.66
}
]
Or, it would also be plenty to just receive the name property of these objects.
I use nested mapping type for both parent_list and child_list, name is text, and score is float.
So far i have tried using nested aggregation/queries and I am able to get max score from each of the child_list lists, so that would be: 0.92, 0.66. But i cannot figure out how to get the corresponding name property. Any help is welcome.

Laravel collection: return only relationship and key

I have a collection that resembles this:
$a = Model::with(['sub' => function($q) {
$q->select('id', 'name')
}])->get();
This returns the following collection:
{
0: {
id: 0001,
name: "item 1",
type: "type a"
'sub' [
{
'id': 10001,
'name': "sub Item 1"
},
{
'id': 10002,
'name': "sub Item 2"
}
]
},
1: {
id: 0002,
name: "item 2",
type: "type a"
'sub' [
{
'id': 11001,
'name': "sub Item 4"
},
{
'id': 11002,
'name': "sub Item 5"
}
]
}
What I am trying to do is key the parent items by their ids and only return the relationship. For example
{
0001: {
'sub' [
{
'id': 10001,
'name': "sub Item 1"
},
{
'id': 10002,
'name': "sub Item 2"
}
]
},
0002: {
'sub' [
{
'id': 11001,
'name': "sub Item 4"
},
{
'id': 11002,
'name': "sub Item 5"
}
]
}
I cannot seem to get this to work. I have tried many variations including:
$a = Model::with(['sub' => function($q) {
$q->select('id', 'name')
}])->pluck('sub', 'id');
This doesn't work as 'Pluck' is obviously looking for a a property of the parent model with the name of 'sub' which doesn't exit. Is there a way to achieve this?
Thanks
You were almost there. You will need to do ->get() before the pluck().
$a = Model::with([
'sub' => function ($q) {
$q->select('id', 'name');
},
])->get()->pluck('sub', 'id');
The pluck() used in your example will be the query builder version of pluck rather than the collection version.
use keyBy to use your pk as array index.
https://laravel.com/docs/5.4/collections#method-keyby
However ignoring other fields you probably would need each and filter. Wouldn't it be easier to select Sub::where(...) and then use collection groupBy on the parent_id: https://laravel.com/docs/5.4/collections#method-groupby
So something like Sub::where(...)->get()->groupBy('parent_id')

Lodash filter unique documents

How can I filter the current data:
[{
key: 'T1',
legs:[{ fno: 'W321',date: '2017-01-02 18:20:00.000+0200'}],
fare: { type: 'B', price: 25 }
},{
key: 'T1',
legs:[{ fno: 'W321', date: '2017-01-02T18:20:00.000+0200'}],
fare: { type: 'E', price: 23 }
},{
key: 'T1',
legs:[{ fno: 'W321', date: '2017-01-02T18:20:00.000+0200'}],
fare: { type: 'E', price: 20}
}]
I want to group by legs[0].fno, legs[0].date and fare.type, and keep the lowest priced items in each group. This is the expected result:
[{
key: 'T1',
legs:[{ fno: 'W321',date: '2017-01-02T18:20:00.000+0200'}],
fare: { type: 'B', price: 25}
},{
key: 'T1',
legs:[{ fno: 'W321',date: '2017-01-02T18:20:00.000+0200'}],
fare: { type: 'E', price: 20}
}]
Use _.groupBy() with a callback to create a string to group by, then _.map() each group to a single item using _.minBy():
var data = [{"key":"T1","legs":[{"fno":"W321","date":"2017-01-02 18:20:00.000+0200"}],"fare":{"type":"B","price":25}},{"key":"T1","legs":[{"fno":"W321","date":"2017-01-02T18:20:00.000+0200"}],"fare":{"type":"E","price":23}},{"key":"T1","legs":[{"fno":"W321","date":"2017-01-02T18:20:00.000+0200"}],"fare":{"type":"E","price":20}}];
var result = _(data)
// group by the combined group keys
.groupBy(function(o) {
// extract all group keys and join them to a string
return _.at(o, ['key', 'legs[0].date', 'fare.type']).join('');
})
.map(function(group) {
// get the object object with the minimum fare.price
return _.minBy(group, 'fare.price');
})
.value();
console.log(result);
<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.4/lodash.min.js"></script>

RethinkDb: Has many query

Have two table, Quiz and Questions
Quiz table store questions id and its display position.
Display position is different for every Quiz.
Quiz table:
{
id: '1'
name: 'Quiz 1'
questions: [
{
question_id: '1',
position: 4
},
{
question_id: '2',
position: 1
},
......
]
}
...
Question Table:
[
{
id: '1',
title: 'Question 1'
},
{
id: '2'
title: 'Question 2'
}
]
I want the the return result like this
{
id: '1',
name: 'Quiz 1',
questions: [
{
position: 4,
title: 'Question 1'
},
{
position: 1,
title: 'Question 2'
}
]
}
How can i get the desired result?
thanks
I got the desired result by using following query
r.db('database').table("quizes").get(id)
.do((quiz) => {
return quiz.merge({'questions': quiz('questions').map((q) => {
return q.merge((r.db('database').table('questions').get(q('id')).without('id')))
})
})
})

RethinkDB - How to return a sliding window on grouped data

I have some objects:
[
{ 'time': 1, 'data': { '1': 10, '2': 100} },
{ 'time': 2, 'data': { '1': 20, '2': 100} },
{ 'time': 3, 'data': { '1': 30, '2': 200} },
{ 'time': 4, 'data': { '1': 40, '2': 100} },
{ 'time': 5, 'data': { '1': 50, '2': 300} },
{ 'time': 6, 'data': { '1': 60, '2': 200} }
]
and two variables width and overlap:
width - max length of 'values' list
overlap - number of mutual values
Assume width = 3 and overlap = 2. Is there any way to obtain following?
[ { 'key': '1',
'rows': [ { 'time': 1, 'values': [10,20,30] },
{ 'time': 2, 'values': [20,30,40] },
{ 'time': 3, 'values': [30,40,50] },
{ 'time': 4, 'values': [40,50,60] }
]
},
{ 'key': '2',
'rows': [ { 'time': 1, 'values': [100,100,200] },
{ 'time': 2, 'values': [100,200,100] },
{ 'time': 3, 'values': [200,100,300] },
{ 'time': 4, 'values': [100,300,200] }
]
} ]
So far I've managed to get this:
[ { 'key': '1',
'row': { 'time': 1, 'values': [10,20,30,40,50,60] }
},
{ 'key': '2',
'row': { 'time': 1, 'values': [100,100,200,100,300,200] }
} ]
using this:
.concatMap(function(item) {
return item('data').keys().map(function(key) {
return {
'key': key,
'row': {
'time': item('time'),
'values': [item('data')(key)]
}
}
})
})
.group('key')
.ungroup()
.map(function(list) {
return list('reduction').reduce(function(left, right) {
return {
'key': left('key'),
'row': {
'time': left('row')('time'),
'values': left('row')('values').union(right('row')('values'))
}
}
})
})
Maybe I need to add something or to change everything?
Thanks.
This is pretty similar to mlucy's solution, but it doesn't assume that the time fields are consecutive integers. The data is sorted by time before the concatMap below - for large datasets, this should be done with an index.
r.expr(data)
.orderBy('time')
.concatMap(function (row) {
return row('data').coerceTo('array').map(function (pair) {
return { key: pair(0), value: pair(1), time: row('time') };
});
})
.group('key')
.ungroup()
.map(function (g) {
let rows = g('reduction').count().do(function (c) {
return r.range(c.sub(2)).map(function (i) {
let values = r.range(3).map(function (j) {
return g('reduction')(i.add(j))('value');
}).coerceTo('array');
return { 'time': g('reduction')(i)('time'), 'values': values };
}).coerceTo('array');
});
return { key: g('group'), rows: rows };
})
You probably want something like this:
r.table('test').orderBy('time').concatMap(function(row) {
return row('data').coerceTo('array');
}).group(function(pair) {
return pair(0);
}).map(function(pair) {
return pair(1);
}).ungroup().map(function(group) {
return {
key: group('group'),
rows: group('reduction').do(function(x) {
return r.range(x.count().sub(2)).map(function(i) {
return {
time: i,
values: r.range(3).map(function(o) {
return x(i.add(o));
}).coerceTo('array')
};
}).coerceTo('array');
})
};
})
(Where the .sub(2) and .range(3) need to change based on the width and overlap.)

Resources