I'm relatively new to MongoDB and trying to figure out how to search for data which has dynamic keys, my dataset is below:
[
'id' => '60bb040f7e589378f644f9a2',
'name' => 'foo',
'settings' => [
'foo' => [
'type' => 'custom',
'enabled' => 'yes'
],
'bar' => [
'type' => 'custom',
'enabled' => 'no'
],
'baz' => [
'type' => 'custom',
'enabled' => 'no'
]
]
]
I want to get the search inside settings, and get all the settings which has enabled as yes.
I've tried this
return Customer::raw(function($collection) {
return $collection->aggregate([
[
'$search' => [
"wildcard" => [
'query' => 'yes',
'path' => 'settings.*.enabled'
]
]
]
]);
});
I am getting the following errror:
Unrecognized pipeline stage name: '$search'
I've also tired using $match, like this
Customer::raw(function($collection)
{
return $collection->aggregate([
[
'$match' => [
"settings.*.enabled" => 'yes'
]
],
]);
})
But it does't return any value.
Any direction on achieving this will be helpful. Thanks!
Ok this is a bit complex but is all I can come up with myself:
Customer::raw(function($collection)
{
return $collection->aggregate([
[
'$addFields' => [
"settingsArray" => [
'$objectToArray' => '$settings'
]
]
],
[
'$unwind' => [
'path' => '$settingsArray'
]
],
[
'$match' => [
'settingsArray.v.enabled' => 'yes'
]
],
[ '$unset' => 'settingsArray' ],
[
'$group' => [
'_id' => '$_id',
doc => [ '$first' => '$$ROOT' ]
]
],
[
'$replaceRoot' => [
'newRoot' => '$doc'
]
]
]);
});
This should return all documents with at least one enabled setting.
A key point about aggregation is that the pipeline works on all documents that the previous stage of the pipeline returned.
Here's the aggregation pipeline explanation:
$addFields will convert the settings object to an array of format [ { k: key, v: value } ] e.g. [ { k: 'foo', 'v': { type: 'custom', enabled: 'yes' } }, ... ]
$unwind will "unwind" the document and "replace" it with X copies of the same document one for each entry in the newly added settingsArray array. The settingsArray at this stage is not an array anymore but a single entry of type { k: key, v: value }. In short you'd get one document per setting
$match will do the actual matching for any document with an enabled setting (remember now each document will only have a single setting as settingsArray
$unset will remove the settingsArray from each document bringing it back to its original format
$group will group all documents by their _id and use the first result of each group as a doc entry. Each document will now be like { _id: <id>, doc: <original document> }
$replaceRoot will set the root of the document to doc and therefore bring the documents back to their original format.
Would love to hear if anyone else has a better solution here.
Related
public function test_show_role_should_return_actions()
{
$this->actingAs($this->user, 'api');
$this->withHeaders(['Accept' => 'application/json',])
->get(route('roles.show', ['role' => $this->role->id]))
->assertJsonStructure([
"data" => [
"name",
"actions" => [
"name",
"code"
]
]
]);
}
when I run the test,I have this error:
Tests\Feature\Role\RoleTest::test_show_role_should_return_actions
Failed asserting that an array has the key 'name'.
Even when I remove the "name",I get an error for the "actions" key.
public function test_show_role_should_return_actions()
{
$this->actingAs($this->user, 'api');
$this->getJson(route('roles.show', ['role' => $this->role->id]))
->assertJsonStructure([
'data' => [
'*' => [
'name',
'actions' => [
'*' => [
'name',
'code'
]
]
]
]
]);
}
In our app, ES holds objects with areas field, where areas field in a type of MultiPyligon. (basically, it's an array of polygons).
Now, we need to search for all the objects in which one of their polygons in at least partially falls within a given polygon (in our case it is the current viewport of the map).
The current query that we are experimenting with is the following:
$params = [
'index' => self::CrimeIndex,
'body' => [
'size' => 10000,
'query' => [
'bool' => [
'filter' => [
'geo_bounding_box' => [
'areas' => [
"top_left" => [
"lat" => $neLat,
"lon" => $neLng
],
"bottom_right" => [
"lat" => $swLat,
"lon" => $swLng
]
],
]
]
]
]
],
];
The problem is that this query gets all the polygons that touch the edges of the bounding box. (see picture). How can we get all the polygons that are at least partially within the bounding box?
Mappings are done as follows:
$params = [
'index' => CrimeService::CrimeIndex,
'body' => [
"mappings" => [
'properties' => [
'areas' => [
'type' => 'geo_shape'
]
],
],
],
];
$client->indices()->create($params);
Based on the docs, geo_shape can be MultiPolygon.
https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-shape.html
And here is the example of how it looks like populated:
GET crimes/_mapping/field/areas provides the following:
UPDATE - More Detailed Steps to reproduce
The dump of the collection/index is attached: https://www.dropbox.com/s/8inavsvcrnuozw1/dump-2021-12-29t21_54_04.639z.json.zip?dl=0
The query that is executed with elasticsearch-php is:
$params = [
'index' => 'crime',
'body' => [
'size' => 10000,
'query' => [
'bool' => [
'filter' => [
'geo_bounding_box' => [
'areas' => [
"top_left" => [
"lat" => $neLat,
"lon" => $neLng
],
"bottom_right" => [
"lat" => $swLat,
"lon" => $swLng
]
],
]
],
]
]
],
];
If we execute it with the parameters:
49.29366604017385,-123.00491857934166,49.19709977562233,-123.26617317321401
We get the following:
In case that the viewport is changed a bit, so the polygons touch the borders of viewport: 49.28031011582358,-122.92300503734472,49.18371770837152,-123.18425963121705,
we get the rest of the polygons:
Your query coordinates are wrong, instead of top_left + bottom_right, you have bottom_left + top_right (see image below)
I think that pretty much explains why you're seeing what you're seeing.
I have this problem, i want to sort on 2 match, and my code look like this right now (its in php array)
$data = [
'query' => [
'bool' => [
'must' => [
'multi_match' => [
'query' => $q,
'fields' => ['Fulltext'],
'type' => 'cross_fields',
'operator' => 'and'
]
],
'should' => [
'match' => [
'Title' => $q,
]
]
]
],
'aggregations' => [
'categorys' => [
'terms' => [
'field' => 'Categorys.value',
'size' => $category_size
]
]
],
'size' => $product_size,
'sort' => [
'Popular' => [
'order' => 'desc'
]
]
];
The problem in this sort i want to sort it by "Popular" but, its not first sort on match in "should" and then after in "must" so my question is.
How can i start sorting in the first match points, and then sort in the secound match points based on Popular field?
The problem is i want to search in prouct where the query match in Title is higher importen then in Fulltext field.
so if i get 10 resualt based on Title and the score are highter then the next 10 resualt based on Fulltext, but we have 3 reusalt from Title and 2 from Fulltext where Probular is gretnder then 0 then this match shut be view'en and sorted by (_score, Popular) if Popular is greather then zero else sort just based on (_score)
Can eny body help me with this question?
I have a question about I can accomplish something.
I have my search algorithm for user documents ready.
I get the list of documents, but I don't wanna have the list to have grouped parts of documents of the same user.
Eg:
doc1: user-1
doc2: user-2
doc3: user-2
doc4: user-3
doc5: user-4
Change to:
doc1: user-1
doc2: user-2
doc4: user-3
doc5: user-4
doc3: user-2
Kind of sorting/randomising...
Any tips, ideas for what I can search?
Or much better, some examples.
I'm quite new to elastic search. The documentation about custom-scoring or ordering is great but not giving me the right answer.
Thanks a million
Stefan
Update 18.08.:
As wished, here also my current query.
'query' => [
'filtered' => [
'query' => [
'bool' => [
'must' => [
'multi_match' => [
'query' => $q,
'fields' => [ 'title^6', 'description^1', 'tags^3']
]
],
'should' => [
[
'match' => [
'isTopDocument' => [
'query' => 'true',
'boost' => 2,
]
]
],[
'range' => [
'online_start' => [
'boost' => 1.8,
'gte' => 'now-7d/d'
]
]
],[
'range' => [
'online_start' => [
'boost' => 1.4,
'gte' => 'now-14d/d'
]
]
],[ // This is to include all available jobs, at least one should must be true if a must is set
// https://www.elastic.co/guide/en/elasticsearch/guide/current/bool-query.html#_controlling_precision
'range' => [
'online_start' => [
'gte' => 'now-61d/d'
]
]
]
]
]
],
'filter' => [
'bool' => [
// Some term filters
'should' => $filter_should,
'must' => $filter_must,
]
]
]
],
'size' => $perPage,
'from' => $from
Even if you find a search trick to score this particular use-case, you probably want to consider just post-processing the search results to get what you need.
Just loop through the list, keeping a reference to the previous user seen, and if you see the same user in the next result, just remove it from the results, and append it to the end of the list.
Generally speaking you'll get your "shuffled" users as desired, with an occasional pileup of your most prolific user at the very end of the list.
I have this aggregators to deduplicate documents :
$searchParams['body'] = [
'aggs' => [
'dedup' => [
'terms' => [
'field' => 'source',
'size' => 50,
'order' => [
'max_hits' => "desc"
]
],
'aggs' => [
'dedup_hits' => [
'top_hits' => [
'size' => 1
]
],
'max_hits' => [
'max' => [
"script" => "doc.score"
]
]
]
]
]
];
This query order by document score. However, i want to order by _timestamp field. It's possible ? I have test with Date Histogram aggregator. But without success.