I'm trying to run a range query ($gte) and ($lte) using the aggregation of Parse-sdk-php and it's returning me empty
Use case
I have a collection of referrers that have the createdAt key, when searching using the parse greaterThanOrEqualTo() and lessThanOrEqualTo() methods, it is returning results, however when querying using the aggregation, it returns an empty array.
Pipeline
$query = new ParseQuery('referrers);
// $dateStart is an DateTime object
// $dateEnd is an DateTime objectThis text will be hidden
$pipeline = [
'match' => [
'referrer' => ['$exists' => true],
'createdAt' => [
'$gte' => ParseClient::_encode($dateEnd, true),
'$lte' => ParseClient::_encode($dateStart, true)
]
];
$query->aggregate($pipeline);
Query String with decode url
Parse with methods greaterThanOrEqualTo() and lessThanOrEqualTo()
{
"referrer": {$exists: true},
"createdAt":{
"$gte": {__type: "Date", iso: "2020-01-01T17:15:40.000Z"},
"$lte": {__type: "Date", iso: "2020-12-31T17:15:40.000Z"}
}
}
Parse with the aggregation
{
"referrer": {$exists: true},
"createdAt":{
"$gte": {__type: "Date", iso: "2020-01-01T17:15:40.000Z"},
"$lte": {__type: "Date", iso: "2020-12-31T17:15:40.000Z"}
}
}
Settings
OS: WIN 10 20.04
php -v:
PHP 7.4.9
parse php sdk version:
"parse/php-sdk" : "1.6.*",
The following code should work:
'match' => [
'referrer' => [
'$exists' => true
],
'createdAt' => [
'$lte' => ParseClient::_encode($dateEnd, true)['iso'],
'$gte' => ParseClient::_encode($dateStart, true)['iso']
]
]
Related
I'm relatively new to MongoDB and trying to figure out how to search for data which has dynamic keys, my dataset is below:
[
'id' => '60bb040f7e589378f644f9a2',
'name' => 'foo',
'settings' => [
'foo' => [
'type' => 'custom',
'enabled' => 'yes'
],
'bar' => [
'type' => 'custom',
'enabled' => 'no'
],
'baz' => [
'type' => 'custom',
'enabled' => 'no'
]
]
]
I want to get the search inside settings, and get all the settings which has enabled as yes.
I've tried this
return Customer::raw(function($collection) {
return $collection->aggregate([
[
'$search' => [
"wildcard" => [
'query' => 'yes',
'path' => 'settings.*.enabled'
]
]
]
]);
});
I am getting the following errror:
Unrecognized pipeline stage name: '$search'
I've also tired using $match, like this
Customer::raw(function($collection)
{
return $collection->aggregate([
[
'$match' => [
"settings.*.enabled" => 'yes'
]
],
]);
})
But it does't return any value.
Any direction on achieving this will be helpful. Thanks!
Ok this is a bit complex but is all I can come up with myself:
Customer::raw(function($collection)
{
return $collection->aggregate([
[
'$addFields' => [
"settingsArray" => [
'$objectToArray' => '$settings'
]
]
],
[
'$unwind' => [
'path' => '$settingsArray'
]
],
[
'$match' => [
'settingsArray.v.enabled' => 'yes'
]
],
[ '$unset' => 'settingsArray' ],
[
'$group' => [
'_id' => '$_id',
doc => [ '$first' => '$$ROOT' ]
]
],
[
'$replaceRoot' => [
'newRoot' => '$doc'
]
]
]);
});
This should return all documents with at least one enabled setting.
A key point about aggregation is that the pipeline works on all documents that the previous stage of the pipeline returned.
Here's the aggregation pipeline explanation:
$addFields will convert the settings object to an array of format [ { k: key, v: value } ] e.g. [ { k: 'foo', 'v': { type: 'custom', enabled: 'yes' } }, ... ]
$unwind will "unwind" the document and "replace" it with X copies of the same document one for each entry in the newly added settingsArray array. The settingsArray at this stage is not an array anymore but a single entry of type { k: key, v: value }. In short you'd get one document per setting
$match will do the actual matching for any document with an enabled setting (remember now each document will only have a single setting as settingsArray
$unset will remove the settingsArray from each document bringing it back to its original format
$group will group all documents by their _id and use the first result of each group as a doc entry. Each document will now be like { _id: <id>, doc: <original document> }
$replaceRoot will set the root of the document to doc and therefore bring the documents back to their original format.
Would love to hear if anyone else has a better solution here.
Below is the code inside my CategoryResource.php
[
'id' => $this->id,
'name' => $this->name,
'categories' => CategoryResource::collection($this->children),
'items' => ItemListResource::collection($this->items)
];
and by this i am getting response like:
{
"status": true,
"data": {
"categories": [
{
"id": 10,
"name": "Fast Food",
"categories": [
{
"id": 11,
"name": "Pizza",
"categories": [],
"items": [
{
"id": 9,
"name": "Ham \u0026 Crab Stick",
"price": "20.00",
"image": "up/products/c11/Mrn4OpjngYRCkRJ28rJ1LbXUOXy0XjTRyXU7eFoi.jpg"
}
]
}
],
"items": []
}
]
}
}
i dont want "categories" : [] when category does not exist for a category.
please suggest some ideas or ways i can deal with this issue.
I assume that CategoryResource::collection($this->children) returns an array, maybe empty or not.
If that's true, then:
$dataArray = [
'id' => $this->id,
'name' => $this->name,
'items' => ItemListResource::collection($this->items)
];
$categories = CategoryResource::collection($this->children);
if (count(categories) > 0) $dataArray['categories'] = $categories;
I have a fairly complex mapping which is storing products, and within each document it contains a nested array of pre-calculated prices for each customer.
There may be multiple versions of each product in the index (with unique codes). Alternative products are grouped by a common xrefs_hash. The query I'm writing needs to select the best product for each customer (i.e. aggregate/collapse on the xrefs_hash), and then select the top product based on the value of the prices.weight nested field.
The prices.weight field is a float which we've pre-calculated based on the shops' customer settings on how they want to prioritise their own items. A hash is created from these settings (stored in prices.pricing_hash) so that we can store a single set of pricing if multiple customers share the same settings.
The index contains up to 300,000 products and can end up with ~100,000,000 documents once all prices are calculated and inserted.
The mapping looks something like this (shortened for brevity):
'mappings' => [
'_source' => [
'enabled' => true,
],
'dynamic' => false,
'properties' => [
'dealer_item_id' => [
'type' => 'integer',
],
'code' => [
'type' => 'text',
'analyzer' => 'custom_code_analyzer',
'fields' => [
'raw' => [
'type' => 'keyword',
],
],
],
'xrefs' => [
'type' => 'text',
'analyzer' => 'custom_code_analyzer',
'fields' => [
'raw' => [
'type' => 'keyword',
],
],
],
'xrefs_hash' => [
'type' => 'keyword',
],
'title' => [
'type' => 'text',
'analyzer' => 'custom_english_analyzer',
'fields' => [
'ngram_title' => [
'type' => 'text',
'analyzer' => 'custom_title_analyzer',
],
'raw' => [
'type' => 'keyword',
],
],
],
...
'prices' => [
'type' => 'nested',
'dynamic' => false,
'properties' => [
'pricing_hash' => [
'type' => 'keyword',
'index' => true,
],
'unit_price' => [
'type' => 'float',
'index' => true,
],
'pricebreaks' => [
'type' => 'object',
'dynamic' => false,
'properties' => [
'quantity' => [
'type' => 'integer',
'index' => false,
],
'price' => [
'type' => 'integer',
'index' => false,
],
],
],
'weight' => [
'type' => 'float',
'index' => true,
],
],
],
],
],
Example documents:
{
"dealer_item_id": 122023,
"code": "ABC123A",
"xrefs": [
"ABC123A",
"ABC123B",
],
"title": "Product A",
"xrefs_hash": "16d5415674c8365f63329b11ffc88da109590cec",
"prices": [
{
"pricebreaks": [
{
"quantity": 1,
"price": 9.75,
"contract": false
}
],
"weight": 0.20512820512820512,
"pricing_hash": "aabe06b7",
"unit_price": 9.75,
},
{
"pricebreaks": [
{
"quantity": 1,
"price": 9.75,
"contract": false
}
],
"weight": 0.20512820512820512,
"pricing_hash": "73643f3b",
"unit_price": 9.75,
}
]
},
{
"dealer_item_id": 124293,
"code": "ABC1234B",
"xrefs": [
"ABC123A",
"ABC123B",
],
"title": "Product B",
"xrefs_hash": "16d5415674c8365f63329b11ffc88da109590cec",
"prices": [
{
"contract_item": false,
"pricebreaks": [
{
"quantity": 1,
"price": 7.39,
"contract": false
}
],
"weight": 0.33829499323410017,
"pricing_hash": "aabe06b7",
"unit_price": 7.39,
},
{
"pricebreaks": [
{
"quantity": 1,
"price": 9.75,
"contract": false
}
],
"weight": 0.20512820512820512,
"pricing_hash": "73643f3b",
"unit_price": 9.75,
}
]
},
Example query:
{
"track_total_hits": 100000,
"query": {
"bool": {
"filter": {
"bool": {
"must": [
{
"nested": {
"path": "prices",
"score_mode": "none",
"inner_hits": {
"_source": {
"include": [
"prices"
]
}
},
"query": {
"bool": {
"must": [
{
"term": {
"prices.pricing_hash": "aabe06b7"
}
}
]
}
}
}
},
{
"term": {
"code.raw": "RX58022"
}
}
],
"must_not": [
{
"term": {
"disabled": true
}
}
]
}
}
}
},
"_source": {
"includes": [
"code",
"dealer_item_id",
"title",
"xrefs"
]
},
"collapse": {
"field": "xrefs_hash",
"inner_hits": {
"name": "best_xrefs",
"sort": {
"prices.weight": "desc"
},
"size": 1
}
},
"aggregations": {
"xrefs_count": {
"cardinality": {
"field": "xrefs_hash",
"precision_threshold": 40000
}
}
}
}
I have tried using a collapse query to select the best product, but this does not seem to support sorting by the nested prices.weight field.
I've also tried aggretating based on the xrefs_hash, but this seems to make pagination at the category level impossible.
The above example query almost works, but does not return the collapsed results in the correct order. When inspecting the query it seems to be replacing the collapse sort with Infinity, which apparently ES does if a document does not contain a sort field.
So what I'm wondering is; is it possible to:
Return 1 document per unique xref_hash value
Return the specific document whith the highest prices.weight value, matching customer's pricing_hash
Also make this work with pagination
Below I have the results from google analytics API. This is the output of the query I am running from developers.google.com.
"start-date": "2015-04-01",
"end-date": "today",
"dimensions": "ga:source",
"metrics": [
"ga:users"
],
"sort": [
"-ga:users"
],
{
"name": "ga:source",
"columnType": "DIMENSION",
"dataType": "STRING"
},
{
"name": "ga:users",
"columnType": "METRIC",
"dataType": "INTEGER"
}
],
"totalsForAllResults": {
"ga:users": "2201795"
},
"rows": [
[
"(direct)",
"869648"
],
[
"sa",
"591843"
],
What I am trying to do in my code (ruby) is pull out the information from "rows". I need both the referral name "sa" and the user count "591843".
Here is how I am attempting to do this.
sa_referral = client.execute(:api_method => analytics.data.ga.get, :parameters => {
'ids' => "ga:" + saprofileID,
'dimensions' => "ga:fullreferrer",
'metrics' => "ga:users",
'sort' => "-ga:users",
'filters' => "ga:ga:source!=us-mg6.mail.yahoo.com;ga:source!=sa;ga:source!=(direct);ga:source!=mail.google.com;ga:source!=us-mg5.mail.yahoo.com;ga:source!=SA;ga:source!=m.mg.mail.yahoo.com;ga:source!=mail.aol.com;ga:source!=us-mg4.mail.yahoo.com;ga:source!=yahoo;ga:source!=bing;ga:source!=google;ga:source!=weibo;ga:fullreferrer!=eccie.net/showthread.php",
'start-date' => startDate,
'end-date' => endDate,
})
sa_referral_data = sa_referral.rows do |row|
rows.map{|r| { referrals:r[0], members:r[1] }}
end
puts sa_referral_data
The result of puts sa_referral_data is the following
scheduler caught exception:
undefined method `rows' for #<Google::APIClient::Result:0x000000044651f0>
It seems like the API is not getting the information from rows at all. Is there a way I can troubleshoot the API to see why it is losing the data for the "rows" section?
instead of sa_referral.rows, use sa_referral.each
Example code from my project..... It uses the same method to query google and return results.
result = client.execute(
:api_method => api.users.list,
:parameters => { 'domain' => domain,
'orderBy' => 'givenName',
'maxResults' => 200,
'pageToken'=> pageToken,
#'fields' => 'users(id,etag,primaryEmail,name,suspended)',
'fields' => 'nextPageToken,users(primaryEmail,suspended)',
'query' => "orgUnitPath=#{orgUnitPath}"
}
)
#pp result # There should be results here
#pp result.methods #Displays what commands you can use on the object
returnabledata = Array.new;
result.data.users.each do |user|
if user["suspended"] == false
returnabledata << user["primaryEmail"]
end
end
i have the following query
$queryDefinition = [
'query' => [
'bool' => [
'must' => [
[
'query_string' => [
'default_field' => '_all',
'query' => $term
]
]
],
'must_not' => [],
'should' => []
],
],
//'filter' => [
// 'limit' => ['value' => 3],
//],
'from' => 0,
'size' => 50,
'sort' => [],
'facets' => [
'types' => [
'terms' => ['field' => '_type']
]
]
];
in the index we have 5 types, and i would like to show only 3 results from each type, for autocomplete. when i set the filter limit only the results from the first type are filtered, for other types i get all the results.
how can i do this?
thanks
{
"aggregations": {
"types": {
"terms": {
"field": "_type"
},
"hits": {
"top_hits": {
"size": 3
}
}
}
}
}
If you're using ElasticSearch 1.4.0, you can use terms aggregation on "_type" field and use top hits aggregation as sub aggregation and set its size to 3 (in your case).
Hope that helps.