Below I have the results from google analytics API. This is the output of the query I am running from developers.google.com.
"start-date": "2015-04-01",
"end-date": "today",
"dimensions": "ga:source",
"metrics": [
"ga:users"
],
"sort": [
"-ga:users"
],
{
"name": "ga:source",
"columnType": "DIMENSION",
"dataType": "STRING"
},
{
"name": "ga:users",
"columnType": "METRIC",
"dataType": "INTEGER"
}
],
"totalsForAllResults": {
"ga:users": "2201795"
},
"rows": [
[
"(direct)",
"869648"
],
[
"sa",
"591843"
],
What I am trying to do in my code (ruby) is pull out the information from "rows". I need both the referral name "sa" and the user count "591843".
Here is how I am attempting to do this.
sa_referral = client.execute(:api_method => analytics.data.ga.get, :parameters => {
'ids' => "ga:" + saprofileID,
'dimensions' => "ga:fullreferrer",
'metrics' => "ga:users",
'sort' => "-ga:users",
'filters' => "ga:ga:source!=us-mg6.mail.yahoo.com;ga:source!=sa;ga:source!=(direct);ga:source!=mail.google.com;ga:source!=us-mg5.mail.yahoo.com;ga:source!=SA;ga:source!=m.mg.mail.yahoo.com;ga:source!=mail.aol.com;ga:source!=us-mg4.mail.yahoo.com;ga:source!=yahoo;ga:source!=bing;ga:source!=google;ga:source!=weibo;ga:fullreferrer!=eccie.net/showthread.php",
'start-date' => startDate,
'end-date' => endDate,
})
sa_referral_data = sa_referral.rows do |row|
rows.map{|r| { referrals:r[0], members:r[1] }}
end
puts sa_referral_data
The result of puts sa_referral_data is the following
scheduler caught exception:
undefined method `rows' for #<Google::APIClient::Result:0x000000044651f0>
It seems like the API is not getting the information from rows at all. Is there a way I can troubleshoot the API to see why it is losing the data for the "rows" section?
instead of sa_referral.rows, use sa_referral.each
Example code from my project..... It uses the same method to query google and return results.
result = client.execute(
:api_method => api.users.list,
:parameters => { 'domain' => domain,
'orderBy' => 'givenName',
'maxResults' => 200,
'pageToken'=> pageToken,
#'fields' => 'users(id,etag,primaryEmail,name,suspended)',
'fields' => 'nextPageToken,users(primaryEmail,suspended)',
'query' => "orgUnitPath=#{orgUnitPath}"
}
)
#pp result # There should be results here
#pp result.methods #Displays what commands you can use on the object
returnabledata = Array.new;
result.data.users.each do |user|
if user["suspended"] == false
returnabledata << user["primaryEmail"]
end
end
Related
I'm trying to run a range query ($gte) and ($lte) using the aggregation of Parse-sdk-php and it's returning me empty
Use case
I have a collection of referrers that have the createdAt key, when searching using the parse greaterThanOrEqualTo() and lessThanOrEqualTo() methods, it is returning results, however when querying using the aggregation, it returns an empty array.
Pipeline
$query = new ParseQuery('referrers);
// $dateStart is an DateTime object
// $dateEnd is an DateTime objectThis text will be hidden
$pipeline = [
'match' => [
'referrer' => ['$exists' => true],
'createdAt' => [
'$gte' => ParseClient::_encode($dateEnd, true),
'$lte' => ParseClient::_encode($dateStart, true)
]
];
$query->aggregate($pipeline);
Query String with decode url
Parse with methods greaterThanOrEqualTo() and lessThanOrEqualTo()
{
"referrer": {$exists: true},
"createdAt":{
"$gte": {__type: "Date", iso: "2020-01-01T17:15:40.000Z"},
"$lte": {__type: "Date", iso: "2020-12-31T17:15:40.000Z"}
}
}
Parse with the aggregation
{
"referrer": {$exists: true},
"createdAt":{
"$gte": {__type: "Date", iso: "2020-01-01T17:15:40.000Z"},
"$lte": {__type: "Date", iso: "2020-12-31T17:15:40.000Z"}
}
}
Settings
OS: WIN 10 20.04
php -v:
PHP 7.4.9
parse php sdk version:
"parse/php-sdk" : "1.6.*",
The following code should work:
'match' => [
'referrer' => [
'$exists' => true
],
'createdAt' => [
'$lte' => ParseClient::_encode($dateEnd, true)['iso'],
'$gte' => ParseClient::_encode($dateStart, true)['iso']
]
]
I have been migrating some of the indexes from self-hosted Elasticsearch to AmazonElasticSearch using Logstash. While migrating the documents, We need to change the field names in the index based on some logic.
Our Logstash Config file
input {
elasticsearch {
hosts => ["https://staing-example.com:443"]
user => "userName"
password => "password"
index => "testingindex"
size => 100
scroll => "1m"
}
}
filter {
}
output {
amazon_es {
hosts => ["https://example.us-east-1.es.amazonaws.com:443"]
region => "us-east-1"
aws_access_key_id => "access_key_id"
aws_secret_access_key => "access_key_id"
index => "testingindex"
}
stdout{
codec => rubydebug
}
}
Here it is one of the documents for the testingIndex from our self-hosted elastic search
{
"uniqueIdentifier" => "e32d331b-ce5f-45c8-beca-b729707fca48",
"createdDate" => 1527592562743,
"interactionInfo" => [
{
"value" => "Hello this is testing",
"title" => "msg",
"interactionInfoId" => "8c091cb9-e51b-42f2-acad-79ad1fe685d8"
},
{
**"value"** => """"{"edited":false,"imgSrc":"asdfadf/soruce","cont":"Collaborated in <b class=\"mention\" gid=\"4UIZjuFzMXiu2Ege6cF3R4q8dwaKb9pE\">#2222222</b> ","chatMessageObjStr":"Btester has quoted your feed","userLogin":"test.comal#google.co","userId":"tester123"}"""",
"title" => "msgMeta",
"interactionInfoId" => "f6c7203b-2bde-4cc9-a85e-08567f082af3"
}
],
"componentId" => "compId",
"status" => [
"delivered"
]
},
"accountId" => "test123",
"applicationId" => "appId"
}
This is what we are expecting when documents get migrated to our AmazonElasticSearch
{
"uniqueIdentifier" => "e32d331b-ce5f-45c8-beca-b729707fca48",
"createdDate" => 1527592562743,
"interactionInfo" => [
{
"value" => "Hello this is testing",
"title" => "msg",
"interactionInfoId" => "8c091cb9-e51b-42f2-acad-79ad1fe685d8"
},
{
**"value-keyword"** => """"{"edited":false,"imgSrc":"asdfadf/soruce","cont":"Collaborated in <b class=\"mention\" gid=\"4UIZjuFzMXiu2Ege6cF3R4q8dwaKb9pE\">#2222222</b> ","chatMessageObjStr":"Btester has quoted your feed","userLogin":"test.comal#google.co","userId":"tester123"}"""",
"title" => "msgMeta",
"interactionInfoId" => "f6c7203b-2bde-4cc9-a85e-08567f082af3"
}
],
"componentId" => "compId",
"status" => [
"delivered"
]
},
"accountId" => "test123",
"applicationId" => "appId"
}
What we need is to change the "value" field to "value-keyword" wherever we find some JSON format. Is there any other filter in Logstash to achieve this
As documented in the Logstash website:
https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-rename
You can use the mutate filter, applying the rename function.
For example:
filter {
mutate {
replace => { "old-field" => "new-field" }
}
}
For nested fields, you could just pass the path of the field:
filter {
mutate {
replace => { "[interactionInfo][value]" => "[interactionInfo][value-keyword]" }
}
}
Try adding this to your filter:
filter {
ruby {
code => "event.get('interactionInfo').each { |item| if item['value'].match(/{.+}/) then item['value-keyword'] = item.delete('value') end }"
}
}
I'm trying to achieve the following JSON results:
{
"movie" =>
[{
"title": "Thor",
"year" : 2011,
},
{
"title": "Iron Man",
"year" : 2008,
}],
"tv" =>
[{
"title": "Parks and Recreation"
"year": 2009
},
{
"title": "Friends"
"year": 1994
}]
}
With JavaScript, I would loop through my results and do something like:
results['movie'].push(item);
results['tv'].push(item);
With Ruby code, the farthest I've gone is this:
#results = Hash.new
results['Search'].each do |r|
if r['Type'] == 'movie'
#results['movie'] << {
'title' => r['Title'],
'year' => r['Year']
}
elsif r['Type'] == 'series'
#results['tv'] << {
'title' => r['Title'],
'year' => r['Year']
}
end
end
What am I missing here?
I think you can get what you want by using Enumerable#each_with_object and assigning a default value to the hash.
def group_search_results(items)
results = Hash.new { |hash, key| hash[key] = [] }
items.each_with_object(results) do |item|
results[item['Type']] << {'title' => item['Title'], 'year' => item['Year']}
end
end
describe "search_results" do
it "groups into an object" do
items = [
{'Type' => 'movie', 'Title' => 'Thor', 'Year' => 2011},
{'Type' => 'movie', 'Title' => 'Iron Man', 'Year' => 2008},
{'Type' => 'series', 'Title' => 'Parks and Recreation', 'Year' => 2009},
{'Type' => 'series', 'Title' => 'Friends', 'Year' => 1994},
]
results = group_search_results(items)
expect(results).to eq({
'movie' => [
{'title' => 'Thor', 'year' => 2011},
{'title' => 'Iron Man', 'year' => 2008},
],
'series' => [
{'title' => 'Parks and Recreation', 'year' => 2009},
{'title' => 'Friends', 'year' => 1994},
],
})
end
end
I believe the problem has to do with the initialization of your hash. The movie and tv keys aren't currently an array. You can initialize your hash like this:
#results = { 'movie' => [], 'tv' => [] }
Here's how it looks with the rest of your code:
#results = { 'movie' => [], 'tv' => [] }
results['Search'].each do |r|
if r['Type'] == 'movie'
#results['movie'] << {
'title' => r['Title'],
'year' => r['Year']
}
elsif r['Type'] == 'series'
#results['tv'] << {
'title' => r['Title'],
'year' => r['Year']
}
end
end
results = {
search: {
movie: [
{ title: 'Thor', year: 2011 },
{ title: 'Iron Man', year: 2008 },
],
tv: [
{ title: 'Parks and Recreation', year: 2009 },
{ title: 'Friends', year: 1994 },
]
}
}
#results = Hash.new{|k, v| k[v] = []}
results[:search].each do |type, array|
#results[type].push(*array)
end
results[:search].each_with_object(Hash.new{|k, v| k[v] = []}) do |(type, array), hash|
hash[type].push(*array)
end
In my use case, I would like to add an OR condition in Elasticsearch query. Here is my query,
query_body = {
'query' => {
'bool' => {
'must' => [{ 'range' => {'#timestamp' => { 'from' => stream_filters[:first_time].gmtime.strftime("%Y-%m-%dT%H:%M:%SZ"), 'to' => stream_filters[:second_time].gmtime.strftime("%Y-%m-%dT%H:%M:%SZ") } } }, {'term' => {"#timeout" => true} }, {'term' => {"#dest" => dest} }, {'term' => {"#source" => source} } ]
}
}, 'facets' => facets
}
I would like to add 'term' => {"#dest" => ' '} empty check for #dest along with 'term' => {"#dest" => dest}
I tried to add an or condition, But it is not working.
query_body = {
'query' => {
'bool' => {
'must' => [{ 'range' => {'#timestamp' => { 'from' => stream_filters[:first_time].gmtime.strftime("%Y-%m-%dT%H:%M:%SZ"), 'to' => stream_filters[:second_time].gmtime.strftime("%Y-%m-%dT%H:%M:%SZ") } } }, {'term' => {"#timeout" => true} }, {'term' => {"#source" => source} } ],
'filter' => {
'or' => [{
'term' => { "#dest" => dest }
'term' => { "#dest" => ' ' }
}]
}
}
}, 'facets' => facets
}
Could someone help me with this?
It seems like a syntax error in your filter clause. Please try with the correct syntax as below :
'filter' => {
'or' => [
{
'term' => { "#dest" => dest }
},
{
'term' => { "#dest" => ' ' }
}
]
}
i have the following query
$queryDefinition = [
'query' => [
'bool' => [
'must' => [
[
'query_string' => [
'default_field' => '_all',
'query' => $term
]
]
],
'must_not' => [],
'should' => []
],
],
//'filter' => [
// 'limit' => ['value' => 3],
//],
'from' => 0,
'size' => 50,
'sort' => [],
'facets' => [
'types' => [
'terms' => ['field' => '_type']
]
]
];
in the index we have 5 types, and i would like to show only 3 results from each type, for autocomplete. when i set the filter limit only the results from the first type are filtered, for other types i get all the results.
how can i do this?
thanks
{
"aggregations": {
"types": {
"terms": {
"field": "_type"
},
"hits": {
"top_hits": {
"size": 3
}
}
}
}
}
If you're using ElasticSearch 1.4.0, you can use terms aggregation on "_type" field and use top hits aggregation as sub aggregation and set its size to 3 (in your case).
Hope that helps.