Search Console API: How to get full result set? - google-api

everyone. I'm working with the Search Console API. I'm authenticated and getting data -- but not all that I'm hoping for.
The docs say that I can request 5,000 rows at a time. But when I set the setrowLimit parameter like this:
$request->setRowLimit(5000);
I get 127 rows returned, with text at the very bottom of the result set that says 'more elements...' -- almost as if it's a paginated result set.
How do I get to those 'more elements'?
Edit: At the top of my result set, I see this response;
array (size=5000)
So it definitely appears there are 5,000 results in the array, I just don't know how to get them all.

Please, post an example of your code. I set row limit to 5000 bit i don't have any problemi and never see the label "more elements". Where do you get this text? Into the array result?

Related

google search appliance accurate result count parameter not making a difference

We are having a result count issue where the pages have 10 results per page. For pagination we are getting 64 result count on page 1 (ie start=0), 25 for page 2, and 21 for page 3.
I understand as per documentation for estimated vs actual results that it is not guaranteed but the above result count is when I set filter=0 and rc=1. The rc=1 does not appear to make a difference when included or not. We are on version 7.2.0.G.252
filter=0&rc=1 should work for you and you should see the same count even after paginating.
What you need to notice is, when you click on pagination link, make sure the filter=0&rc=1 are carried over. i.e., after pagination, see if you still have the filter and rc parameters intact.
Also check using the default_frontend as your custom frontend may not be handling it?
The problem was related to the collection not the query. The content match pattern did not include a "/" at end which when resolved gave an accurate count. Thanks for the assistance.

How does Soundcloud API with pagination work?

I used the code given via SoundCloud:
This is the error in my terminal results read:
I can't figure out what I did wrong.. :\
This is the result of when I type just "puts tracks"
You can see where the next_href is displayed at the bottom.
If you take a look at the data returned, it might be that it is in a sub-property called "collection", so the correct iterator is "tracks.collection.each"

Yahoo Pipes: Extracting number from feed item for use in URL builder

Been looking all over the place for a solution to this issue. I have a Yahoo Pipe (http://pipes.yahoo.com/pipes/pipe.info?_id=e5420863cfa494ee40e4c9be43f0e812) that I've created to pull back image content from the Bing Search API. The URL builder includes a $skip attribute that takes an integer and uses it to select the starting (index) point for the result set that the query returns.
My initial plan had been to use the math engine in the Wolfram Alpha API to generate a random number (randomInteger[1000]) that I could use to seed the $skip value each time that the pipe is run. I have an earlier version of the pipe where I was able to get the query / result steps working using either "XPath Fetch" and "Fetch Data". However, regardless of how I Fetch the result, the response returns as an attribute / value pair in a list item.Even when I use "Emit items as string" in XPath Fetch, I still get a list with a single item, when what I really want is the integer that I can plug into my $skip attribute.
I've tried everything in Pipes I can think of, and spent a lot of time online looking for an answer. Is there anyway to extract text (in this case, a number) from a single list item and then use the output as input to "wire" a text parameter in another Pipes block? Any suggestions / ideas welcome. In the meantime, I'm generating a sorta-random number by manipulating a timecode hash, but it just feels tacky :-)
Thanks!
All the sources are for repeated items. You can't have a source that just makes a single number.
I'm not really clear what you're trying to do. You want to put a random number into part of the URL string that gets an RSS feed?

Trouble with facet counts

I'm attempting to use ElasticSearch for analytics -- specifically to track "top content" for hand-rolled Rails CMS. The requirement is quite a bit more complicated than keeping a counter for each piece of content. I won't get into the depth of problem right now, as I can't seem to get even the basics working.
My problem is this: I'm using facets and the counts aren't what I expect them to be. For example:
Query:
{"facets":{"el_ids":{"terms":{"field":"el_id","size":1,"all_terms":false,"order":"count"}}}}
Result:
{"el_ids":{"_type":"terms","missing":0,"total":16672,"other":16657,"terms":[{"term":"quis","count":15}]}}
Ok, great, the piece of content with id "quis" had 15 hits and since the order is count, it should be my top piece of content. Now lets get the top 5 pieces of content.
Query:
{"facets":{"el_ids":{"terms":{"field":"el_id","size":5,"all_terms":false,"order":"count"}}}}
Result (just the facet):
[
{"term":"qgz9","count":26},
{"term":"quis","count":15},
{"term":"hnqn","count":15},
{"term":"higp","count":15},
{"term":"csns","count":15}
]
Huh? So the piece of content w/ id "qgz9" had more hits with 26? Why wasn't it the top result in the first query?
Ok, lets get the top 100 now.
Query:
{"facets":{"el_ids":{"terms":{"field":"el_id","size":100,"all_terms":false,"order":"count"}}}}
Results (just the facet):
[
{"term":"qgz9","count":43},
{"term":"difc","count":37},
{"term":"zryp","count":31},
{"term":"u65r","count":31},
{"term":"sxsi","count":31},
...
]
So now "qgz9" has 43 hits instead of 26? How can that be? I can assure you there's nothing happening in the background modifying the index. If I repeat these queries, I get the same results.
As I repeat this process of increasing the result size, counts continue to change and new content ids emerge at the top. Can someone explain to me what I'm doing wrong or where my understanding of how this works is flawed?
It turns out that this is a known issue:
...the way top N facets work now is by getting the top N from each shard, and merging the results. This can give inaccurate results.
By default, my index was being created with 5 shards. By changing this so the index only has a single shard, the counts behave inline with my expectations. Another workaround would be to always set size to a value greater than the number of expected facets and peel off the top N results.

How to change array index to start from 1?

How can I change my array's indices to start from 1 instead of 0. I am trying to fetch news from a site (JSON) and after parsing it:
#news = JSON.parse(Net::HTTP.get(URI.parse('http://api.site.com/news?format=json')))
But to see the individual news title, I have to do #news["items"][0] for the first link's title. Is it possible to change that behavior so when I do #news["items"][1] it shows me the first link's title?
You should intercept user input and adjust entered value to map to a correct array element. In general, you should always validate user input and check if it makes sense.

Resources