I've tried to search videos using YouTube Data API v3 (https://developers.google.com/youtube/v3/docs/search/list).
I successfully got some information, but I figured out the result of pageInfo wasn't something I expected.
Here is the query and the response I got:
https://www.googleapis.com/youtube/v3/search?part=id&channelId=UC5fslQUCCo3OE1EaA1n5ZmQ&key=...&publishedAfter=2019-01-15T08:09:38Z&publishedBefore=2019-01-15T10:09:38Z&order=date .
{
"kind": "youtube#searchListResponse",
"etag": "iLvO2LgGDksqADNU-_xPwG4QMCU",
"nextPageToken": "CAUQAA",
"regionCode": "JP",
"pageInfo": {
"totalResults": 49,
"resultsPerPage": 5
},
"items": [
{
"kind": "youtube#searchResult",
"etag": "Adtll-9lbQDiKGJEkBb2IEjPfjw",
"id": {
"kind": "youtube#video",
"videoId": "d4Z7MZo3-Ac"
}
}
]
}
I specified date strings to publishAfter and publishBefore so that the result include single video information only.
I guessed both pageInfo.totalResult and pageInfo.resultPerPage were 1.
Contrary to my anticipation, pageInfo.totalResult was 49 and pageInfo.resultsPerPage was 5.
What describes pageInfo? Is my understanding incorrect?
Here are the answers to your two questions:
1. Why pageInfo.resultsPerPage is 5?
The specification of this property is the following:
pageInfo.resultsPerPage (integer)
The number of results included in the API response.
One may read this specification as (am rephrasing the text above): the number of items in the result set obtained from the API, i.e the number of elements of the JSON array items.
Unfortunately, this does not correspond to reality. In my experience, pageInfo.resultsPerPage is simply the value of the request parameter maxResults:
maxResults (unsigned integer)
The maxResults parameter specifies the maximum number of items that should be returned in the result set. Acceptable values are 0 to 50, inclusive. The default value is 5.
Since your API invoking URL above does not specify maxResults, this parameter is taken as having the default value; hence pageInfo.resultsPerPage gets to be 5.
One more remark: indeed the official specification of this property is misleading; this issue may well be reported to Google (via its own issue tracker site) as a documentation bug.
2. Why pageInfo.totalResults is 49 and is not 1?
The specification of this property is the following (the emphasis below is mine):
pageInfo.totalResults (integer)
The total number of results in the result set. Please note that the value is an approximation and may not represent an exact value. In addition, the maximum value is 1,000,000.
You should not use this value to create pagination links. Instead, use the nextPageToken and prevPageToken property values to determine whether to show pagination links.
According to this specification, pageInfo.totalResults is not to be taken as a precise quantity that the user can rely upon. This peculiarity is also confirmed officially by Google's staff.
Moreover, if experimenting a bit with your URL, eliminating the two date request parameters publishedAfter and publishedBefore, you'll see that the value of pageInfo.totalResults becomes unstable: repeating the same API call several times in a row (only seconds apart one from another), I never got the same value of out of the API, but, alternating, 1374 and 1375.
Related
Is it possible to use wild card in Kibana visualize search bar.
Tried to use it like below, but did not work.
operation: "Revers" NOT file:"*Test.Revers"
This returns 2 because there are two Revers terms ("Revers", "/test/count/Test.Revers" ) even though only one data entry is in the stats data.
The following also returns the same value as 2.
operation: "Revers"
Stat data sample is as below.
"_source": {
"status": 0,
"trstime": 1819,
"username": "test",
"operation": "Revers",
"file": "/test/count/Test.Revers"
}
I have tested it in ES 7.10 as you not mentioned ES version.
Answer to your question is YES, you can use wildcrad in Kibana visualize search bar but value should be without double quotes. Because if you give value in doble quotes it will consider as text and search for it.
You can try below query and it will give you your expected output:
operation: Revers AND NOT file.keyword: *Test.Revers
The result given for the below query as 1 without double quotes.
operation: Revers AND NOT file: *Test.Revers
I have a list of a few thousand locations for which I need to find the coordinates. I'm using Google Geocode API for this.
The information I have is something like "USA, California, Napa Valley"
So my Geocode call would look something like this:
https://maps.googleapis.com/maps/api/geocode/json?address=Napa+Valley,California,USA&key=<apikey>
(I put them in reverse order and replace spaces with "+" characters)
I'm of course mostly interested in the values found under ['results'][0]['geometry']['location'] key (where latitude and longitude will be found).
Now in a lot of cases I get very different results by simply waiting a few hours or 1 day and making the call again. And I make those calls using the exact same data, same parameters, same everything
Usually the coordinates are different by a fraction of a degree or at most 1.0 - 1.5 degrees. For example 39.8 latitude yesterday becomes 40.4 latitude today. And then it becomes 40.1. And then a day after it comes back to 39.8. And then it's 40.25. And so on...
However in some extreme cases the differences are massive.
Actual concrete example:
https://maps.googleapis.com/maps/api/geocode/json?address=Côtes+du+Rhône-Brézème,Northern+Rhône,Rhône,France&key=<apikey>
The result will be something like:
{
"results": [
{
.....
"formatted_address": "Rhône",
"geometry": {
.........
"location": {
"lat": 44.952398,
"lng": 6.4840823
}
},
"partial_match": true,
......
}
],
"status": "OK"
}
So it's somewhat correctly identified as the Rhone region in France.
However if I wait just a few minutes and make the call again, I get this:
{
"results" : [
{
......
"formatted_address" : "148 Chambers St A, New York, NY 10007, USA",
"geometry" : {
"location" : {
"lat" : 40.715666,
"lng" : -74.009906
},
......
},
.....
}
],
"status" : "OK"
}
Since when is Rhone/France even close to New-York ?
And this is the annoying part: I don't change anything about the URL. I simply make the EXACT same call a few minutes apart.
My thinking was that it was caused by me not properly encoding the URL.
However, while this may be true, I found it happens for queries that use NO special characters at all, for example something like Clarksburg,California,USA. In this case though I get 2 different results (coordinates), both in California, about 200 miles from each other.
Why does this happen? I need consistent results by this API. I don't need extreme precision or something like that. But at least some strong consistency.
I can't just go through a few thousand locations and put them manually in Google-Maps website to find their coordinates. It would take me 1 week of non-stop work...
This looks like a bug that would be best reported in the Google Maps APIs public issue tracker at https://issuetracker.google.com/code/p/gmaps-api-issues/
I'm practicing d3 with Mike Bostok's example: https://bl.ocks.org/mbostock/3884955
I'd like to see how the series that he made - cities - looks like.
I opened the html file using Python local server and in the console, I wrote 'console.log(cities);' but it didn't work. 'console.log(data);' also didn't work. They all showed this error message:
VM100:1 Uncaught ReferenceError: cities is not defined
at <anonymous>:1:13
I didn't make any modification in his code. So I don't think there is an error in the code. I assume perhaps the problem is in line with d3 setting?
So, I've tried to open the file in Firefox and I've also downloaded d3 but those two ways also didn't work.
Does anyone happen to know what is the cause of the problem?
If someone can explain how the 'cities' in his code looks like, then you are the most welcome!
Thanks a lot,
At a guess, the reason cities is undefined is because your console.log statement is somewhere outside of the callback function provided to d3.tsv. If you look closely at Mike's code, you'll notice that the third argument to d3.tsv is a function that receives as an argument an error object and the processed data. Inside that function, he defines the cities variable, so if you put console.log(cities) anywhere outside of that function, cities will be undefined.
Now, on to the format of the data. If you look further down that block, there's another file: data.tsv. It has four columns: date, New York, San Francisco, Austin. d3.tsv will create an array where each element in the array corresponds to one row in the TSV (except for the header row). Each row is converted to a plain JavaScript Object with properties that correspond to the columns of the file. That array is passed into the callback as the data variable in this block. So data[0] will be
{
"date": "20111001",
"New York": "63.4",
"San Francisco": "62.7",
"Austin": "72.2"
}
When the cities variable is defined, that array is transformed into an array that contains one item per city, and each object representing the city contains the time-series data of temperature for that city. So the cities variable will look like:
[
{
"id": "New York",
"values": [
{"date": Date 2011-10-01T00:00:00, "temperature": 63.4},
{"date": Date 2011-10-02T00:00:00, "temperature": 48.0},
...
]
},
{
"id": "San Francisco",
"values": [...]
},
{
"id": "Austin",
"values": [...]
}
]
It's probably worth pointing here the second argument to d3.tsv in this example: the type function. This is where all the strings are converted into Date or Number objects. Without this argument, all of the property values in data would be strings.
For more information on how d3.tsv works, you can check the docs for d3-request.
CAVEAT: that block is for d3v4, the latest version of d3 is v5. In v5, d3-request is deprecated in favor of d3-fetch. d3-fetch provides pretty much the same set of utilities for fetching data, but instead of using callback functions, they return Promises.
How can I get the optional result data values from the Steam Web API?
For example, from GetPlayerAchievements I also want the name and description.
So far I use this URL: http://api.steampowered.com/ISteamUserStats/GetPlayerAchievements/v0001/?appid=MYAPPID&key=MYKEY&steamid=MYSTEAMID
You need to add &l=en to the end of your URL. This adds the language dependent fields of name and description to the results.
en can be substituted for other supported languages
Example for a player's Team Fortress 2 stats:
URL: http://api.steampowered.com/ISteamUserStats/GetPlayerAchievements/v0001/?appid=MYAPPID&key=MYKEY&steamid=MYSTEAMID&l=en
Returns a JSON result that has this block in the ['playerstats']['achievements'] array
{
"apiname": "TF_PLAY_GAME_EVERYCLASS",
"achieved": 1,
"name": "Head of the Class",
"description": "Play a complete round with every class."
}
While creating a document that is got from web interface, it does not rightly typecast integer date and other type. for example
{"_id": "<some_object_id>", "name": "hello", "age": "20", "dob": "1994-02-22"}
Since attributes are entered dynamically, their types can not be prejudged. Is there any way I can get them entered from client side, like
{"_id": "<some_object_id>", "name": "hello", "age": "$int:20", "dob": "$date:1994-02-22"}
Any help is highly appreciated.
Since you appear to be concerned about the strings that come in from a form such as a POST, the simple answer is that you cast them in Ruby.
If the field is what you expect to be a number then cast it to an int. And the same goes for dates as well.
Your mongo driver will correctly interpret these and store them as the corresponding BSON types in your MongoDB collection. The same goes in reverse, when you read collection data you will get it back cast into your native types.
"1234".to_i
Date.strptime("{ 2014, 2, 22 }", "{ %Y, %m, %d }")
But that's be basic Ruby part.
Now you could do something like you pseudo-suggested and store your information, not as native types but as strings with some form of type tagging. But see, I just don't see the point as you would have to
Detect the type at some stage and apply the tag
Live with the fact that you just ruined all the benefits of having the native types in the collection. Such as query and aggregation for date ranges and basic summing of values.
And while we seem to be going down the track of the anything type where users just arbitrarily insert data and something else has to work out what type it is, consider the following examples of MongoDB documents:
{
name: "Fred",
values: [ 1, 2, 3, 4],
}
{
name: "Sally",
values: "a"
}
So in Mongo terminology, that document structure is considered bad. Even though Mongo does have a flexible schema concept, this type of mixing will break things. So don't do it, but rather handle in the following way, which is quite acceptable even though the schema's are different:
{
name: "Fred",
values: [ 1, 2, 3, 4],
}
{
name: "Sally",
mystring: "a"
}
The long story short, Your application should be aware of the types of data that are coming in. If you allow user defined forms then your app needs to be able to attach a type to them. If you have a field that could be a string or a Date, then your app need to determine which type it is, and cast it, or otherwise store it correctly.
As it stands you will benefit from re-considering you use case, rather than waiting for something else to work all that out for you.