I need to be able to guesstimate an area's population density.
For example, if I selected Time's Square, I need to get the rough population density in a 1KM radius.
I know the Places API does not have a specific function for this, but I think something like this could work:
Fetch the count of all the businesses or premises in an area, and compare them to known results. For example, if central Mumbai has a businesses/premises count of 1000, and a rural town area has a businesses/premises count of 10, then it would be fair to say low density is probably < 100, medium density is probably 100 - 700, and high density is over 700. Or something along those lines.
So is there a way to fetch the count of businesses or premises in an area using Google Places API?
This functionality is not currently supported by the Google Maps Platform -- Places API.
If you are interested to have this feature, you can file this as feature request in Google's Public Issue Tracker:
https://developers.google.com/maps/support/#issue_tracker
Issue Tracker is a tool used internally at Google to track bugs and feature requests during product development.
Related
I have retrieved "Visually Similar Images" using Google Vision API. I would like to know how given a photo (that could pertain to a blog or article), Google Vision API finds a list of visually similar images? I cannot seem to find a white paper describing this.
Additionally, I would like to know if it makes sense to consider these visually similar images if the labels predicted by Google Vision API have a score lower than 70% confidence?
According to the documentation, Google Cloud’s Vision API offers powerful pre-trained machine learning models through REST and RPC APIs, like the Web Detection which are in charge of processing and analyzing the images received in order to identify other images with characteristics similar to the original, as is mentioned here; however, since it is a pre-trained model of Google there isn’t a public documentation of its development.
Regarding your question about considering a confidence score lower than 70%, it completely depends on your use-case, you have to evaluate the acceptance limits required in order to satisfy your requirements.
Please note that the object returned in the "visuallySimilarImages" field of the JSON response is a WebImage object and its score field is deprecated, you may be referring to the score within the WebEntity object that is an overall relevancy score for the entity. Not normalized and not comparable across different image queries.
Google changed its PSI definition of fast-ranking FCP from 90-percentile to 75-percentile below 1000ms
From PSI documentation:
Why does the FCP in v4 and v5 have different values?
FCP in v5 reports the 75th percentile (as of November 4th 2019),
previously it was the 90th percentile. In v4, FCP reports the median
(50th percentile).
Good data/tips in top answer from Rick below still.
ORIGINAL QUESTION:
Does using the 90-percentile instead of previous median score, or a lower percentile, when saying, "based on field data the 'page is slow'" make it impossible for heavily trafficked websites, such as google.com, from ever getting ranked "Fast"? This due to the long tail that occurs when monthly traffic is in the 10M+ ranges and globally distributed?
Last time I checked (early Feb. 2018), the Desktop google.com received a 100 Lighthouse synthetic score, which is supposed to be interpreted as "there is little room for improvement," and yet, the page is ranked "slow" because the 90th percentile FCP is way over 3s.
Will a page like nytimes.com ever be considered fast with this standard, when even google.com's desktop page is ranked slow based on field data?
Recent example (Feb. 14, 2019)
Former example with even longer tail for FCP:
To directly answer the question, no it's not impossible to get a fast FCP label. There's more to the question so I'll try to elaborate.
Another way to phrase the "fast" criteria is: "Do at least 90% of user experiences have an FCP less than 1 second?"
Why 90%? Because it's inclusive of a huge proportion of user experiences. As the PSI docs say:
Our goal is to make sure that pages work well for the majority of users. By focusing on 90th and 95th percentile values for our metrics, this ensures that pages meet a minimum standard of performance under the most difficult device and network conditions.
Why 1 second? It's a subjective value for how quickly users expect the page to start showing meaningful progress. After 1 second, users may become distracted or even frustrated. Of course the holy grail is to have instant loading, but this is chosen as a realistic benchmark to strive towards.
So at worst 10% of the FCP experience is 1 second or slower. That specific kind of guarantee is a high enough bar to be confident that users ~consistently have a fast experience.
That explains why the bar is set where it is. To the question of how realistic it is to achieve, we can actually answer that using the publicly available CrUX data on BigQuery.
#standardSQL
SELECT
p90,
COUNT(0) AS numOrigins
FROM (
SELECT
origin,
MIN(start) AS p90
FROM (
SELECT
origin,
bin.start,
SUM(bin.density) OVER (PARTITION BY origin ORDER BY bin.start) AS cdf
FROM
`chrome-ux-report.all.201901`,
UNNEST(first_contentful_paint.histogram.bin) AS bin)
WHERE
cdf >= 0.9
GROUP BY
origin)
GROUP BY
p90
ORDER BY
p90
This is a query that counts where in the FCP histogram origins have their 90th percentile. If that sounds confusing, here's a visualization:
Where the red cumulative distribution line crosses the 1000ms mark tells us the percent of origins who would be labelled as fast. It isn't very many; just 2% or 110153 origins in the dataset.
Anecdotally, browsing through the list of "fast FCP" origins, many of them have .jp and .kr TLDs. It's reasonable to assume they are localized Japanese and Korean websites whose users are almost entirely from those countries. And these are countries with fast internet speeds. So naturally it'd be easier to serve a fast website 90+% of the time when your users have consistently fast connection speeds.
Another thing we can do to get a sense of origin popularity is join it with the Alexa Top 1 Million Domains list:
#standardSQL
SELECT
Alexa_rank,
Alexa_domain,
COUNT(0) AS numOrigins,
ARRAY_AGG(origin LIMIT 3) AS originSample
FROM (
SELECT
origin,
MIN(start) AS p90
FROM (
SELECT
origin,
bin.start,
SUM(bin.density) OVER (PARTITION BY origin ORDER BY bin.start) AS cdf
FROM
`chrome-ux-report.all.201901`,
UNNEST(first_contentful_paint.histogram.bin) AS bin)
WHERE
cdf >= 0.9
GROUP BY
origin)
JOIN
`httparchive.urls.20170315`
ON
NET.REG_DOMAIN(origin) = Alexa_domain
WHERE
p90 < 1000
GROUP BY
Alexa_rank,
Alexa_domain
ORDER BY
Alexa_rank
There are 35985 origins whose domains are in the top 1M. You can run the query for yourself to see the full results.
You can see that there are ~100 origins on top 20 domains that qualify as fast for FCP. Cherrypicking some interesting examples further down the list:
[#139] https://mobile.alibaba.com
[#178] https://www.google.se
[#422] http://www.design.samsung.com
[#744] http://taxes.ca.gov
Big caveat that these origins are not necessarily top ranked, just their domains. Without having origin ranking data this is the best approximation I can do.
Lesser caveat that BigQuery and PSI are slightly different datasets and PSI segments by desktop/mobile while my analysis combines them together. So this research is not a perfect representation of what to expect on PSI.
Finally, I just want to address something else that was in the question about getting 100 scores in Lighthouse. A score of 100 doesn't necessarily mean that there isn't anything left to improve. Synthetic tests like that need to be calibrated to be representative of the actual user experience. So for example the performance audits might start failing if tested under conditions representative of user experiences in the Philippines. Actually running the test from that location might turn up performance problems, eg content distribution issues, in addition to the conditions that we could simulate anywhere like connection speed.
To summarize everything:
The bar is set high because we want to ensure that a vast majority of user experiences are fast
Many websites are already exceeding this bar, albeit a small proportion of the overall dataset
The Alexa ranking shows us that it's possible to have a heavily trafficked website and also provide consistently fast experiences
You are misinterpreting the google lighthouse results. First of all, no performance test is absolute. It's impossible to have a fully 100% performant page simply because even if it loads in 1 second for me, it might not load in 1 second for a person in Ghana due to network issues and delays. Even if I have a pure HTML page with no javascript which is served as a static file from a super fast web server, that page might load in 10 seconds for a person with a dial up internet somewhere in Cuba or Jamaica.
Heavy traffic simply means "I get traffic not just from USA or Europe where the internet is blazing fast, I also get traffic from Jamaica where internet speed is a joke". Every serious web application has this issue. So yes, there is little room for improvement because you do everything right - it's a local internet issue.
I guess this immediately translates to a sociological/political "first world problem" mind set issue. You are obviously living in a first world country or at least have 3G/4G internet and you can't imagine that people in Jamaica have 2G internet. So don't fret about the lighthouse percentages. Making a web site fully 100% performant which loads in under 1 second anywhere on the globe is impossible due to technical limitations of that country - impossible for you to fix.
Will a page like nytimes.com ever be considered fast with this standard, when even google.com's desktop page is ranked slow based on field data?
The answer is: YES, absolutely.
I get your confusion. It is caused by the false assumption that Google has a well-performing website. Please note that Googles homepage is ridiculously large. The HTML alone is over 200kb. The javascript it loads weighs a massive 436kb. The total weight of the page is over 1Mb. And what do we see on this page? Absolutely nothing. It is literally an empty white page. One megabyte is the amount of code that could fill 500 pages of a book. The code in these two Harry Potter novels needs to be executed by your browser as soon as you load this empty page.
Just to give you another idea of how rediculously large this is: I own a web development agency in Amsterdam and my website (front page) is just as emtpy as this Google page. However, it weighs only 41kb (including a completely unnecessary custom woff2 font file, which takes up 17kb).
When you connect to the Google homepage with a regular 3G connection, the page takes over 3.5 seconds to load. Just think about what that means for people in Jamaica or Cuba! They will have close to no access to Google on desktop, or at least a very bad experience. As a comparison: my website loads in 0.7 seconds over regular 3G. It is important to know that size is the main speed influencer when you have slow(er) internet (which is half of the world).
So... the Google homepage on desktop is a very bad example and more than deserves its low (speed) score. The New York Times can easily get a better score, simply by reducing the weight of its pages below the weight of the Google homepage.
Performance score versus FCP
Last time I checked (early Feb. 2018), the Desktop google.com received a 100 Lighthouse synthetic score, which is supposed to be interpreted as "there is little room for improvement," and yet, the page is ranked "slow" because the 90th percentile FCP is way over 3s.
In the part above you relate the score of 100 to the FCP. It is not as simple as that (anymore). The performance score is a complex metric. It is the weighted avarage of the variables below (note that the FCP is no longer part of this).
First meaningful paint - weight: 5
First interactive - weight: 5
Consistently interactive - weight: 5
Speed index metric - weight: 1
Estimated input latency - weight: 1
Note that the Google homepage takes 3.5 seconds to be interactive (according to Lighthouse). However, it currently still scores 97 on performance, due to the way the metric is calculated, which is at least remarkable. This confirms that the (near) 100 score can be a misleading figure.
I have used YouTube API v3 to retrieve the number of video searches as a function of location. The purpose of this data is to study the distribution of YouTube's public: urban/rural, center of the cities or the metropolitan area, etc.
As part of this study, I checked the stability of the public (if there are changes over time of public behavior), retrieving the information regularly over the last 3 months.
Around 3 weeks ago, I observed a sudden drop of the number of counts in total results. More specifically, I retrieve pageInfo.totalResults from search.list. Before the change, totalResults yielded up to tens of thousands in the cities and provided useful information about the rural areas. However, now the highest count gets to a few dozens in the same areas.
I have been using the exact same script to study these areas, yielding the same information with no change. All my searches through the API documentation obtained no result. There is no record of any change on the API or the way the results are stored/retrieved from the API.
Has there been any change on how pageInfo.totalResults is stored/retrieved? Or, is totalResults a accumulative variable, restoring the counts at a certain time of the year.
I will be happy to forward this issue to the correct place, but it seems the issue tracker of YouTube API is inactive currently.
Thanks and best regards,
Diego
I am getting started with the Google Places API for Java/Android and JavaScript and have something running.
I would like to be able to implement a "fall-back" approach where I return results to the user sorted by distance, but if nothing interesting (place of business, airport, shopping mall, etc.) is found within a certain radius, then I would go to a coarser granularity and return street intersection, neighborhood, or city in that order. What would be best way to do this, preferably in one query?
Also, I noticed that I can specify "rankby=distance", but I cannot provide a radius, which is very inconvenient. Is there any workaround to this limitation?
The only workaround I can think of to sort by distance within a set radius, would be to manually process the returned results to see if their location falls within your specified radius. If you believe that this would be a useful feature, I would also recommend submitting a 'Places API - Feature Request' here.
A possible solution to your "fall-back" approach would be, if the a Places API Search Request returns "status" : "ZERO_RESULTS" or if no results returned fall within your desired radius, you could perform a Geocoding API Reverse Geocoding Request using the latitude and longitude from your Places API Search Request to find the closest address.
I am using the Google Places API to retrieve all the POI (Places of Interest) around a the current location, it works ok but I have noticed that whatever the value of the radius is, I always get the same number of results (~ 20). As a result, if I give a radius that is too big, I don't necessarily get the nearest POIs. If I reduce the amount of the radius to be small enough, I will retrieve those nearest places again (from experimentation, I have noticed that 100 meters is a proper value) but that means that I will not get any POIs beyond 100 meters which is not quite what I want.
My question is: is there any way by which I can get all the POIs (with no limitations) within a certain radius.
Thank you!
The Google Places API always returns 20 results by design, selecting the 20 results that best fit the criteria you define in your request. The Developer's Guide / Docs don't explicitly cite that number anywhere that I have seen. I learned about the limit watching the Autocomplete Demo & Places API Demo & Discussion Video, given by Paul Saxman, a Developer Advocate at Google and Marcelo Camelo, Google's Technical Lead for the Places API.
The entire video is worth watching, but more specific to your question, if you set the playback timer at about 11:50, Marcelo Camelo is contrasting the Autocomplete tool versus the general Places API, and that's the portion of the video where he mentions the 20 result limit. He mentions 20 as the standard result count several times.
There are many other good Places API and Google Maps videos linked to that area on YouTube as well.
As mentioned on the Google Places Issue Tracker here: http://code.google.com/p/gmaps-api-issues/issues/detail?id=3425
We are restricted by our data provider licenses to enable apps to display no more than 20 places results at a time. Consequently we are not able to increase this limit at this time.
It does sound like you are however trying to return results that are closest to a specified location, this is now possible by using the 'rankby=distance' parameter instead of 'radius' in your request.
e.g.
https://maps.googleapis.com/maps/api/place/search/json?location=-33.8670522,151.1957362&rankby=distance&types=food&name=harbour&sensor=false&key=YOUR_API_KEY
Try google.maps.places.RankBy.DISTANCE; as default is google.maps.places.RankBy.PROMINENCE;
An easy example of this is shown Here
(Chrome only)