Using Alexa.com I can find out that 0.05 % of all internet users visit some site, but how many people equals that 0.05% ?
Is there any facts like: in US 1% from Alexa statistics is nearly equals 15 mln of people, and in France 1% is about 3 mln of people, for example?
Compete.com reckon that google has something like 147m monthly users, and alexa says they have 34% monthly. Ergo, you could estimate it to be approx 450million. That's one way of estimating...
Of course the data from both Compete and Alexa gets progressively more rubbish the smaller the site gets. Data for the biggest sites is likely to be the least skewed, but I still wouldn't trust it for anything serious.
InternetWorldStats.com has a number of 1.6 billion internet users worldwide
You can get world population statistics online - estimates are available here:
Wikipedia World Population
This will help you to rough-up some statistics, but you need to remember...
Population is not equal to "has an internet connection"
0.5% does not really equate to "internet users" - it's more like 0.5% of people who are the kind of people that would install a random toolbar that offers them very little - so you need to bear in mind it's a certain "type" of person and that the statistics will be skewed (which is why www.alexa.com isn't ranked as EVERYONE with the Alexa toolbar is going to visit that website at some point
The smaller your website, the less accurate the statistics are. If you aren't in the top 100,000 websites in the world, the statistics become largely an anomaly as they "estimate up" the statistics from the toolbar users into an "average if everyone had a toolbar".
Hope this helps.
Alexa doesn't show "X% of France users use this site". Instead it shows "X% of worldwide users use this site". So you don't have such information except the margin cases when 100% of site users are from one country.
Also most toolbars show just Alexa Rank. You can get online converter "Alexa Rank -> Monthly Traffic" here - http://netberry.co.uk/alexa-rank-explained.htm
Well, here (http://netberry.co.uk/alexa-rank-explained.htm) is described a way to make a traffic estimation based on the alexa rank. Basically, the author has offered an exponential function, not linear or polynomial.
There is also a web service which aggregated alexa rank information and has already performed all the calculations: http://www.rank2traffic.com/
I checked it, and for 80% of the websites the results are very satisfying. Still, there is 20% of (possibly, manipulated by webmasters) incorrect data (the estimated traffic is much higher than in reality)
Related
Google changed its PSI definition of fast-ranking FCP from 90-percentile to 75-percentile below 1000ms
From PSI documentation:
Why does the FCP in v4 and v5 have different values?
FCP in v5 reports the 75th percentile (as of November 4th 2019),
previously it was the 90th percentile. In v4, FCP reports the median
(50th percentile).
Good data/tips in top answer from Rick below still.
ORIGINAL QUESTION:
Does using the 90-percentile instead of previous median score, or a lower percentile, when saying, "based on field data the 'page is slow'" make it impossible for heavily trafficked websites, such as google.com, from ever getting ranked "Fast"? This due to the long tail that occurs when monthly traffic is in the 10M+ ranges and globally distributed?
Last time I checked (early Feb. 2018), the Desktop google.com received a 100 Lighthouse synthetic score, which is supposed to be interpreted as "there is little room for improvement," and yet, the page is ranked "slow" because the 90th percentile FCP is way over 3s.
Will a page like nytimes.com ever be considered fast with this standard, when even google.com's desktop page is ranked slow based on field data?
Recent example (Feb. 14, 2019)
Former example with even longer tail for FCP:
To directly answer the question, no it's not impossible to get a fast FCP label. There's more to the question so I'll try to elaborate.
Another way to phrase the "fast" criteria is: "Do at least 90% of user experiences have an FCP less than 1 second?"
Why 90%? Because it's inclusive of a huge proportion of user experiences. As the PSI docs say:
Our goal is to make sure that pages work well for the majority of users. By focusing on 90th and 95th percentile values for our metrics, this ensures that pages meet a minimum standard of performance under the most difficult device and network conditions.
Why 1 second? It's a subjective value for how quickly users expect the page to start showing meaningful progress. After 1 second, users may become distracted or even frustrated. Of course the holy grail is to have instant loading, but this is chosen as a realistic benchmark to strive towards.
So at worst 10% of the FCP experience is 1 second or slower. That specific kind of guarantee is a high enough bar to be confident that users ~consistently have a fast experience.
That explains why the bar is set where it is. To the question of how realistic it is to achieve, we can actually answer that using the publicly available CrUX data on BigQuery.
#standardSQL
SELECT
p90,
COUNT(0) AS numOrigins
FROM (
SELECT
origin,
MIN(start) AS p90
FROM (
SELECT
origin,
bin.start,
SUM(bin.density) OVER (PARTITION BY origin ORDER BY bin.start) AS cdf
FROM
`chrome-ux-report.all.201901`,
UNNEST(first_contentful_paint.histogram.bin) AS bin)
WHERE
cdf >= 0.9
GROUP BY
origin)
GROUP BY
p90
ORDER BY
p90
This is a query that counts where in the FCP histogram origins have their 90th percentile. If that sounds confusing, here's a visualization:
Where the red cumulative distribution line crosses the 1000ms mark tells us the percent of origins who would be labelled as fast. It isn't very many; just 2% or 110153 origins in the dataset.
Anecdotally, browsing through the list of "fast FCP" origins, many of them have .jp and .kr TLDs. It's reasonable to assume they are localized Japanese and Korean websites whose users are almost entirely from those countries. And these are countries with fast internet speeds. So naturally it'd be easier to serve a fast website 90+% of the time when your users have consistently fast connection speeds.
Another thing we can do to get a sense of origin popularity is join it with the Alexa Top 1 Million Domains list:
#standardSQL
SELECT
Alexa_rank,
Alexa_domain,
COUNT(0) AS numOrigins,
ARRAY_AGG(origin LIMIT 3) AS originSample
FROM (
SELECT
origin,
MIN(start) AS p90
FROM (
SELECT
origin,
bin.start,
SUM(bin.density) OVER (PARTITION BY origin ORDER BY bin.start) AS cdf
FROM
`chrome-ux-report.all.201901`,
UNNEST(first_contentful_paint.histogram.bin) AS bin)
WHERE
cdf >= 0.9
GROUP BY
origin)
JOIN
`httparchive.urls.20170315`
ON
NET.REG_DOMAIN(origin) = Alexa_domain
WHERE
p90 < 1000
GROUP BY
Alexa_rank,
Alexa_domain
ORDER BY
Alexa_rank
There are 35985 origins whose domains are in the top 1M. You can run the query for yourself to see the full results.
You can see that there are ~100 origins on top 20 domains that qualify as fast for FCP. Cherrypicking some interesting examples further down the list:
[#139] https://mobile.alibaba.com
[#178] https://www.google.se
[#422] http://www.design.samsung.com
[#744] http://taxes.ca.gov
Big caveat that these origins are not necessarily top ranked, just their domains. Without having origin ranking data this is the best approximation I can do.
Lesser caveat that BigQuery and PSI are slightly different datasets and PSI segments by desktop/mobile while my analysis combines them together. So this research is not a perfect representation of what to expect on PSI.
Finally, I just want to address something else that was in the question about getting 100 scores in Lighthouse. A score of 100 doesn't necessarily mean that there isn't anything left to improve. Synthetic tests like that need to be calibrated to be representative of the actual user experience. So for example the performance audits might start failing if tested under conditions representative of user experiences in the Philippines. Actually running the test from that location might turn up performance problems, eg content distribution issues, in addition to the conditions that we could simulate anywhere like connection speed.
To summarize everything:
The bar is set high because we want to ensure that a vast majority of user experiences are fast
Many websites are already exceeding this bar, albeit a small proportion of the overall dataset
The Alexa ranking shows us that it's possible to have a heavily trafficked website and also provide consistently fast experiences
You are misinterpreting the google lighthouse results. First of all, no performance test is absolute. It's impossible to have a fully 100% performant page simply because even if it loads in 1 second for me, it might not load in 1 second for a person in Ghana due to network issues and delays. Even if I have a pure HTML page with no javascript which is served as a static file from a super fast web server, that page might load in 10 seconds for a person with a dial up internet somewhere in Cuba or Jamaica.
Heavy traffic simply means "I get traffic not just from USA or Europe where the internet is blazing fast, I also get traffic from Jamaica where internet speed is a joke". Every serious web application has this issue. So yes, there is little room for improvement because you do everything right - it's a local internet issue.
I guess this immediately translates to a sociological/political "first world problem" mind set issue. You are obviously living in a first world country or at least have 3G/4G internet and you can't imagine that people in Jamaica have 2G internet. So don't fret about the lighthouse percentages. Making a web site fully 100% performant which loads in under 1 second anywhere on the globe is impossible due to technical limitations of that country - impossible for you to fix.
Will a page like nytimes.com ever be considered fast with this standard, when even google.com's desktop page is ranked slow based on field data?
The answer is: YES, absolutely.
I get your confusion. It is caused by the false assumption that Google has a well-performing website. Please note that Googles homepage is ridiculously large. The HTML alone is over 200kb. The javascript it loads weighs a massive 436kb. The total weight of the page is over 1Mb. And what do we see on this page? Absolutely nothing. It is literally an empty white page. One megabyte is the amount of code that could fill 500 pages of a book. The code in these two Harry Potter novels needs to be executed by your browser as soon as you load this empty page.
Just to give you another idea of how rediculously large this is: I own a web development agency in Amsterdam and my website (front page) is just as emtpy as this Google page. However, it weighs only 41kb (including a completely unnecessary custom woff2 font file, which takes up 17kb).
When you connect to the Google homepage with a regular 3G connection, the page takes over 3.5 seconds to load. Just think about what that means for people in Jamaica or Cuba! They will have close to no access to Google on desktop, or at least a very bad experience. As a comparison: my website loads in 0.7 seconds over regular 3G. It is important to know that size is the main speed influencer when you have slow(er) internet (which is half of the world).
So... the Google homepage on desktop is a very bad example and more than deserves its low (speed) score. The New York Times can easily get a better score, simply by reducing the weight of its pages below the weight of the Google homepage.
Performance score versus FCP
Last time I checked (early Feb. 2018), the Desktop google.com received a 100 Lighthouse synthetic score, which is supposed to be interpreted as "there is little room for improvement," and yet, the page is ranked "slow" because the 90th percentile FCP is way over 3s.
In the part above you relate the score of 100 to the FCP. It is not as simple as that (anymore). The performance score is a complex metric. It is the weighted avarage of the variables below (note that the FCP is no longer part of this).
First meaningful paint - weight: 5
First interactive - weight: 5
Consistently interactive - weight: 5
Speed index metric - weight: 1
Estimated input latency - weight: 1
Note that the Google homepage takes 3.5 seconds to be interactive (according to Lighthouse). However, it currently still scores 97 on performance, due to the way the metric is calculated, which is at least remarkable. This confirms that the (near) 100 score can be a misleading figure.
On social networks like twitter where millions follow single account, it must be very challenging to update all followers instantly when a new tweet is posted. Similarly on facebook there are fan pages with millions of followers and we see updates from them instantly when posted on page. I am wondering what are best known techniques and algorithms to achieve this. I understand with billion accounts, they have huge data centers across globe and even if we reduce this problem for just one computer in following manner - 100,000 nodes with average 200 edges per node, then every single node update will require 200 edge updates. So what are best techniques/algorithms to optimize such large updates. Thanks!
The best way is usually just to do all the updates. You say they can be seen "instantly", but actually the updates probably propagate through the network and can take up to a few seconds to show up in followers' feeds.
Having to do all those updates may seem like a lot, but on average follower will check for updates much more often than a person being followed will produce them, and checking for updates has to be much faster.
The choices are:
Update 1 million followers, a couple times a day, within a few seconds; or
Respond to checks from 1 million followers, a couple hundred times a day, within 1/10 second or so.
There are in-between strategies involving clustering users and stuff, but usage patterns like you see on Facebook and Twitter are probably so heavily biased toward option (1) that such strategies don't pay off.
My department has installed google analytics on our companies website and none of us are exactly experts on understanding why the data is the way it is.
Anyways, our company is fairly large, but I wouldn't say we are exactly a well known company. We provide internet and Video on Demand to Hotels worldwide. Anyways, as of right now, since I have installed our code last month, we have a total session number of over 78,000. Our average session duration is only 24 seconds, with an average page view per visitor at 1.18 and a bounce rate of 91%.
I don't doubt the session average time. Me and my co-workers are just a little confused as to how with that many visitors, we are consistently across the board getting such a fairly small session duration and a high bounce rate. Could visitors possibly just come to our website, look for our phone number and than leave the site? I'm just trying to find a way to reduce the bounce rate and hopefully increase the session duration average. Or is it possible to add a filter that will exclude visits to the site that are less than 30 seconds, or something like that? I apologize for asking such fairly basic questions I'm sure. I am trying to get up to speed and familiarize myself with how this all works. Just thought I'd maybe ask and see if I am missing something important. Any advice would be greatly appreciated. Thank you!
Its hard to tell why your metrics are so low across the board. Your referral traffic could be to blame here - possibly bad ad copy from adwords or bing, which is making users think they're going to a different page. You can always create a filter by the acquisition, and where the clicks and sessions are coming from. From there it'll be easier to see which source of traffic is to blame, and also how you can improve your site overall for traffic optimization and user friendliness. For more info, reach us at RLCppc.com on the matter. hope that answers your question
What technology goes in behind the screens of Amazon recommendation technology? I believe that Amazon recommendation is currently the best in the market, but how do they provide us with such relevant recommendations?
Recently, we have been involved with similar recommendation kind of project, but would surely like to know about the in and outs of the Amazon recommendation technology from a technical standpoint.
Any inputs would be highly appreciated.
Update:
This patent explains how personalized recommendations are done but it is not very technical, and so it would be really nice if some insights could be provided.
From the comments of Dave, Affinity Analysis forms the basis for such kind of Recommendation Engines. Also here are some good reads on the Topic
Demystifying Market Basket Analysis
Market Basket Analysis
Affinity Analysis
Suggested Reading:
Data Mining: Concepts and Technique
It is both an art and a science. Typical fields of study revolve around market basket analysis (also called affinity analysis) which is a subset of the field of data mining. Typical components in such a system include identification of primary driver items and the identification of affinity items (accessory upsell, cross sell).
Keep in mind the data sources they have to mine...
Purchased shopping carts = real money from real people spent on real items = powerful data and a lot of it.
Items added to carts but abandoned.
Pricing experiments online (A/B testing, etc.) where they offer the same products at different prices and see the results
Packaging experiments (A/B testing, etc.) where they offer different products in different "bundles" or discount various pairings of items
Wishlists - what's on them specifically for you - and in aggregate it can be treated similarly to another stream of basket analysis data
Referral sites (identification of where you came in from can hint other items of interest)
Dwell times (how long before you click back and pick a different item)
Ratings by you or those in your social network/buying circles - if you rate things you like you get more of what you like and if you confirm with the "i already own it" button they create a very complete profile of you
Demographic information (your shipping address, etc.) - they know what is popular in your general area for your kids, yourself, your spouse, etc.
user segmentation = did you buy 3 books in separate months for a toddler? likely have a kid or more.. etc.
Direct marketing click through data - did you get an email from them and click through? They know which email it was and what you clicked through on and whether you bought it as a result.
Click paths in session - what did you view regardless of whether it went in your cart
Number of times viewed an item before final purchase
If you're dealing with a brick and mortar store they might have your physical purchase history to go off of as well (i.e. toys r us or something that is online and also a physical store)
etc. etc. etc.
Luckily people behave similarly in aggregate so the more they know about the buying population at large the better they know what will and won't sell and with every transaction and every rating/wishlist add/browse they know how to more personally tailor recommendations. Keep in mind this is likely only a small sample of the full set of influences of what ends up in recommendations, etc.
Now I have no inside knowledge of how Amazon does business (never worked there) and all I'm doing is talking about classical approaches to the problem of online commerce - I used to be the PM who worked on data mining and analytics for the Microsoft product called Commerce Server. We shipped in Commerce Server the tools that allowed people to build sites with similar capabilities.... but the bigger the sales volume the better the data the better the model - and Amazon is BIG. I can only imagine how fun it is to play with models with that much data in a commerce driven site. Now many of those algorithms (like the predictor that started out in commerce server) have moved on to live directly within Microsoft SQL.
The four big take-a-ways you should have are:
Amazon (or any retailer) is looking at aggregate data for tons of transactions and tons of people... this allows them to even recommend pretty well for anonymous users on their site.
Amazon (or any sophisticated retailer) is keeping track of behavior and purchases of anyone that is logged in and using that to further refine on top of the mass aggregate data.
Often there is a means of over riding the accumulated data and taking "editorial" control of suggestions for product managers of specific lines (like some person who owns the 'digital cameras' vertical or the 'romance novels' vertical or similar) where they truly are experts
There are often promotional deals (i.e. sony or panasonic or nikon or canon or sprint or verizon pays additional money to the retailer, or gives a better discount at larger quantities or other things in those lines) that will cause certain "suggestions" to rise to the top more often than others - there is always some reasonable business logic and business reason behind this targeted at making more on each transaction or reducing wholesale costs, etc.
In terms of actual implementation? Just about all large online systems boil down to some set of pipelines (or a filter pattern implementation or a workflow, etc. you call it what you will) that allow for a context to be evaluated by a series of modules that apply some form of business logic.
Typically a different pipeline would be associated with each separate task on the page - you might have one that does recommended "packages/upsells" (i.e. buy this with the item you're looking at) and one that does "alternatives" (i.e. buy this instead of the thing you're looking at) and another that pulls items most closely related from your wish list (by product category or similar).
The results of these pipelines are able to be placed on various parts of the page (above the scroll bar, below the scroll, on the left, on the right, different fonts, different size images, etc.) and tested to see which perform best. Since you're using nice easy to plug and play modules that define the business logic for these pipelines you end up with the moral equivalent of lego blocks that make it easy to pick and choose from the business logic you want applied when you build another pipeline which allows faster innovation, more experimentation, and in the end higher profits.
Did that help at all? Hope that give you a little bit of insight how this works in general for just about any ecommerce site - not just Amazon. Amazon (from talking to friends that have worked there) is very data driven and continually measures the effectiveness of it's user experience and the pricing, promotion, packaging, etc. - they are a very sophisticated retailer online and are likely at the leading edge of a lot of the algorithms they use to optimize profit - and those are likely proprietary secrets (you know like the formula to KFC's secret spices) and guaarded as such.
This isn't directly related to Amazon's recommendation system, but it might be helpful to study the methods used by people who competed in the Netflix Prize, a contest to develop a better recommendation system using Netflix user data. A lot of good information exists in their community about data mining techniques in general.
The team that won used a blend of the recommendations generated by a lot of different models/techniques. I know that some of the main methods used were principal component analysis, nearest neighbor methods, and neural networks. Here are some papers by the winning team:
R. Bell, Y. Koren, C. Volinsky, "The BellKor 2008 Solution to the Netflix Prize", (2008).
A. Töscher, M. Jahrer, “The BigChaos Solution to the Netflix Prize 2008", (2008).
A. Töscher, M. Jahrer, R. Legenstein, "Improved Neighborhood-Based Algorithms for Large-Scale Recommender Systems", SIGKDD Workshop on Large-Scale Recommender Systems and the Netflix Prize Competition (KDD’08) , ACM Press (2008).
Y. Koren, "The BellKor Solution to the Netflix Grand Prize", (2009).
A. Töscher, M. Jahrer, R. Bell, "The BigChaos Solution to the Netflix Grand Prize", (2009).
M. Piotte, M. Chabbert, "The Pragmatic Theory solution to the Netflix Grand Prize", (2009).
The 2008 papers are from the first year's Progress Prize. I recommend reading the earlier ones first because the later ones build upon the previous work.
I bumped on this paper today:
Amazon.com Recommendations: Item-to-Item Collaborative Filtering
Maybe it provides additional information.
(Disclamer: I used to work at Amazon, though I didn't work on the recommendations team.)
ewernli's answer should be the correct one -- the paper links to Amazon's original recommendation system, and from what I can tell (both from personal experience as an Amazon shopper and having worked on similar systems at other companies), very little has changed: at its core, Amazon's recommendation feature is still very heavily based on item-to-item collaborative filtering.
Just look at what form the recommendations take: on my front page, they're all either of the form "You viewed X...Customers who also viewed this also viewed...", or else a melange of items similar to things I've bought or viewed before. If I specifically go to my "Recommended for You" page, every item describes why it's recommended for me: "Recommended because you purchased...", "Recommended because you added X to your wishlist...", etc. This is a classic sign of item-to-item collaborative filtering.
So how does item-to-item collaborative filtering work? Basically, for each item, you build a "neighborhood" of related items (e.g., by looking at what items people have viewed together or what items people have bought together -- to determine similarity, you can use metrics like the Jaccard index; correlation is another possibility, though I suspect Amazon doesn't use ratings data very heavily). Then, whenever I view an item X or make a purchase Y, Amazon suggests me things in the same neighborhood as X or Y.
Some other approaches that Amazon could potentially use, but likely doesn't, are described here: http://blog.echen.me/2011/02/15/an-overview-of-item-to-item-collaborative-filtering-with-amazons-recommendation-system/
A lot of what Dave describes is almost certainly not done at Amazon. (Ratings by those in my social network? Nope, Amazon doesn't have any of my social data. This would be a massive privacy issue in any case, so it'd be tricky for Amazon to do even if they had that data: people don't want their friends to know what books or movies they're buying. Demographic information? Nope, nothing in the recommendations suggests they're looking at this. [Unlike Netflix, who does surface what other people in my area are watching.])
I don't have any knowledge of Amazon's algorithm specifically, but one component of such an algorithm would probably involve tracking groups of items frequently ordered together, and then using that data to recommend other items in the group when a customer purchases some subset of the group.
Another possibility would be to track the frequency of item B being ordered within N days after ordering item A, which could suggest a correlation.
As far I know, it's use Case-Based Reasoning as an engine for it.
You can see in this sources: here, here and here.
There are many sources in google searching for amazon and case-based reasoning.
If you want a hands-on tutorial (using open-source R) then you could do worse than going through this:
https://gist.github.com/yoshiki146/31d4a46c3d8e906c3cd24f425568d34e
It is a run-time optimised version of another piece of work:
http://www.salemmarafi.com/code/collaborative-filtering-r/
However, the variation of the code on the first link runs MUCH faster so I recommend using that (I found the only slow part of yoshiki146's code is the final routine which generates the recommendation at user level - it took about an hour with my data on my machine).
I adapted this code to work as a recommendation engine for the retailer I work for.
The algorithm used is - as others have said above - collaborative filtering. This method of CF calculates a cosine similarity matrix and then sorts by that similarity to find the 'nearest neighbour' for each element (music band in the example given, retail product in my application).
The resulting table can recommend a band/product based on another chosen band/product.
The next section of the code goes a step further with USER (or customer) based collaborative filtering.
The output of this is a large table with the top 100 bands/products recommended for a given user/customer
Someone did a presentation at our University on something similar last week, and referenced the Amazon recommendation system. I believe that it uses a form of K-Means Clustering to cluster people into their different buying habits. Hope this helps :)
Check this out too: Link and as HTML.
Of course the best metric would be a happiness of your users.
But what metrics do you know for GUI usability measurements?
For example, one of the common metrics is a average click count to perform action.
What other metrics do you know?
Jakob Nielsen has several articles regarding usability metrics, including one that is entitled, well, Usability Metrics:
The most basic measures are based on the definition of usability as a quality metric:
success rate (whether users can perform the task at all),
the time a task requires,
the error rate, and
users' subjective satisfaction.
I just look at where I want users to go and where (physically) they are going on screen, I do this with data from Google Analytics.
Not strictly usability, but we sometimes measure the ratio of the GUI and the backend code. This is for the managers, to remind them, that while functionality is importaint, the GUI should get a proportional budget for user testing and study too.
check:
http://www.iqcontent.com/blog/2007/05/a-really-simple-metric-for-measuring-user-interfaces/
Here is a simple pre-launch check you
should do on all your web
applications. It only takes about 5
seconds and one screeshot
Q: “What percentage of your interface contains stuff that your customers
want to see?”
10%
25%
100%
If you answer a, or b then you might
do well, but you’ll probably get blown
out of the water once someone decides
to enter the market with option c.