I have a network connection where I pay per megabyte, so I'm interested in reducing my bandwidth usage as far as possible while still having a reasonably good browsing experience. I use this wonderful extension (https://bandwidth-hero.com/). This extension runs a image-compression proxy on my heroku account that accepts images URLs, and returns a low-quality version of those images.This reduces bandwidth usage by 30-40% when images are loaded.
To further reduce usage, I typically browse with both JavaScript and images disabled (there are various extensions for doing this in firefox/firefox-esr/google-chrome). This has an added bonus of blocking most ads (since they usually need JavaScript to run).
For daily browsing, the most efficient solution is using a text-mode browser in a virtual console such as elinks/lynx/links2 running over ssh (with zlib compression) on a VPS server. But sometimes using JavaScript becomes necessary, as sites will not render without it .Elinks is the only text-mode browser that even tries to support JavaScript, and even that support is quite rudimentary. When I have to come back to using firefox/chrome, I find my bandwidth usage shooting up. I would like to avoid this.
I find that bandwidth is used partially to get the 'raw' html files of the sites I'm browsing, but more often for the associated .js/.css files. These are typically highly compressible. On my local workstation, html+css+javascript files typically compress by a factor of more than 10x when using lzma(2) compression.
It seems to me that one way to do drastically reduce bandwidth consumption would be to use the same template as the bandwidth-hero extension, i.e. run a compression proxy either on a vps or on my heroku account but do so for text content (.html/.js/.css).
Ideally, I would like to run a compression proxy on my local machine. When I open a site (say www.stackoverflow.com), the browser should send a request to this local proxy. This local proxy then sends a request to a back-end running on heroku/vps. The heroku/vps back-end actually fetches all the content, and compresses it (lzma/bzip/gzip). The compressed content is sent back to my local proxy. The local proxy decompresses the content and finally gives it to the browser.
There is something like this mentioned in this answer (https://stackoverflow.com/a/42505732/10690958) for node.js . I am thinking of the same for python.
From what google searches show, HTTP can "automatically" ask for gzip versions of pages. But does this also apply for the associated files that are loaded by JavaScript, and for the css files? Perhaps, what I am thinking about is already implemented by default ?
Any pointers would be welcome. I was thinking of writing a local proxy in python,as I am reasonably fluent in it. But I know little about heroku or the intricacies of HTTP.
thanks.
Update: I found a possible solution here https://github.com/barnacs/compy
which does almost exactly what I need (minify+compress with brotli/gzip+transcode jpeg/gif/png). It uses go instead of python, but that does not really matter. It also has a docker image here https://hub.docker.com/r/andrewgaul/compy/ . Since I'm not very familiar with heroku, I cant figure out how to use this to run the compression proxy service on my account. The heroku docs also weren't of much help to me. Any pointers would be welcome.
Related
I am making a lot of _all_docs requests to CouchDB's HTTP server. One thing I'm realizing is that the data is not compressed, so this results in large file sizes. Even by using limit and skip, the files can sometimes be 10MB each. That doesn't cause any problems for my app, but it does mean that if a connection to our CouchDB server is slower than our office connection, it will go rather slow.
Is there any way I can enable HTTP compression? I am not referring to attachments - just the JSON files.
Also, I am using Windows Server - not Linux/Unix.
Thanks!
There is no support in CouchDB directly, but it has been requested. (so voice your support there if you want this included)
That being said, there are a number of options you have. First, you can set up nginx as a reverse proxy and allow it to compress (and possibly cache) responses for you. After a quick search, I found this plugin that you install in CouchDB directly.
Another thing is that CouchDB does a pretty solid job of allowing clients to cache reliably. You can leverage this to prevent repeatedly downloading the same large resource.
We're currently doing optimizations to our web project when our lead told us to push the use of CDNs for external libraries as opposed to including them into a compile+compress process and shipping them off a cache-enabled nginx setup.
His assumption is that if the user has visits example.com which uses a CDN'ed version of jQuery, the jQuery is cached that time. If the user happens to visit example2.com and happen to use the same CDN'ed jQuery, the jQuery will be loaded from cache instead of over the network.
So my question is: Do domains actually share their cache?
I argued that even if it is possible the browser does share cache, the problem is that we are running on the assumption that the previous sites use the same exact CDN'ed file from the same exact CDN. What are the chances of running into a user browsing through a site using the same CDN'ed file? He said to use the largest CDN to increase chances.
So the follow-up question would be: If the browser does share cache, is it worth the hassle to optimize based on his assumption?
I have looked up topics about CDNs and I have found nothing about this "shared domain cache" or CDNs being used this way.
Well your lead is right this is basic HTTP.
All you are doing is indicating to the client where it can find the file.
The client then handles sending a request to the CDN in compliance with their caching rules.
But you shouldn't over-use CDNs for libraries either, keep in mind that if you need a specific version of the library, especially older ones, you won't be likely to get much cache hits because of version fragmentation.
For widely used and heavy libraries like jQuery you want the latest version of it is recommended.
If you can take them all from the same CDN all the better (ie: Google's) especially as http2 is coming.
Additionally they save you bandwidth, which can amount to a lot when you have high loads of traffic, and can reduce the load time for users far from your server (Google's is great for this).
I'm having multiple downloads in a swf, downloading external data.
Can I limit the download speed of one download thread or prioritize one?
Flash will download them in the order you ask for them. If you need one first, then ask for that one first.
When running within a browser it is actually the browser that makes the download on behalf of the client container (the client merely requests that the browser does the download). A lot of browsers have restrictions on concurrent file downloads anyway, especially in the same domain, and that is not something you will be able to circumvent with the in-built classes.
My guess is that you could roll your own (or use an existing client as a base) using Socket instead of URLLoader. This would be a large amount of work though - the browser does a lot of free work for you (SSL, cookies from the current session, gzip, keep-alives etc).
Is loading fonts via storing them on your server and using #font-face slower than loading them from Google's font API? Or does it always depend on the font and vary from situation to situation?
And the same for Javascript and other similar files: is it faster or slower to load from CDN's than to store the files on your server and load them (locally on the server)?
Or are there too many variables involved from situation to situation to generalize to a single answer? I would imagine that it depends on which CDN you're accessing and/or your personal server settings and the size/nature of the files you're loading, etc, but I was just curious if there might be a general rule or strategy to knowing which is faster?
A CDN might be faster, on the base that it is built with speed in mind (high performance, tuned web servers, good caching...) and it is usually composed by a network of geographically distributed servers, lowering latence both because they are nearer and because they share the load. Also, they could be directly placed on backbones, which allow for much faster transfer rates than a low-to-mid-priced server will ever do.
Thus said, for a low traffic website mostly visited from one specific country, in turn near to the server location, the difference in load is irrelevant.
The reason for using Google or jQuery CDN is both saving bandwidth (if the respective owner allows you to use theirs, of course) on your server and be sure you do not miss urgent patches, as they will push fixed versions on the CDN as soon as possible, while you have to get notified, download the new version, then load it on your server (although I guess that this is not a great issues in modern, sanboxed browsers).
Is there an advantage of some sort (speed or performance wise) to embed your CSS and JS into your web page, as opposed to keeping the code in sparate files? I was raised to believe that keeping code separate in separate files makes things easier to maintain. However, on high profile websites like amazon or google even facebook, I see a lot of embed code. Is there a performance reason they choose to do so or is it just an old/new way of doing things. I suppose my question is similar to this one: Should I inline CSS & JS in mobile sites to save bandwidth?
But I would like to hear form experts, most notably from people who worked on high profile web sties and have done so, if any.
P.S.
Bonus Question: Last html comment on amazon web pages is <!-- MEOW --> does it mean anything or is it just a funny prank?
There are good reasons to inline resources, but as with most things, it also has its tradeoffs. The simplest case for inlining is cases where the cost of an HTTP connection is much more than the resource itself, ex: if you have a 10x10 icon you need to show, a dedicated request for that may not be worth it vs. inlining the data via a data URI.
This is especially true when and if you have many small resources that need to be fetchd. Most browsers limit themselves to a max of 6 connections per host, so if you have 60 resources which need to be fetched, then you'll be blocked for a significant chunk of time.
To work around these case we've invented other workarounds: domain sharding to go over the 6 connection limit, and "spriting" to fetch one resource vs multiple.
If you take a look at mod_pagespeed (Apache module), which does many of these optimizations on the fly for you, then the recommended setting we provide is to inline any resource that's below 2kb. That's a pretty good rule of thumb for today's stack.
Once SPDY is more widely deployed, many of these workarounds can be eliminated: no need to do domain sharding, cost of extra requests is much less, etc.
Stoyan did an experiment that you might find interesting http://www.phpied.com/style-tag-to-inline-style-attrrib/
CSS/JS external files typically get cached on the user's hard drive under that users browser's profile. So unless you change the code frequently, you won't really be doing yourself a favor by putting it inline.
Definitely saves you time from maintenance, but you can easily call in a javascript/css file and embed the code on the page you're populating on the server side, but that also means you're making your server do additional work.
As for the MEOW - yeah, them trying to be funny, or it's code... for... cat...