Should HttpUrl.Builder.addPathSegment %2A encode an asterisk? - okhttp

I'm curious why this passes.. eg the asterisk below isn't percent encoded as %2A.
HttpUrl url = new HttpUrl.Builder()
.scheme("http")
.host("host")
.addPathSegment("foo *")
.build();
assertEquals("http://host/foo%20*", url.toString());
assertEquals("/foo%20*", url.encodedPath());

Major browsers (Firefox, Safari, and Chrome) choose not to percent encode * as %20 when it is in the path segment, and therefore OkHttp chooses to be consistent with those behaviors.
The only part of a URL that the browsers and OkHttp differ on for * is the host in which Safari and OkHttp will error, Firefox leaves as is, and Chrome percent encodes as %20.
Research, Github,
Blog Post

Related

Encoding of web socket channel and/or [encoding convertto...] messages?

I have data in SQLite that is text including a curly apostrophe (U-2019 &#8217) but the character is typed as in work’s. A browser makes the request for the data over a web socket connection which is configured as chan configure $sock -encoding iso8859-1 -translation crlf -buffering full simply because that is what I set it to before sending the headers to the browser in response to the request to upgrade the connection to a web socket and failed to consider it again thereafter.
I was mistaken it is endoded as chan configure $sock -buffering full -blocking 0 -translation binary.
If I use replace( ..., '’', '&#8217') in the SQL then the browser displays the text with the curly apostrophe rendered normally. And if I follow the example in this SO question and encode the result of the SQL query as set encoded [encoding convertto utf-8 $SQLResult] it renders correctly also.
What is the correct way to handle this? Should the SQL results be endoded to utf-8 or should the web socket be configured differently? If the socket is encoded as utf-8 then it fails on the first message sent from the browser/client.
Thank you.
The data in SQLite should be Unicode (probably UTF-8 but that shouldn't matter; the package binding handles that). That means that the data arrives in Tcl correctly encoded. Tcl's internal encoding schemes are complicated and should be ignored for the purposes of this discussion: pretend that's all Unicode too.
With a websocket, you need to say what sort of messages ("frames") you are sending. For normal payloads, they can be either binary frames or text frames. Binary frames are a bunch of bytes. Text frames are UTF-8 (as specified by the standard). STOMP frames are text frames where the content is JSON (with a few basic rules and some higher-order things over what operations can be stated).
If you are sending text, you should use UTF-8. Ideally by configuring the socket to do that in the part of the websocket client that mediates between the point that receives the text to send (maybe JSON) and the part that writes the frame onto the underlying socket itself. Above that point, you should just work with ordinary Tcl strings. If you're forming JSON messages in Tcl, the rl_json package is recommended, or you can get SQLite to produce JSON directly. (JSON's serialized form is always UTF-8, and it is conceptually Unicode.)
If you are sending binary frames, you handle all the details yourself. Using encoding convertto utf-8 may help with forming the message frame body, or you can use any other encoding scheme you prefer (such as the one you mention in your question).

Caching seems to be breaking my website and i don't know where to look

Hoping someone here can assist with giving me a few ideas on what to check and where to look with a new issue I have.
I have implemented W3TC Caching on a website to improve loading times but it has created a problem with page loading - the pages now load garbled text on first load and then after refresh it seems to work fine.
I am 99% sure it's W3TC because when I clear the cache, the issue happens and then after a refresh, it goes away.
This is what is displayed before refreshing:
����v�'����"fd��t�o�o���<{Jz�F'ؼ�Tʡ ���>����ꛯ�EM����7ֈ��e����l{��#�ƛ��oA�
G�b�r,��z�U�]�3e��<����hI���2�,�Z�}RuO�i��Ck�T,����|^�q�~�F#M�R�E$k�k4��c��b��ޔ�C�����F�gc���#/H}V���AM�2�ӡE��"u�Zz �U���tFM�Ed��qduڞ3��#�����
��{ޔ&��������GJM;�b��)�
��oN�3�Tq7ɦZϞ�?aUo���
k�h�!A��D�CF�Y?�ȟ��?O8�5X#')+?.د��!����V�Q�q6�E�
0s���g3�1[�T �Nq�ң��beLq"��^YhhƓ�H&�O��8Zn�1�2�znon�� ��Z<�W�0�L��̖0ܞ����GqhQ�]g��U�z�}�����'��'��UG����gS��������o��*�3N=E��!i�,��\�ĝ�
d�C�F"
�۳���e�U�H�1�bʛV�J�>Y3�Әk$����9��Q�$��厁�q� ��Hy�.�vh]'�(�SqL�Z�m�è5�wL,�Bޑ#/����ʷV�����٦V�
ίB2=�nR#���4��b�J�B�Ë��{go�4R���,�g�d���V��خ�j͊V%����V��V���1�ɇ{������Q�Ѩ�?�(��ڃ�^���4� 0T��E?�N�[�D��W�H�Q�x/����S��7��m69�dr�.��Ǘ���,~�C6fRm"��m�C�΢�,"�id% ��U2F+�����h o/hU�&�J�^N�Z�,�#$�s�"M"v�Y�[f��0G�N��B0�����Ci���� 5��&k������F�r���*��jR����g���v��
�{�W�=�T��09�o�wC���e9���l�}��/_޾{\�S9=��$����ת����DlN����!�0xy0<��!4N��Z䏋�t��.�{�xwLkT."o��&���#_H�A8,Q?2P
;#��>>����*��45Պ�A�ZZrY�9�:̬��K������ =�����n����]ہw�*��H�:��t�0���B¯�a���Q�mW>c�:ID�G��1;=�T���0!��"�1 �X}}0Bm�!�����;_k2��'<R�qY�'ё��W L�sR7�7W��|߇�ۊ�1�Kfg�"�P�}�Cb׷�3A����wk��b d����s��ϸL��Ŕ�����6���-�^0���&�e$� � ��<�F�ZO�x��V�!��Y�N$���A�.�s�1������~T�OE���g�����s�������2&l1���DB�EB�B9O#����#r��F��{]ⴻ'u7cpxũW�[fo�MSH���%Uz/cH�Wc�e�����֥:M�M[��2�w[�͌(B?:��-���������8�o��N�֓^�^.'��EL�e����4z6sڽ^�iz�^�S�� �r탇֠m����"ˆ�q���#,\���H�,g)<��t����Ԉ+֢�>�7�f,�ݨ���j-�Y��JLpEC�u<�5������V��2��՚/���ze�n^�E�47���^O���r b)�4^��B��Z%ֱ���1��ћ]��C�Hh�ݪM�����F?#��o"���%Ո�.tR�fP{��[���T�6�S��Iy��p����H��k^�e�����O��[U�����S;�β��+ݪ��.Dw?~h�27��F�t;�Ӹ/S���L�1�X�����\������/�~|�T莥JR�_�{ ���y3+��
��p���7�h|�c��&b���z�2��-S��3�� �J$�4��ɫUۊ�z\o��?��w�wd�n��O�q�ݺ��]ݻa!a�����[}�Nc.���1cI�[n ����Vv��t���0?��E�4�J�n�h�qԐ۹#M1�臭���Ku���� ��k��g&t��s���ػ���nyk݋�b�4i���fn�����{���T^�F�Z�V]� �{i��>{��/�Oy�l搗��2)wa|yd�֮k
w����N�]UaLme�
�V����q�폋#y��"V��d��2������K{'4Q�6�1��R^�:o�"&FQ�$U����
���)���ڔ
��Y�IW�H�#�-E��HK��6���I<:�ㄆlOW0A�"�1G�q��"�����nHc���p� �>~4ᣵ��-������7��N����a�Ys��n�1/���[�����A�i5z�^��k�7���߄-�R�4���&�u����Ӫ9�p�'�����6p�p��ߢ�����'���N{ܣ�n�ў�ߔ��y�j�s����ͳ��
��K�|8�׳�Ö�kc�i
����wXF�d���~{�������5�e�j�������-[#ӡ��Z���� �^�X���Ђ�B��3m��~IcU �f5(���4P��J��ERI�����FD4�<�:�~$�⧛�a�-�;/��Ƞ��&(m
��Ͱ��~a3�J�):�q�z�A�2��&�heTeT�§��]�?]>�E䣝j!ƣ�_�v�8-�t�i���s1��%��Up�M��C�����T�0^��\��(���.��q�9(���zK�v���h����T� "����2�hUPTɇf����<�M�Q����'���,bh)s���.Q���p�^�Dv"SۿC��r��qfpo9�&��:�%�-�W����ư�.U�u������ʜr�4a4T ���QE֎Qzw:F)}f���c��4�^��IX���,8��r�Y��.̳5��m�L#+���+a��Om����[= V��$Dx��'c���E����y�J�7�E��q�ٱ�S��I|�;Pkg8�Yۉ�X#�2�����NJ��2�y�6�Td�B�,�P� �*t4�猒,^�\�]pĎ��N�j��0�����ˌg���+�aNX�5���R��ld�֭97���ʢK덆�Ӥ���5~ ��[M�4X��X�u�ʼn+��n��
(���[�F'PS6�j��)�9��k*�e �����[�
����d����hO�#m��>�౩t���<h���ele!��L���M�V�3��+�٣F�=�/m*a�}Ly��M}��Z89\�o ��z�S߇�,w(��L{�qd#��FA�Y�o~iHۙ<B/l}#"H:��� �^���,c��n(
o�>�/I��c�㲨=��!WKm]���6q����M$6��Qۭ~_7�e�?�}��C,��HCR,���G�m�B��o�t�Uk-t�\QY��<�٧�e���E.��J�/!VC�����)�d�Y��
7/���ugm��#�-R\e�g��#����:2�|Y}�]���gk������Ag���X<�6�<�kj�M�n�ݹ�l�R�H��E�k/�h�K|��.q�.Q\|�Av���S�eQ�JR0���=�kR�뷌��F�]cu�m�v6����g*�� �++�-hP��ܺOY�zm4�?�^Z����z���tzǒ�����&s��;�E�Oгi�w~��5��L|���{���4����ܾAzno��ۍmJ�����o)��s��nB���7�3S���r{�2�;��;��vroI�V�rj/����2I�E�oed٭Z��|�6k+߬�ܣ�O�4���!���ӛ_�-��7'��IlsEd���6F�òo�,&����Q�����pz����[��}��O��HRi�lF�?���gŹ�0�E�E}�\͚�$�D�c>�x;��O��~���?���REn>�Կ$G~B��[?I�iT���Fe4� 1{��g+a��7���g+���uH���,��?����K�d�,1�{*B!��Bլ����C>r�����}�?�ޚ�BVs��{��u�3 �{2��_*_F
This kind of thing happens when there's an issue with the content encoding, which is typically down to compression settings.
Usually either the Content-Encoding header is invalid or missing, or the Vary one is. I don't know W3TC, but a quick search for "W3TC content encoding error" brought up a few results, so fortunately this one is an issue that has happened to a few people.
Apache default compression settings
Again I don't know W3TC, but from implementing similar caching setups, the first time it sees a request for a file which hasn't been cached yet, it'll build a .html file, compress it with something like gzip, and save it as a .html.gz file. Whenever a secondary request comes in, Apache can then directly serve up that static file as-is (knowing it's compressed already because of the file extension).
This issue occurs because it then outputs the gzip data to that first requester. By default, Apache compresses responses (unless it knows not to), so the result is it's being compressed twice.
So, the possible options:
Turn off Apache's default compression settings by disabling mod_deflate on your website (Assuming all your requests go through W3TC anyway, this is probably the route W3TC would expect)
Edit W3TC or your website by adding something like apache_setenv('no-gzip', '1'); which has the same effect as the above but is more controllable as to which requests it applies to
Turn off W3TC's compression (I wouldn't do this though; consider it a last resort!)

Force S3 multipart uploads

This is a follow-up to knt's question about PUT vs POST, with more details. The answer may be independently more useful to future answer-seekers.
can I use PUT instead of POST for uploading using fineuploader?
We have a mostly S3-compatible back-end that supports multipart upload, but not form POST, specifically policy signing. I see in the v5 migration notes that "even if chunking is enabled, a chunked upload request is only sent to traditional endpoints if the associated file must be broken into more than 1 chunk". How is the threshold determined for whether a file needs to be chunked? How can the threshold be adjusted? (or ideally, set to zero)
Thanks,
Fine Uploader will chunk a file if its size is less than the number of bytes specified in the chunking.partSize option (default value is: 2000000 bytes). If your file is smaller than the size specified in that value, then it will not be chunked.
To effectively set it to "zero", you could just increase the partSize to an extremely large value. I also did some experimenting, and it seems like a partSize of -1 will make Fine Uploader NOT chunk files at all. AFAIK, that is not supported behavior, and I have not looked at why that is even possible.
Note that S3 requires chunks to be a minimum of 5MB.
Also, note that you may run into limitations on request size as imposed by certain browsers if you make partSize extremely large.

Peeking inside the chrome protocol in Firefox

I was wondering whether it is possible to look inside the input stream
when the "chrome://" protocol is used in Firefox. Let me be more
clear. Let's take the following call sequence, for example:
nsXULDocument.cpp has a nsXULDocument::ResumeWalk() method.
It calls LoadScript() [around line:3004].
LoadScript() calls NS_NewStreamLoader [nsXULDocument.cpp, line
3440].
NS_NewStreamLoader calls NS_NewChannel() [nsNetUtil.h, line:593].
NS_NewChannel() then calls ioservice->NewChannelFromURI()
[nsNetUtil.h, line:226].
NewChannelFromURI() calls NewChannelFromURIWithProxyFlags()
[nsIOService.cpp line:596].
NewChannelFromURIWithProxyFlags() calls handler->newChannel() which
is resolved at runtime to become nsChromeProtocolHandler->newChannel()
[nsChromeProtocolHandler.cpp, line:182].
This in turn calls ioServ->NewChannelFromURI()
[nsChromeProtocolHandler.cpp, line:196].
Step 6 is repeated.
Step 7 is repeated, however, at different times, it can load
different handlers based on the protocol (chrome, jar, file etc.)
My intention for describing the above call sequence was to set up the
context for my problem. I want to know when "chrome://" protocol is
used, and when it is used, I want to process the input stream. For
example, if Firefox is loading a script like "chrome://package/content/
script.js" I want to intercept the file when it is accessed from the
disk. After intercepting the file, I might change it's contents, or
dump the file's content in a folder of my choice.
So, whenever Firefox reads a file (using a method like fread(),
probably, I would like to know that as well), I would like to determine whether the read request was from the chrome protocol or not, and at that moment I can make some changes to the file based on my needs. Any helps regarding this?
For those who've stumbled here curious about the 'chrome' protocol, here are some references that may be handy:
- Chrome Protocol, part of SPDY
- Lets make the web faster project

Maximum length of a String Preference in Firefox?

I would like to know what is the maximum length of a String when saving in the classic preferences System:
var prefs = Components.classes["#mozilla.org/preferences-service;1"]
.getService(Components.interfaces.nsIPrefBranch);
prefs.setCharPref("com.exemple.namespace.preference", potentiallyLongString);
Couldn't find it in official documentation.
Note: I tried to type in more than 255, it works on Firefox 3.6, but I'm looking for a documented answer which would certify that length L works since version V.
Since there was no documentation, I just though of trying it anyway (at the most it'd have crashed my browser). I tried to the extent of 1,51,276 characters - and it worked perfectly fine. Trust me, I even matched the characters to test the reliability. :) Of course, doesn't mean you should use it regularly to those extents. ;). Just to give you an idea, that'd be around 30,000 words of English language, and has more number of characters than the whole script of the Movie Matrix (part 1).
I didn't try for more, as Notepad++ had started becoming slow in copying and pasting those many characters.
(Test Environment: Firefox 24, Windows 7 64 bit)
Edit: While I was trying this, I also noticed that even though it works for enormously large values, after some 4000 characters, Firefox starts giving a performance warning in the Error Console. So take what you want to from this.
There's no documentation on that. You shouldn't store unreasonably large strings in prefs -- if you're not sure if it will be reasonable, it's likely not a pref.

Resources