How does one force Firefox to not cache a response? We've tried:
response.setHeader("Cache-Control", "no-cache, no-store, must-revalidate"); // HTTP 1.1.
response.setHeader("Pragma", "no-cache"); // HTTP 1.0.
response.setDateHeader("Expires", 0); // Proxies.
but in about:cache we still see:
Cache entry information
key: http://myhost.my.domain.com:42133/ducc-servlet/timestamp
fetch count: 4
last fetched: 2013-03-04 09:21:40
last modified: 2013-03-04 09:12:53
expires: 1969-12-31 19:00:00
Data size: 24
file on disk: none
Security: This document does not have any security info associated with it.
Client: HTTP
request-method: GET
response-head: HTTP/1.1 200 OK Content-Type: text/html;charset=UTF-8 Cache-Control: no-cache, no-store, must-revalidate
Date: Sun, 03 Mar 2013 14:12:57 GMT
Expires: Sun, 03 Mar 2013 14:12:57 GMT
Content-Length: 24
Server: Jetty(7.4.4.v20110707)
00000000: 32 30 31 33 2e 30 33 2e 30 34 20 30 39 3a 31 32 2013.03.04 09:12
00000010: 3a 35 36 20 4d 6f 6e 0a :56 Mon.
Thanks.
Lou.
Firefox will always (unless it runs out of space) store the data in its cache, so that it can do things like view source, save as, etc by just pulling the data from the cache. It might not use it for normal page loads, of course, if the data was set "no-cache".
If the data is no-store, it will put the cache entry in memory, not on disk, as requested.
Related
I'm working on a server-side HTTP/2 implementation.
Sometimes clients disconnect with a GOAWAY.COMPRESSION_ERROR in response to the server headers.
According to the Via header, in all of those cases, the client is an intermediary, 'Websense Web Security Gateway v7.6' (looks like a transparent caching solution)
The RFC says that COMPRESSION_ERROR can be sent in 2 cases.
1) A decoding error in a header block MUST be treated as a connection error of type COMPRESSION_ERROR
2) An endpoint receiving HEADERS, PUSH_PROMISE, or CONTINUATION frames needs to reassemble header blocks and perform decompression even if
the frames are to be discarded. A receiver MUST terminate the connection with a connection error of type COMPRESSION_ERROR if it does not decompress a header block
So in my case it's either an error on my side (bad headers encoding) or it's not an error but the client just discarded the response and sent a COMPRESSION_ERROR as specified by RFC.
If it's the latter then I don't need to fix anything.
But if it's the former then it's a bug in our implementation.
I'd like to rule out the bug case.
Below is a sample of such a session terminated with a COMPRESSION_ERROR.
It's hex dumps of data with my interpretation in {}.
S>C stands for server->client direction, C>S - client->server direction.
Can someone review the dump and tell me if it has some obvious issues with S>C frames or my interpretation of the data?
{preface stripped}
========== S>C frame: ============
00000000: 000024040000000000 ..$......
{type=SETTINGS stream=0 flags=NO_FLAGS(0) len=36 data={...}}
00000000: 00020000000000010000100000030000006400047fffffff000600007f380005 .................d...........8..
00000020: 00007ff7 ....
==================================
========== C>S frame: ============
00000000: 000012040000000000 .........
{type=SETTINGS stream=0 flags=NO_FLAGS(0) len=18 data={...}}
00000000: 000200000000000300000064000600008000 ...........d......
==================================
========== S>C frame: ============
00000000: 000000040100000000 .........
{type=SETTINGS stream=0 flags=ACK(1) len=0 data={}}
==================================
========== C>S frame: ============
00000000: 000004080000000000 .........
{type=WINDOW_UPDATE stream=0 flags=NO_FLAGS(0) len=4 data={inc=10485760}}
00000000: 00a00000 ....
==================================
========== C>S frame: ============
00000000: 000000040100000000 .........
{type=SETTINGS stream=0 flags=ACK(1) len=0 data={}}
==================================
========== C>S frame: ============
00000000: 00015c01050000000d ..\......
{type=HEADERS stream=13 flags=END_STREAM|END_HEADERS(5) len=348 data={...}}
00000000: 418aa4b2186cae824952fdea82449360d4cccb02005f603109a637e36a7c475e A....l..IR...D.`....._`1..7.j|G^
00000020: baa687539e352398ac782c75fd1a91cc56075d537d1a91cc563e7ebe58f9fbed ...S.5#..x,u....V.]S}...V>~.X...
00000040: 00177b73dc9d29ad171863c78f0ba0b54875fbd4c7f96a698a92c86b698072ca ..{s..)...c.....Hu....ji...ki.r.
00000060: 6a0c3497e2d4d3120f6d842c03210a8426c1c85f81cb2c4834903cefede170ed j.4......m.,.!..&.._..,H4.<...p.
00000080: 665435fb875c9b1a7b2dfceda35f7a67967e7ea3a19fde8dd02fdbf97d5e2ee9 fT5..\..{-..._zg.~~....../..}^..
000000a0: c97ad5d07f66a281b0dae053fae46aa43f8429a77a8102e0fb5396ae4e35fda9 .z...f.....S..j.?.).z....S..N5..
000000c0: 0d75d05e458f3192c36cbabb2e29fd66c7bf467fa5283752a988a4ea7fed4bd3 .u.^E.1..l...).f..F..(7R......K.
000000e0: d87a4ac3acae05d971e65708195370e51d8661b65d5d973f508d9bd9abfa5242 .zJ.....q.W..Sp...a.]].?P.....RB
00000100: cb40d25fa523b351b82d4b70ddf45abefb4005dffaee6fbed00177bebdcff7da .#._.#.Q.-Kp..Z..#....o...w.....
00000120: 002eefd7b9d6bda7efb4005dcfaf73ad8f37df6800bb7f5ee75b7f2fbed00176 ...........]..s..7.h...^.[./...v
00000140: bf7c99e458d05a9055391635371496d864fa5310d25f03f4a7775773 .|..X.Z.U9.57...d.S.._...wWs
==================================
========== S>C frame: ============
00000000: 00007c01040000000d ..|......
{type=HEADERS stream=13 flags=END_HEADERS(4) len=124 data={...}}
00000000: 3fe11f887684aa6355e76196c361be940bea435d8a08017d4006e32edc684a62 ?...v..cU.a..a....C]...}#....hJb
00000020: d1bf5f87352398ac5754df0f0d83644f3f6c96df3dbf4a09a5328ea50400bea0 .._.5#..WT....dO?l..=.J..2......
00000040: 1ab826ee32da98b46f628cfe5b234fb2566558923ffcff0f1596dd6d5f4a09e5 ..&.2...ob..[#O.VeX.?......m_J..
00000060: 21aec50400bea05ab8cbf700253168df588aa47e561cc581c034f001 !......Z....%1h.X..~V....4..
==================================
========== S>C frame: ============
00000000: 000cd900010000000d .........
{type=DATA stream=13 flags=END_STREAM(1) len=3289 data={...}}
{data stripped}
==================================
========== C>S frame: ============
00000000: 000014070000000000 .........
{type=GOAWAY stream=0 flags=NO_FLAGS(0) len=20 data={...}}
00000000: 0000000000000009000000000000033200000333 ...............2...3
{last_stream_id=0, error=COMPRESSION_ERROR}
==================================
========== S>C frame: ============
00000000: 000008070000000000 .........
{type=GOAWAY stream=0 flags=NO_FLAGS(0) len=8 data={...}}
00000000: 0000000d00000000 ........
==================================
C>S headers decoded:
:path: /img3/2019/1_24/TH4hGo.png HTTP/2.0
:authority: media3.scdn.vn
Accept: image/webp,image/apng,image/*,*/*;q=0.8
Referer: https://www.sendo.vn/?utm_medium=affiliate&utm_source=accesstrade&aff_sid=...
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,vi;q=0.8,zh;q=0.7,zh-CN;q=0.6,zh-HK;q=0.5,zh-TW;q=0.4
Via: Websense Web Security Gateway v7.6
S>C HEADERS frame decoded:
3fe11f
{set dyn table size 4096}
{dyn table size=0/4096}
88
{indexed header ix=8 :status: 200}
{dyn table not changed}
7684aa6355e7
{literal indexed header ix=54 Server: nginx }
{dyn table size=43/4096:
ix=62 off=0 size=43 Server: nginx
}
6196c361be940bea435d8a08017d4006e32edc684a62d1bf
{literal indexed header ix=33 Date: Fri, 19 Apr 2019 01:37:42 GMT }
{dyn table size=108/4096:
ix=62 off=0 size=65 Date: Fri, 19 Apr 2019 01:37:42 GMT
ix=63 off=65 size=43 Server: nginx
}
5f87352398ac5754df
{literal indexed header ix=31 Content-Type: image/png}
{dyn table size=161/4096:
ix=62 off=0 size=53 Content-Type: image/png
ix=63 off=53 size=65 Date: Fri, 19 Apr 2019 01:37:42 GMT
ix=64 off=118 size=43 Server: nginx
}
0f0d83644f3f
{literal not indexed header ix=28 Content-Length: 3289}
{dyn table not changed}
6c96df3dbf4a09a5328ea50400bea01ab826ee32da98b46f
{literal indexed header ix=44 Last-Modified: Thu, 24 Jan 2019 04:25:35 GMT }
{dyn table size=235/4096:
ix=62 off=0 size=74 Last-Modified: Thu, 24 Jan 2019 04:25:35 GMT
ix=63 off=74 size=53 Content-Type: image/png
ix=64 off=127 size=65 Date: Fri, 19 Apr 2019 01:37:42 GMT
ix=65 off=192 size=43 Server: nginx
}
628cfe5b234fb2566558923ffcff
{literal indexed header ix=34 ETag: "5c493e3f-cd9"}
{dyn table size=285/4096:
ix=62 off=0 size=50 ETag: "5c493e3f-cd9"
ix=63 off=50 size=74 Last-Modified: Thu, 24 Jan 2019 04:25:35 GMT
ix=64 off=124 size=53 Content-Type: image/png
ix=65 off=177 size=65 Date: Fri, 19 Apr 2019 01:37:42 GMT
ix=66 off=242 size=43 Server: nginx
}
0f1596dd6d5f4a09e521aec50400bea05ab8cbf700253168df
{literal not indexed header ix=36 Expires: Sun, 28 Apr 2019 14:39:02 GMT}
{dyn table not changed}
588aa47e561cc581c034f001
{literal indexed header ix=24 Cache-Control: max-age=604800}
{dyn table size=344/4096:
ix=62 off=0 size=59 Cache-Control: max-age=604800
ix=63 off=59 size=50 ETag: "5c493e3f-cd9"
ix=64 off=109 size=74 Last-Modified: Thu, 24 Jan 2019 04:25:35 GMT
ix=65 off=183 size=53 Content-Type: image/png
ix=66 off=236 size=65 Date: Fri, 19 Apr 2019 01:37:42 GMT
ix=67 off=301 size=43 Server: nginx
}
The decoded S>C headers:
:status: 200
Server: nginx
Date: Fri, 19 Apr 2019 01:37:42 GMT
Content-Type: image/png
Content-Length: 3289
Last-Modified: Thu, 24 Jan 2019 04:25:35 GMT
ETag: "5c493e3f-cd9"
Expires: Sun, 28 Apr 2019 14:39:02 GMT
Cache-Control: max-age=604800
I added an hpack decoder to h2get here: https://github.com/deweerdt/h2get/pull/18/files and i'm seeing the same results as you are:
$ ./hpack_decoder 3fe11f887684aa6355e76196c361be940bea435d8a08017d4006e32edc684a62d1bf5f87352398ac5754df0f0d83644f3f6c96df3dbf4a09a5328ea50400bea01ab826ee32da98b46f628cfe5b234fb2566558923ffcff0f1596dd6d5f4a09e521aec50400bea05ab8cbf700253168df588aa47e561cc581c034f001
dyn table resize to 4096
[:status] => [200]
[server] => [nginx]
[date] => [Fri, 19 Apr 2019 01:37:42 GMT]
[content-type] => [image/png]
[content-length] => [3289]
[last-modified] => [Thu, 24 Jan 2019 04:25:35 GMT]
[etag] => ["5c493e3f-cd9"]
[expires] => [Sun, 28 Apr 2019 14:39:02 GMT]
[cache-control] => [max-age=604800]
In Firefox 64 on macOS, when I view one of the disk cache entries under about:cache I see a page that looks something like the code I pasted below. I have deleted most of the long string of code after "security-info" and all but the first three lines of the hex and ascii codes at the end.
How can I restore the content of the file into a human readable format?
I have found quite a few solutions which require tools that only run under Windows, but I'm on macOS. I have also tried to save the hex code (e.g. for the first line: 00000000: 1f 8b 08 00 00 00 00 00 00 03 ed bd fb 72 db 48) into a file with UTF-8 encoding and UNIX linefeeds and .gzip as the file ending and then unpacking it, but all I got was the same hex code again. Do I need to delete the 00000000: at the beginning? Use another file format? Another file ending and/or decoder?
Cache entry information
key: https://www.some-site.com/
fetch count: 2
last fetched: 2018-12-16 20:17:21
last modified: 2018-12-17 06:31:53
expires: Expired Immediately
Data size: 17911 B
Security: This is a secure document.
strongly-framed: 1
security-info: FnhllAKWRHGAlo+ESXykKAAAAAAAAAAAwAAAAAAAAEap ... (and so on)
request-method: POST
request-Accept-Encoding: gzip, deflate, br
request-User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:64.0) Gecko/20100101 Firefox/64.0
response-head: HTTP/1.1 200 OK
Server: Server
Date: Sun, 16 Dec 2018 19:17:14 GMT
Content-Type: text/html; charset=utf-8
Status: 200 OK
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Request-Id: 2K84AFOOBARX4FNG9D4
ETag: W/"76e8cacfoobar0b8be5"
Cache-Control: max-age=0, private, must-revalidate
X-Runtime: 0.300085
X-Content-Type-Options: nosniff, nosniff
Content-Encoding: gzip
x-amz-rid: 2K84AFOOBARX4FNG9D4
Vary: Accept-Encoding,User-Agent
original-response-headers: Server: Server
Date: Sun, 16 Dec 2018 19:17:14 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Status: 200 OK
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Request-Id: 2K84AFOOBARX4FNG9D4
ETag: W/"76e8cacfoobar0b8be5"
Cache-Control: max-age=0, private, must-revalidate
X-Runtime: 0.300085
X-Content-Type-Options: nosniff
Content-Encoding: gzip
Set-Cookie: _session_id2=9f298foobarfb208b; path=/; expires=Mon, 17 Dec 2018 01:17:14 -0000; HttpOnly
x-amz-rid: 2K84AFOOBARX4FNG9D4
Vary: Accept-Encoding,User-Agent
net-response-time-onstart: 1266
net-response-time-onstop: 1274
00000000: 1f 8b 08 00 00 00 00 00 00 03 ed bd fb 72 db 48 .............r.H
00000010: 92 2f fc f7 f8 29 d0 9c d9 96 fc b5 78 27 75 b3 ./...)......x'u.
00000020: 25 1f 59 b6 dc ee 76 bb dd 96 3c 9e 6e af 83 01 %.Y...v...<.n...
... (and so on)
You need to extract the hexadecimal from between the colon and the dots or whatever on the right, and convert that back to binary. You will then have, in the case shown, a gzip stream that can be decompressed by gzip. You can use xxd -r -p to convert the hex into binary.
I'm using Spring reactive WebClient for sending requests to a http server. Inorder to view the underlying request & response that's being sent, I enabled debug logging for reactor.ipc.netty package.
The headers for the outgoing requests can be viewed normally.
Tho I'm sending & receiving plain text over http, the log contains the request & responses in the below format (is it hex?)
I'm not sure how to view the logged data in a easy to understand way. Better yet log the request & response in a understandable way
Here is a snippet of the logged data
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 47 45 54 20 2f 53 65 61 72 63 68 5f 47 43 2e 61 |GET /Search_GC.a|
|00000010| 73 70 78 20 48 54 54 50 2f 31 2e 31 0d 0a 75 73 |spx HTTP/1.1..us|
|00000020| 65 72 2d 61 67 65 6e 74 3a 20 52 65 61 63 74 6f |er-agent: Reacto|
|00000030| 72 4e 65 74 74 79 2f 30 2e 37 2e 32 2e 52 45 4c |rNetty/0.7.2.REL|
|00000040| 45 41 53 45 0d 0a 68 6f 73 74 3a 20 63 65 6f 6b |EASE..host: ceok|
|00000050| 61 72 6e 61 74 61 6b 61 2e 6b 61 72 2e 6e 69 63 |arnataka.kar.nic|
|00000060| 2e 69 6e 0d 0a 61 63 63 65 70 74 3a 20 2a 2f 2a |.in..accept: */*|
|00000070| 0d 0a 61 63 63 65 70 74 2d 65 6e 63 6f 64 69 6e |..accept-encodin|
|00000080| 67 3a 20 67 7a 69 70 0d 0a 63 6f 6e 74 65 6e 74 |g: gzip..content|
|00000090| 2d 6c 65 6e 67 74 68 3a 20 30 0d 0a 0d 0a |-length: 0.... |
+--------+-------------------------------------------------+----------------+
Found an unanswered question that must be happening because of the same library: Reading a HttpContent that has a PooledUnsafeDirectByteBuf
Raised an issue here
There seems to an orthodox view that debugging is not required for reactive clients. This is a completely pointless arguments as we use tools like rest client, postman, curl, httpie & others to send request and view response
They changed the reactor.netty.http.client.HttpClient class, after I upgraded to io.projectreactor.netty:reactor-netty-http:1.0.5 the following code is compilable and does what you expect. (I am not sure which is the minimal version, I upgraded from something older, but I guess it's 1.0.0. It is a transitive dependency, I upgraded spring-boot-starter-webflux from 2.3.4.RELEASE to 2.4.4.)
The crucial part is the call of wiretap():
wiretap("reactor.netty.http.client.HttpClient", LogLevel.DEBUG, AdvancedByteBufFormat.TEXTUAL, StandardCharsets.UTF_8)
It logs also the header and body of the request and of the response.
The whole context is this:
package com.example;
import io.netty.handler.logging.LogLevel;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.junit.jupiter.api.Test;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.http.client.reactive.ReactorClientHttpConnector;
import org.springframework.web.reactive.function.BodyInserters;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.publisher.Mono;
import reactor.netty.http.client.HttpClient;
import reactor.netty.transport.logging.AdvancedByteBufFormat;
import java.nio.charset.StandardCharsets;
#Slf4j
class RestClientTest {
private WebClient createWebClient() {
final HttpClient httpClient = HttpClient.create()
.wiretap(HttpClient.class.getCanonicalName(), LogLevel.DEBUG, AdvancedByteBufFormat.TEXTUAL, StandardCharsets.UTF_8);
return WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(httpClient))
.build();
}
#Data
#Builder
#NoArgsConstructor
#AllArgsConstructor
private static class User {
int id;
int userId;
String title;
String body;
}
#Test
void createUsersReactive() {
final WebClient webClient = createWebClient();
final String url = "http://jsonplaceholder.typicode.com/posts";
final Mono<User> userMono = webClient.method(HttpMethod.POST)
.uri(url)
.contentType(MediaType.APPLICATION_JSON_UTF8)
.header("X-My-Header", "MyValue1", "MyValue2")
.body(BodyInserters.fromValue(User.builder().userId(1).title("foo").body("bar").build()))
.retrieve()
.bodyToMono(User.class);
final User user = userMono.block();
log.info("Created user: " + user);
}
}
And the output to log is human readable as you request:
... reactor.netty.http.client.HttpClient : [id:e7d7ed93] REGISTERED
... reactor.netty.http.client.HttpClient : [id:e7d7ed93] CONNECT: jsonplaceholder.typicode.com/<unresolved>:80
... reactor.netty.http.client.HttpClient : [id:e7d7ed93] ACTIVE
... r.netty.http.client.HttpClientConnect : [id:e7d7ed93-1] Handler is being applied: {uri=http://jsonplaceholder.typicode.com/posts, method=POST}
... reactor.netty.http.client.HttpClient : [id:e7d7ed93-1] WRITE: 217B POST /posts HTTP/1.1
user-agent: ReactorNetty/1.0.5
host: jsonplaceholder.typicode.com
accept: */*
Content-Type: application/json;charset=UTF-8
X-My-Header: MyValue1
X-My-Header: MyValue2
content-length: 46
... reactor.netty.http.client.HttpClient : [id:e7d7ed93-1] WRITE: 46B {"id":0,"userId":1,"title":"foo","body":"bar"}
... reactor.netty.http.client.HttpClient : [id:e7d7ed93-1] FLUSH
... reactor.netty.http.client.HttpClient : [id:e7d7ed93-1] READ: 1347B HTTP/1.1 201 Created
Date: Tue, 13 Apr 2021 12:49:33 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 65
X-Powered-By: Express
X-Ratelimit-Limit: 1000
X-Ratelimit-Remaining: 999
X-Ratelimit-Reset: 1618318233
Vary: Origin, X-HTTP-Method-Override, Accept-Encoding
Access-Control-Allow-Credentials: true
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
Access-Control-Expose-Headers: Location
Location: http://jsonplaceholder.typicode.com/posts/101
X-Content-Type-Options: nosniff
Etag: W/"41-0LtsWqhuQ7Zsjlj0tYnOrT/Vw5o"
Via: 1.1 vegur
CF-Cache-Status: DYNAMIC
cf-request-id: 096ce0bd560000736722853000000001
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?s=laAKjgGcoi8SLu%2F6VX5pQIAksdmj9xi31elC5Ld97eljznKIpYjdkQsittoMJp3lJoQIwOACmj89bKSa%2Ff15gRHRmyasV2Xcl%2FmVjJBJm7ytbWocp39UBd90JwVM"}],"max_age":604800,"group":"cf-nel"}
NEL: {"max_age":604800,"report_to":"cf-nel"}
Server: cloudflare
CF-RAY: 63f4d0a88ed07367-CPH
alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Set-Cookie: __cfduid=d11c86fbd953f7cf768cf7db0c346f22b1618318173; expires=Thu, 13-May-21 12:49:33 GMT; path=/; domain=.typicode.com; HttpOnly; SameSite=Lax
{
"id": 101,
"userId": 1,
"title": "foo",
"body": "bar"
}
... r.n.http.client.HttpClientOperations : [id:e7d7ed93-1] Received response (auto-read:false) : [Date=Tue, 13 Apr 2021 12:49:33 GMT, Content-Type=application/json; charset=utf-8, X-Powered-By=Express, X-Ratelimit-Limit=1000, X-Ratelimit-Remaining=999, X-Ratelimit-Reset=1618318233, Vary=Origin, X-HTTP-Method-Override, Accept-Encoding, Access-Control-Allow-Credentials=true, Cache-Control=no-cache, Pragma=no-cache, Expires=-1, Access-Control-Expose-Headers=Location, Location=http://jsonplaceholder.typicode.com/posts/101, X-Content-Type-Options=nosniff, Etag=W/"41-0LtsWqhuQ7Zsjlj0tYnOrT/Vw5o", Via=1.1 vegur, CF-Cache-Status=DYNAMIC, cf-request-id=096ce0bd560000736722853000000001, Report-To={"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?s=laAKjgGcoi8SLu%2F6VX5pQIAksdmj9xi31elC5Ld97eljznKIpYjdkQsittoMJp3lJoQIwOACmj89bKSa%2Ff15gRHRmyasV2Xcl%2FmVjJBJm7ytbWocp39UBd90JwVM"}],"max_age":604800,"group":"cf-nel"}, NEL={"max_age":604800,"report_to":"cf-nel"}, Server=cloudflare, CF-RAY=63f4d0a88ed07367-CPH, alt-svc=h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400, Proxy-Connection=Keep-Alive, Connection=Keep-Alive, Set-Cookie=__cfduid=d11c86fbd953f7cf768cf7db0c346f22b1618318173; expires=Thu, 13-May-21 12:49:33 GMT; path=/; domain=.typicode.com; HttpOnly; SameSite=Lax, content-length=65]
... r.n.http.client.HttpClientOperations : [id:e7d7ed93-1] Received last HTTP packet
... reactor.netty.http.client.HttpClient : [id:e7d7ed93] READ COMPLETE
... com.example.RestClientTest : Created user: RestClientIT.User(id=101, userId=1, title=foo, body=bar)
You can do it with doOnNext(), if you use DataBuffer as your reader:
public Mono<ServerResponse> selectByPost(ServerRequest request) {
Flux<DataBuffer> requestBodyFlux = request.bodyToFlux(DataBuffer.class)
.doOnNext(dataBuffer -> {
if (debug ) {
log.debug(new String(dataBuffer.asByteBuffer().array()));
}
Scannable.from(dataBuffer).tags().forEach(System.out::println);
});
}
This is probably not the best way to do it, it would of course be a nice feature, if netty would provide different ways of logging the payload. Hex does have its benefits, depending on what you need to debug.
Seems like the responding server is returning gzipped content, so it makes sense that you're not able to read it.
If you really want to intercept at the raw HTTP level, ensure your request header does not specify it can accept GZipped content (accept-encoding: gzip).
Another alternative may be to log the request at another layer, when it's already been unzipped from the raw data stream, but not yet processed by your application code - not sure how this would work in Reactive webclient though ;)
I create one proxy cache using (tor + privoxy + squid) follow
How to create an advanced proxy server using Tor, Privoxy, and Squid
I try call to Proxy cache
curl -x 'my-ip:3400' -I 'https://github.global.ssl.fastly.net/images/modules/dashboard/bootcamp/octocat_setup.png'
curl -x 'my-ip:3400' -I http://hadoop.apache.org/images/hadoop-logo.jpg
https request is ok (through Pivoxy)
HTTP/1.1 200 Connection established
Proxy-Agent: Privoxy/3.0.21
HTTP/1.1 200 OK
Date: Sat, 08 Mar 2014 04:38:37 GMT
Server: GitHub.com
Content-Type: image/png
Last-Modified: Fri, 16 Aug 2013 09:42:18 GMT
Expires: Sun, 08 Mar 2015 04:38:37 GMT
Cache-Control: max-age=31536000, public
Timing-Allow-Origin: https://github.com
Content-Length: 10722
Accept-Ranges: bytes
Via: 1.1 varnish
Age: 0
X-Served-By: cache-am72-AMS
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1394253517.449177980,VS0,VE97
Vary: Accept-Encoding
but http requets not working
HTTP/1.0 200 OK
Date: Sat, 08 Mar 2014 04:39:19 GMT
Server: Apache/2.4.8-dev (Unix) mod_wsgi/3.4 Python/2.7.5 OpenSSL/1.0.1e
Last-Modified: Tue, 12 Feb 2008 17:38:57 GMT
ETag: "24e3-445f987fe69d8"
Accept-Ranges: bytes
Content-Length: 9443
Content-Type: image/jpeg
X-Cache: MISS from SVR226L-696.localdomain
X-Cache-Lookup: MISS from SVR226L-696.localdomain:3128
Via: 1.0 SVR226L-696.localdomain (squid)
Connection: keep-alive
Please suggest a specific solution
I have an aspx web page with various Telerik RadControls on it and each time this page is loaded, Safari mobile seems to download all the resource files again - for example the following files are downloaded on each page request to an ipad - nothing seems to get cached as it would be on other non-mobile browsers.
http://aspnet-scripts.telerikstatic.com/ajaxz/2010.3.1109/Common/Navigation/NavigationScripts.js
http://aspnet-scripts.telerikstatic.com/ajaxz/2010.3.1109/PanelBar/RadPanelBarScripts.js
http://aspnet-scripts.telerikstatic.com/ajaxz/2010.3.1109/Input/TextBox/RadInputScript.js
http://aspnet-scripts.telerikstatic.com/ajaxz/2010.3.1109/ListBox/RadListBoxScripts.js
http://aspnet-scripts.telerikstatic.com/ajaxz/2010.3.1109/ComboBox/RadComboBoxScripts.js
http://aspnet-scripts.telerikstatic.com/ajaxz/2010.3.1109/Grid/RadGridScripts.js
This is over 100K of javascript being passed down each time even though the CDN resources are set to cache and have ETAGS.
I've rigged up fiddler to both the ipad and a laptop running IE/FF/Safari each time the laptop browser is caching the resource files above and the ipad isn't.
I think this is why my app performs notably slower on the Ipad although it may be the slow javascipt parsing on the Ipad as well...
Has anyone else had this problem? Am i missing something obvious here?
Thanks.
EDIT
I am monitoring the iPad traffic on the laptop by making the laptop an http proxy for the iPad.
Extra Info from fiddler:
Fiddler v2.3.0.0, laptop results from IE8, iPad vers. 3.2.2.
From my laptop, If I go to http://demos.telerik.com/aspnet-ajax/listbox/examples/default/defaultcs.aspx
and after click "First Look" a few times the following in in fiddler:
# Result Protocol Host URL Body Caching Content-Type Process Comments Custom
1 200 HTTP demos.telerik.com /aspnet-ajax/listbox/examples/default/defaultcs.aspx 54,608 private text/html; charset=utf-8 iexplore:6172
2 200 HTTP demos.telerik.com /aspnet-ajax/Common/CodeViewer.js.aspx 5,534 private text/javascript; charset=utf-8 iexplore:6172
4 200 HTTP demos.telerik.com /aspnet-ajax/listbox/examples/default/defaultcs.aspx 54,608 private text/html; charset=utf-8 iexplore:6172
5 200 HTTP demos.telerik.com /aspnet-ajax/Common/CodeViewer.js.aspx 5,534 private text/javascript; charset=utf-8 iexplore:6172
7 200 HTTP demos.telerik.com /aspnet-ajax/listbox/examples/default/defaultcs.aspx 54,608 private text/html; charset=utf-8 iexplore:6172
8 200 HTTP demos.telerik.com /aspnet-ajax/Common/CodeViewer.js.aspx 5,534 private text/javascript; charset=utf-8 iexplore:6172
10 200 HTTP demos.telerik.com /aspnet-ajax/listbox/examples/default/defaultcs.aspx 54,608 private text/html; charset=utf-8 iexplore:6172
11 200 HTTP demos.telerik.com /aspnet-ajax/Common/CodeViewer.js.aspx 5,534 private text/javascript; charset=utf-8 iexplore:6172
If I do the same on my iPad the following is recorded by fiddler:
# Result Protocol Host URL Body Caching Content-Type Process Comments Custom
1 200 HTTP demos.telerik.com /aspnet-ajax/listbox/examples/default/defaultcs.aspx 54,380 private text/html; charset=utf-8
2 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/Common/Navigation/NavigationScripts.js 7,856 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
3 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/ComboBox/RadComboBoxScripts.js 16,238 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
4 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/Ajax/Ajax.js 7,509 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
5 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/ToolBar/RadToolBarScripts.js 13,372 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
6 200 HTTP demos.telerik.com /aspnet-ajax/Common/CodeViewer.js.aspx 5,534 private text/javascript; charset=utf-8
8 200 HTTP demos.telerik.com /aspnet-ajax/listbox/examples/default/defaultcs.aspx 54,380 private text/html; charset=utf-8
9 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/Common/Navigation/NavigationScripts.js 7,856 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
10 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/ComboBox/RadComboBoxScripts.js 16,238 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
11 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/ListBox/RadListBoxScripts.js 13,199 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
12 200 HTTP demos.telerik.com /aspnet-ajax/Common/CodeViewer.js.aspx 5,534 private text/javascript; charset=utf-8
13 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/ToolBar/RadToolBarScripts.js 13,372 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
15 200 HTTP demos.telerik.com /aspnet-ajax/listbox/examples/default/defaultcs.aspx 54,380 private text/html; charset=utf-8
16 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/Common/Navigation/NavigationScripts.js 7,856 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
17 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/ComboBox/RadComboBoxScripts.js 16,238 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
18 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/ToolBar/RadToolBarScripts.js 13,372 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
19 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/ListBox/RadListBoxScripts.js 13,199 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
20 200 HTTP aspnet-scripts.telerikstatic.com /ajaxz/2010.3.1317/Ajax/Ajax.js 7,509 public, max-age=31536000 Expires: Sun, 17 Jan 2021 15:05:25 GMT application/x-javascript
21 200 HTTP demos.telerik.com /aspnet-ajax/Common/CodeViewer.js.aspx 5,534 private text/javascript; charset=utf-8
EDIT 2:
I have tried exactly the same test as above again today and noticed that:
1) The IPad is now caching the javascript
2) Telerik demos are not using the CDN but they were yesterday! (see results below compared to results above)
# Result Protocol Host URL Body Caching Content-Type Process Comments Custom
1 200 HTTP demos.telerik.com /aspnet-ajax/listbox/examples/default/defaultcs.aspx 53,036 private text/html; charset=utf-8
2 200 HTTP demos.telerik.com /aspnet-ajax/Telerik.Web.UI.WebResource.axd?compress=1&_TSM_CombinedScripts_=%3b%3bTelerik.Web.UI%2c+Version%3d2010.3.1317.40%2c+Culture%3dneutral%2c+PublicKeyToken%3d121fae78165ba3d4%3aen-US%3a9506cff2-3a86-46c5-b869-6ba1478f3c27%3a1c2121e%3ae8945d51%3af4636213%3ac531f12c%3a4d471440%3af7131199%3ac17ca3a3 10,599 public, max-age=31533429 Expires: Sat, 18 Feb 2012 10:06:39 GMT text/css
3 200 HTTP demos.telerik.com /aspnet-ajax/Telerik.Web.UI.WebResource.axd?_TSM_HiddenField_=RadScriptManager1_TSM&compress=1&_TSM_CombinedScripts_=%3b%3bSystem.Web.Extensions%2c+Version%3d4.0.0.0%2c+Culture%3dneutral%2c+PublicKeyToken%3d31bf3856ad364e35%3aen-US%3a8f393b2b-3315-402f-b504-cd6d2db001f6%3aea597d4b%3ab25378d2%3bTelerik.Web.UI%3aen-US%3a9506cff2-3a86-46c5-b869-6ba1478f3c27%3a16e4e7cd%3af7645509%3a24ee1bba%3af46195d3%3a1e771326%3aaa288e2d%3a628ebdd0%3a9cdfc6e7%3aed16cbdc%3a2bef5fcc%3aa7e79140 160,387 public, max-age=31533429 Expires: Sat, 18 Feb 2012 10:06:40 GMT application/x-javascript
4 200 HTTP demos.telerik.com /aspnet-ajax/Common/CodeViewer.js.aspx 5,534 private text/javascript; charset=utf-8
10 200 HTTP demos.telerik.com /aspnet-ajax/listbox/examples/default/defaultcs.aspx 53,032 private text/html; charset=utf-8
11 200 HTTP demos.telerik.com /aspnet-ajax/Common/CodeViewer.js.aspx 5,534 private text/javascript; charset=utf-8
14 200 HTTP demos.telerik.com /aspnet-ajax/listbox/examples/default/defaultcs.aspx 53,032 private text/html; charset=utf-8
15 200 HTTP demos.telerik.com /aspnet-ajax/Common/CodeViewer.js.aspx 5,534 private text/javascript; charset=utf-8
18 200 HTTP demos.telerik.com /aspnet-ajax/listbox/examples/default/defaultcs.aspx 53,032 private text/html; charset=utf-8
19 200 HTTP demos.telerik.com /aspnet-ajax/Common/CodeViewer.js.aspx 5,534 private text/javascript; charset=utf-8
I've just tested it with Fiddler - opened a RadListBox on a page with version 2010.3.1109 on an iPad (iOS 4.2) and it seems to me the CDN resources are cached properly and are not refreshed from the CDN every time as you are experiencing.
Can you share some more details about this issue, like a sample page or a live URL where we can test it? You can open a support ticket in your Telerik account to send those if you prefer the privacy.
EDIT:
I've also tested with 3.2.2 and I still can't reproduce this issue. Are you using some specific iPad settings or network setup worth mentioning?