I'm trying to load log text files from a ftp server to elastic .
The log files look like this :
0:0:21: Processing events from events
0:0:21: Processing croned build types from q_type
0:0:21: Process croned releases from trls
0:0:22: Processing croned regression list from regression
0:0:22: Processing commit loop
in data provenance (hex view , because other views not showing anything)i see the data like this :
0x00000090 66 69 65 6C 64 3A 20 52 4E 20 53 74 61 74 75 73 field: RN Status
0x000000A0 2E 20 4F 62 6A 65 63 74 20 72 65 66 65 72 65 6E . Object referen
0x000000B0 63 65 20 6E 6F 74 20 73 65 74 20 74 6F 20 61 6E ce not set to an
0x000000C0 20 69 6E 73 74 61 6E 63 65 20 6F 66 20 61 6E 20 instance of an
0x000000D0 6F 62 6A 65 63 74 2E 0D 0A 30 3A 30 3A 31 34 3A object...0:0:14:
0x000000E0 20 43 61 6E 27 74 20 72 65 61 64 20 69 73 73 75 Can't read issu
0x000000F0 65 3A 20 41 49 2D 32 34 37 20 63 75 73 74 6F 6D e: AI-247 custom
0x00000100 20 66 69 65 6C 64 3A 20 52 4E 20 53 65 63 74 69 field: RN Secti
0x00000110 6F 6E 2E 20 4F 62 6A 65 63 74 20 72 65 66 65 72 on. Object refer
0x00000120 65 6E 63 65 20 6E 6F 74 20 73 65 74 20 74 6F 20 ence not set to
0x00000130 61 6E 20 69 6E 73 74 61 6E 63 65 20 6F 66 20 61 an instance of a
0x00000140 6E 20 6F 62 6A 65 63 74 2E 0D 0A 30 3A 30 3A 31 n object...0:0:1
0x00000150 34 3A 20 43 61 6E 27 74 20 72 65 61 64 20 69 73 4: Can't read is
0x00000160 73 75 65 3A 20 41 49 2D 32 34 37 20 63 75 73 74 sue: AI-247 cust
0x00000170 6F 6D 20 66 69 65 6C 64 3A 20 52 4E 20 44 6F 63 om field: RN Doc
0x00000180 20 69 6E 20 56 65 72 2E 20 4F 62 6A 65 63 74 20 in Ver. Object
0x00000190 72 65 66 65 72 65 6E 63 65 20 6E 6F 74 20 73 65 reference not se
0x000001A0 74 20 74 6F 20 61 6E 20 69 6E 73 74 61 6E 63 65 t to an instance
0x000001B0 20 6F 66 20 61 6E 20 6F 62 6A 65 63 74 2E 0D 0A of an object...
0x000001C0 30 3A 30 3A 31 34 3A 20 43 61 6E 27 74 20 72 65 0:0:14: Can't re
0x000001D0 61 64 20 69 73 73 75 65 3A 20 41 49 2D 32 34 37 ad issue: AI-247
I can get the file with "getftp" processor, but how do I convert it to json so I can send it to Elastic ?
I am new to nifi hope im not missing something basic, any help will be appreciated.
Thanks
You can use the ConvertRecord processor with a CSVReader for the input (configure to use : as the delimiter) and a JsonRecordSetWriter for the output.
NiFi can automatically infer the schema, but as it doesn't appear you have a header line for the incoming data, this will probably not be helpful. In that case, you can use the Schema Registry to hold two schemas -- one for the incoming log lines, indicating what each field should be called and the data type, and one for the JSON output. Bryan Bende has written a great article about this process.
Related
The hexadecimal file is from the charles proxy's hex format, and it may contains invisible characters, the example format content is:
00000000 7b 22 73 75 70 70 6f 72 74 73 5f 69 6d 70 6c 69 {"supports_impli
00000010 63 69 74 5f 73 64 6b 5f 6c 6f 67 67 69 6e 67 22 cit_sdk_logging"
00000020 3a 74 72 75 65 2c 22 67 64 70 76 34 5f 6e 75 78 :true,"gdpv4_nux
00000030 5f 65 6e 61 62 6c 65 64 22 3a 66 61 6c 73 65 2c _enabled":false,
00000040 22 61 6e 64 72 6f 69 64 5f 73 64 6b 5f 65 72 72 "android_sdk_err
00000050 6f 72 5f 63 61 74 65 67 6f 72 69 65 73 22 3a 5b or_categories":[
00000060 7b 22 6e 61 6d 65 22 3a 22 6c 6f 67 69 6e 5f 72 {"name":"login_r
00000070 65 63 6f 76 65 72 61 62 6c 65 22 2c 22 69 74 65 ecoverable","ite
00000080 6d 73 22 3a 5b 7b 22 63 6f 64 65 22 3a 31 30 32 ms":[{"code":102
00000090 7d 2c 7b 22 63 6f 64 65 22 3a 31 39 30 7d 5d 2c },{"code":190}],
000000a0 22 72 65 63 6f 76 65 72 79 5f 6d 65 73 73 61 67 "recovery_messag
000000b0 65 22 3a 22 5c 75 38 62 66 37 5c 75 39 31 63 64 e":"\u8bf7\u91cd
000000c0 5c 75 36 35 62 30 5c 75 37 36 37 62 5c 75 35 66 \u65b0\u767b\u5f
000000d0 35 35 5c 75 35 65 39 34 5c 75 37 35 32 38 5c 75 55\u5e94\u7528\u
000000e0 37 61 30 62 5c 75 35 65 38 66 5c 75 66 66 30 63 7a0b\u5e8f\uff0c
000000f0 5c 75 35 31 38 64 5c 75 36 62 32 31 5c 75 38 66 \u518d\u6b21\u8f
00000100 64 65 5c 75 36 33 61 35 20 46 61 63 65 62 6f 6f de\u63a5 Faceboo
00000110 6b 20 5c 75 35 65 31 30 5c 75 36 32 33 37 5c 75 k \u5e10\u6237\u
00000120 33 30 30 32 22 7d 5d 2c 22 61 70 70 5f 65 76 65 3002"}],"app_eve
00000130 6e 74 73 5f 73 65 73 73 69 6f 6e 5f 74 69 6d 65 nts_session_time
00000140 6f 75 74 22 3a 36 30 2c 22 61 70 70 5f 65 76 65 out":60,"app_eve
00000150 6e 74 73 5f 66 65 61 74 75 72 65 5f 62 69 74 6d nts_feature_bitm
00000160 61 73 6b 22 3a 36 35 35 35 39 2c 22 73 65 61 6d ask":65559,"seam
00000170 6c 65 73 73 5f 6c 6f 67 69 6e 22 3a 31 2c 22 73 less_login":1,"s
00000180 6d 61 72 74 5f 6c 6f 67 69 6e 5f 62 6f 6f 6b 6d mart_login_bookm
00000190 61 72 6b 5f 69 63 6f 6e 5f 75 72 6c 22 3a 22 68 ark_icon_url":"h
000001a0 74 74 70 73 3a 5c 2f 5c 2f 73 74 61 74 69 63 2e ttps:\/\/static.
000001b0 78 78 2e 66 62 63 64 6e 2e 6e 65 74 5c 2f 72 73 xx.fbcdn.net\/rs
000001c0 72 63 2e 70 68 70 5c 2f 76 33 5c 2f 79 73 5c 2f rc.php\/v3\/ys\/
000001d0 72 5c 2f 43 36 5a 75 74 59 44 53 61 61 56 2e 70 r\/C6ZutYDSaaV.p
000001e0 6e 67 22 2c 22 73 6d 61 72 74 5f 6c 6f 67 69 6e ng","smart_login
000001f0 5f 6d 65 6e 75 5f 69 63 6f 6e 5f 75 72 6c 22 3a _menu_icon_url":
00000200 22 68 74 74 70 73 3a 5c 2f 5c 2f 73 74 61 74 69 "https:\/\/stati
00000210 63 2e 78 78 2e 66 62 63 64 6e 2e 6e 65 74 5c 2f c.xx.fbcdn.net\/
00000220 72 73 72 63 2e 70 68 70 5c 2f 76 33 5c 2f 79 73 rsrc.php\/v3\/ys
00000230 5c 2f 72 5c 2f 30 69 61 72 70 6e 77 64 6d 45 78 \/r\/0iarpnwdmEx
00000240 2e 70 6e 67 22 2c 22 72 65 73 74 72 69 63 74 69 .png","restricti
00000250 76 65 5f 64 61 74 61 5f 66 69 6c 74 65 72 5f 70 ve_data_filter_p
00000260 61 72 61 6d 73 22 3a 22 7b 7d 22 2c 22 61 61 6d arams":"{}","aam
00000270 5f 72 75 6c 65 73 22 3a 22 7b 7d 22 2c 22 73 75 _rules":"{}","su
00000280 67 67 65 73 74 65 64 5f 65 76 65 6e 74 73 5f 73 ggested_events_s
00000290 65 74 74 69 6e 67 22 3a 22 7b 5c 22 70 72 6f 64 etting":"{\"prod
000002a0 75 63 74 69 6f 6e 5f 65 76 65 6e 74 73 5c 22 3a uction_events\":
000002b0 5b 5d 2c 5c 22 65 6c 69 67 69 62 6c 65 5f 66 6f [],\"eligible_fo
000002c0 72 5f 70 72 65 64 69 63 74 69 6f 6e 5f 65 76 65 r_prediction_eve
000002d0 6e 74 73 5c 22 3a 5b 5c 22 66 62 5f 6d 6f 62 69 nts\":[\"fb_mobi
000002e0 6c 65 5f 61 64 64 5f 74 6f 5f 63 61 72 74 5c 22 le_add_to_cart\"
000002f0 2c 5c 22 66 62 5f 6d 6f 62 69 6c 65 5f 70 75 72 ,\"fb_mobile_pur
00000300 63 68 61 73 65 5c 22 2c 5c 22 66 62 5f 6d 6f 62 chase\",\"fb_mob
00000310 69 6c 65 5f 63 6f 6d 70 6c 65 74 65 5f 72 65 67 ile_complete_reg
00000320 69 73 74 72 61 74 69 6f 6e 5c 22 2c 5c 22 66 62 istration\",\"fb
00000330 5f 6d 6f 62 69 6c 65 5f 69 6e 69 74 69 61 74 65 _mobile_initiate
00000340 64 5f 63 68 65 63 6b 6f 75 74 5c 22 5d 7d 22 2c d_checkout\"]}",
00000350 22 69 64 22 3a 22 31 36 33 35 34 33 35 31 34 39 "id":"1635435149
00000360 30 39 30 34 35 22 7d 09045"}
Read the file and convert the hexadecimal content to golang slice []byte, is there any convenient method? how to split the lines in the file?
I have solved it by using fixed length of the hexadecimal content. the offset length of each line is 8, so the hexadecimal content begins at offset 8, and the max length of the hexadecimal is 57. The code is as follows:
const (
beginOffset = 8 // the offset of the hexadecimal content begins
endLength = 57 // the length of the hexadecimal content ends
)
func ReadHexData(src []byte) ([]byte, error) {
lines := strings.Split(string(src), "\n")
var data string
for _, line := range lines {
for i := beginOffset; i < endLength; i++ {
if line[i] != ' ' {
data = data + string(line[i])
}
}
}
return hex.DecodeString(data)
}
I used ffmpeg and libx264 to encode my video,when I print the AVPacket data,I found there is something only exist before the first IDR.Could someone tell me what's that,thanks.
0 0 1 6 5 ff ff 73 dc 45 e9 bd e6 d9 48 b7 96 2c d8 20 d9 23 ee ef 78 32 36 34 20 2d 20 63 6f 72 65 20 31 34 38 20 72 32 36 34 33 20 35 63 36 35 37 30 34 20 2d 20 48 2e 32 36 34 2f 4d 50 45 47 2d 34 20 41 56 43 20 63 6f 64 65 63 20 2d 20 43 6f 70 79 6c 65 66 74 20 32 30 30 33 2d 32 30 31 35 20 2d 20 68 74 74 70 3a 2f 2f 77 77 77 2e 76 69 64 65 6f 6c 61 6e 2e 6f 72 67 2f 78 32 36 34 2e 68 74 6d 6c 20 2d 20 6f 70 74 69 6f 6e 73 3a 20 63 61 62 61 63 3d 31 20 72 65 66 3d 31 20 64 65 62 6c 6f 63 6b 3d 31 3a 30 3a 30 20 61 6e 61 6c 79 73 65 3d 30 78 33 3a 30 78 31 31 33 20 6d 65 3d 68 65 78 20 73 75 62 6d 65 3d 32 20 70 73 79 3d 31 20 70 73 79 5f 72 64 3d 31 2e 30 30 3a 30 2e 30 30 20 6d 69 78 65 64 5f 72 65 66 3d 30 20 6d 65 5f 72 61 6e 67 65 3d 31 36 20 63 68 72 6f 6d 61 5f 6d 65 3d 31 20 74 72 65 6c 6c 69 73 3d 30 20 38 78 38 64 63 74 3d 31 20 63 71 6d 3d 30 20 64 65 61 64 7a 6f 6e 65 3d 32 31 2c 31 31 20 66 61 73 74 5f 70 73 6b 69 70 3d 31 20 63 68 72 6f 6d 61 5f 71 70 5f 6f 66 66 73 65 74 3d 30 20 74 68 72 65 61 64 73 3d 34 20 6c 6f 6f 6b 61 68 65 61 64 5f 74 68 72 65 61 64 73 3d 34 20 73 6c 69 63 65 64 5f 74 68 72 65 61 64 73 3d 31 20 73 6c 69 63 65 73 3d 34 20 6e 72 3d 30 20 64 65 63 69 6d 61 74 65 3d 31 20 69 6e 74 65 72 6c 61 63 65 64 3d 30 20 62 6c 75 72 61 79 5f 63 6f 6d 70 61 74 3d 30 20 63 6f 6e 73 74 72 61 69 6e 65 64 5f 69 6e 74 72 61 3d 30 20 62 66 72 61 6d 65 73 3d 30 20 77 65 69 67 68 74 70 3d 31 20 6b 65 79 69 6e 74 3d 31 32 20 6b 65 79 69 6e 74 5f 6d 69 6e 3d 31 20 73 63 65 6e 65 63 75 74 3d 34 30 20 69 6e 74 72 61 5f 72 65 66 72 65 73 68 3d 30 20 72 63 3d 61 62 72 20 6d 62 74 72 65 65 3d 30 20 62 69 74 72 61 74 65 3d 32 30 30 30 20 72 61 74 65 74 6f 6c 3d 31 2e 30 20 71 63 6f 6d 70 3d 30 2e 36 30 20 71 70 6d 69 6e 3d 30 20 71 70 6d 61 78 3d 36 39 20 71 70 73 74 65 70 3d 34 20 69 70 5f 72 61 74 69 6f 3d 31 2e 34 30 20 61 71 3d 31 3a 31 2e 30 30 0 80
That is user_data_unregistered SEI with libx264 version info and params. And if you look at it as text you will see: x264 - core 148 r2643 5c65704 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=1 keyint=12 keyint_min=1 scenecut=40 intra_refresh=0 rc=abr mbtree=0 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
I want to send an http delete request with custom header.
I tried to do this with Spring's WebClient, with following code:
#Test
public void validateDeleteCouldCarryHeader() {
WebClient.create("https://jira.spring.io/")
.delete()
.header("X-FOO", "BAR")
.retrieve()
.bodyToMono(Map.class)
.block();
}
I am expecting something like:
DELETE / HTTP/1.1
X-FOO: BAR
...
content-length: 0
But, the actual request made is:
DELETE / HTTP/1.1
user-agent: ReactorNetty/0.7.0.RELEASE
host: jira.spring.io
accept: */*
accept-encoding: gzip
content-length: 0
Did I miss some thing to include the custom header X-FOO ?
Update 1: I upgraded reactor-netty to 0.7.3.RELEASE,
And the missing header is still missing:
DELETE / HTTP/1.1
user-agent: ReactorNetty/0.7.3.RELEASE
host: jira.spring.io
accept: */*
accept-encoding: gzip
content-length: 0
2018-12-28 22:32:32.813 DEBUG 12064 --- [ctor-http-nio-4] r.ipc.netty.http.client.HttpClient : [id: 0x2c6a9cea, L:/172.17.1.131:54532 - R:jira.spring.io/35.199.60.33:443] WRITE: 138B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 44 45 4c 45 54 45 20 2f 20 48 54 54 50 2f 31 2e |DELETE / HTTP/1.|
|00000010| 31 0d 0a 75 73 65 72 2d 61 67 65 6e 74 3a 20 52 |1..user-agent: R|
|00000020| 65 61 63 74 6f 72 4e 65 74 74 79 2f 30 2e 37 2e |eactorNetty/0.7.|
|00000030| 33 2e 52 45 4c 45 41 53 45 0d 0a 68 6f 73 74 3a |3.RELEASE..host:|
|00000040| 20 6a 69 72 61 2e 73 70 72 69 6e 67 2e 69 6f 0d | jira.spring.io.|
|00000050| 0a 61 63 63 65 70 74 3a 20 2a 2f 2a 0d 0a 61 63 |.accept: */*..ac|
|00000060| 63 65 70 74 2d 65 6e 63 6f 64 69 6e 67 3a 20 67 |cept-encoding: g|
|00000070| 7a 69 70 0d 0a 63 6f 6e 74 65 6e 74 2d 6c 65 6e |zip..content-len|
|00000080| 67 74 68 3a 20 30 0d 0a 0d 0a |gth: 0.... |
+--------+-------------------------------------------------+----------------+
Update 2: I found there's a reactor's BOM (Bismuth-RELEASE) imported in my pom, after removed that BOM, Spring Boot 2.0.5.RELEASE included the custom header I want.
DELETE / HTTP/1.1
user-agent: ReactorNetty/0.7.9.RELEASE
host: jira.spring.io
accept: */*
accept-encoding: gzip
X-FOO: BAR
Reactor Netty 0.7.0.RELEASE is quite old now, and you should upgrade to a newer version.
I've tried to reproduce this and couldn't.
Setting the log level of reactor.ipc.netty.http.client.HttpClient to "DEBUG" shows this:
[reactor-http-nio-4] DEBUG reactor.ipc.netty.http.client.HttpClient - [id: 0x69202b84, L:/192.168.0.28:60880 - R:jira.spring.io/35.199.60.33:443] WRITE: 150B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 44 45 4c 45 54 45 20 2f 20 48 54 54 50 2f 31 2e |DELETE / HTTP/1.|
|00000010| 31 0d 0a 75 73 65 72 2d 61 67 65 6e 74 3a 20 52 |1..user-agent: R|
|00000020| 65 61 63 74 6f 72 4e 65 74 74 79 2f 30 2e 37 2e |eactorNetty/0.7.|
|00000030| 30 2e 52 45 4c 45 41 53 45 0d 0a 68 6f 73 74 3a |0.RELEASE..host:|
|00000040| 20 6a 69 72 61 2e 73 70 72 69 6e 67 2e 69 6f 0d | jira.spring.io.|
|00000050| 0a 61 63 63 65 70 74 3a 20 2a 2f 2a 0d 0a 61 63 |.accept: */*..ac|
|00000060| 63 65 70 74 2d 65 6e 63 6f 64 69 6e 67 3a 20 67 |cept-encoding: g|
|00000070| 7a 69 70 0d 0a 58 2d 46 4f 4f 3a 20 42 41 52 0d |zip..X-FOO: BAR.|
|00000080| 0a 63 6f 6e 74 65 6e 74 2d 6c 65 6e 67 74 68 3a |.content-length:|
|00000090| 20 30 0d 0a 0d 0a | 0.... |
+--------+-------------------------------------------------+----------------+
So the client is indeed sending those headers. Maybe there's something wrong with the way you're capturing the outgoing request information?
I'm implementing a small web server into a wifi micro. To aid in development and test, I have ported it to Windows console program.
I use chunked transfer processing. The following is what shows up on the browser:
0059
Hello World
0
The 59 is the hex size of the chunk and the 0 is the chunked terminating size
This is the data captured via wireshark:
This is the first message I send which are the headers
0000 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK.
0010 0a 53 65 72 76 65 72 3a 20 54 72 61 6e 73 66 65 .Server: Transfe
0020 72 2d 45 6e 63 6f 64 69 6e 67 3a 20 63 68 75 6e r-Encoding: chun
0030 6b 65 64 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 ked..Content-Typ
0040 65 3a 20 74 65 78 74 2f 68 74 6d 6c 0d 0a 43 61 e: text/html..Ca
0050 63 68 65 2d 43 6f 6e 74 72 6f 6c 3a 20 6d 61 78 che-Control: max
0060 2d 61 67 65 3d 33 36 30 30 2c 20 6d 75 73 74 2d -age=3600, must-
0070 72 65 76 61 6c 69 64 61 74 65 0d 0a 0d 0a revalidate....
The next block is the chunked data
0000 30 30 35 39 0d 0a 3c 68 74 6d 6c 3e 0a 3c 68 65 0059..<html>.<he
0010 61 64 3e 3c 74 69 74 6c 65 3e 57 65 62 20 53 65 ad><title>Web Se
0020 72 76 65 72 3c 2f 74 69 74 6c 65 3e 0a 3c 2f 68 rver</title>.</h
0030 65 61 64 3e 0a 3c 62 6f 64 79 3e 0a 3c 68 31 3e ead>.<body>.<h1>
0040 48 65 6c 6c 6f 20 57 6f 72 6c 64 3c 2f 68 31 3e Hello World</h1>
0050 0a 3c 2f 62 6f 64 79 3e 3c 2f 68 74 6d 6c 3e 0d .</body></html>.
0060 0a 30 0d 0a 0d 0a .0....
The chunked values are being displayed on both Chrome and IE.
Can anyone see an issue with my data that would cause the issue.
Thanks
Solved:
I mistakenly remove the server name so now the browser is taking the transfer encoding as the server name and does not understand the chunked message size -- it thinks its just data to display.
I am trying to connect to the reference websocket echo server "manually", in order to learn how the protocol works (I am using socat for that). However, the server invariably closes the connection without providing an answer. Any idea why?
Here is what I do:
socat - TCP:echo.websocket.org:80
Then, I paste the following text in the terminal:
GET /?encoding=text HTTP/1.1
Origin: http://www.websocket.org
Connection: Upgrade
Host: echo.websocket.org
Sec-WebSocket-Key: P7Kp2hTLNRPFMGLxPV47eQ==
Upgrade: websocket
Sec-WebSocket-Version: 13
I sniffed the parameters of the connection with the developer tools, in firefox, on the same machine, where this works flawlessly: therefore, I would assume they are correct. However after that, the server closes the connection immediately, without providing an answer. Why? How can I implement the protocol "manually"?
I would like type test in my terminal and get the server to reply with what I typed (It works in a web browser).
I think you want to modify the socket stream to translate \n (line feed) to CRLF (Carriage return & line feed). Doing info socat produces detailed information which includes this modifier:
crnl Converts the default line termination character NL ('\n', 0x0a)
to/from CRNL ("\r\n", 0x0d0a) when writing/reading on this chan-
nel (example). Note: socat simply strips all CR characters.
So I think you should be able to do this:
socat - TCP:echo.websocket.org:80,crnl
I'd like to add that my WebSocket tool websocat can help in debugging the WebSocket protocol, especially when combined with socat:
$ websocat - ws-c:sh-c:"socat -v -x - tcp:echo.websocket.org:80" --ws-c-uri ws://echo.websocket.org
> 2018/07/03 16:30:06.021658 length=157 from=0 to=156
47 45 54 20 2f 20 48 54 54 50 2f 31 2e 31 0d 0a GET / HTTP/1.1..
48 6f 73 74 3a 20 65 63 68 6f 2e 77 65 62 73 6f Host: echo.webso
63 6b 65 74 2e 6f 72 67 0d 0a cket.org..
43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 55 70 67 72 Connection: Upgr
61 64 65 0d 0a ade..
55 70 67 72 61 64 65 3a 20 77 65 62 73 6f 63 6b Upgrade: websock
65 74 0d 0a et..
53 65 63 2d 57 65 62 53 6f 63 6b 65 74 2d 56 65 Sec-WebSocket-Ve
72 73 69 6f 6e 3a 20 31 33 0d 0a rsion: 13..
53 65 63 2d 57 65 62 53 6f 63 6b 65 74 2d 4b 65 Sec-WebSocket-Ke
79 3a 20 59 76 36 32 44 31 57 6d 7a 79 79 31 65 y: Yv62D1Wmzyy1e
69 6d 62 47 6d 68 69 61 67 3d 3d 0d 0a imbGmhiag==..
0d 0a ..
--
< 2018/07/03 16:30:06.164057 length=201 from=0 to=200
48 54 54 50 2f 31 2e 31 20 31 30 31 20 57 65 62 HTTP/1.1 101 Web
20 53 6f 63 6b 65 74 20 50 72 6f 74 6f 63 6f 6c Socket Protocol
20 48 61 6e 64 73 68 61 6b 65 0d 0a Handshake..
43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 55 70 67 72 Connection: Upgr
61 64 65 0d 0a ade..
44 61 74 65 3a 20 54 75 65 2c 20 30 33 20 4a 75 Date: Tue, 03 Ju
6c 20 32 30 31 38 20 31 33 3a 31 35 3a 30 30 20 l 2018 13:15:00
47 4d 54 0d 0a GMT..
53 65 63 2d 57 65 62 53 6f 63 6b 65 74 2d 41 63 Sec-WebSocket-Ac
63 65 70 74 3a 20 55 56 6a 32 74 35 50 43 7a 62 cept: UVj2t5PCzb
58 49 32 52 4e 51 75 70 2f 71 48 31 63 5a 44 6e XI2RNQup/qH1cZDn
38 3d 0d 0a 8=..
53 65 72 76 65 72 3a 20 4b 61 61 7a 69 6e 67 20 Server: Kaazing
47 61 74 65 77 61 79 0d 0a Gateway..
55 70 67 72 61 64 65 3a 20 77 65 62 73 6f 63 6b Upgrade: websock
65 74 0d 0a et..
0d 0a ..
--
ABCDEF
> 2018/07/03 16:30:12.707919 length=13 from=157 to=169
82 87 40 57 f5 88 01 15 b6 cc 05 11 ff ..#W.........
--
< 2018/07/03 16:30:12.848398 length=9 from=201 to=209
82 07 41 42 43 44 45 46 0a ..ABCDEF.
--
ABCDEF
> 2018/07/03 16:30:14.528333 length=6 from=170 to=175
88 80 18 ec 05 a8 ......
--
< 2018/07/03 16:30:14.671629 length=2 from=210 to=211
88 00 ..
--
In case of failures with manually driven socat -v -x - TCP:echo.websocket.org:80,crnl (mentioned in the other answer), you can compare it with WebSocat-driven socat like in session depicted above.
Reverse (server) example with socat debug dump:
socat -v -x tcp-l:1234,fork,reuseaddr exec:'websocat -t ws-u\:stdio\: mirror\:'
Alternatively, here is a way to connect and read the stream from a wss secure websocket stream from the command line using solely core php.
php -r '$sock=stream_socket_client("tls://echo.websocket.org:443",$e,$n,30,STREAM_CLIENT_CONNECT,stream_context_create(null));if(!$sock){echo"[$n]$e".PHP_EOL;}else{fwrite($sock,"GET / HTTP/1.1\r\nHost: echo.websocket.org\r\nAccept: */*\r\nConnection: Upgrade\r\nUpgrade: websocket\r\nSec-WebSocket-Version: 13\r\nSec-WebSocket-Key: ".rand(0,999)."\r\n\r\n");while(!feof($sock)){var_dump(fgets($sock,2048));}}'
Other similar example, pulling from another wss server: (Do not get rekt)
php -r '$sock=stream_socket_client("tls://stream.binance.com:9443",$e,$n,30,STREAM_CLIENT_CONNECT,stream_context_create(null));if(!$sock){echo"[$n]$e".PHP_EOL;}else{fwrite($sock,"GET /stream?streams=btcusdt#kline_1m HTTP/1.1\r\nHost: stream.binance.com:9443\r\nAccept: */*\r\nConnection: Upgrade\r\nUpgrade: websocket\r\nSec-WebSocket-Version: 13\r\nSec-WebSocket-Key: ".rand(0,999)."\r\n\r\n");while(!feof($sock)){var_dump(explode(",",fgets($sock,512)));}}'