CSV file can have data with new line . it can be with any column. Also some line can have data without any new line so it should work in all case
Sample input
ID,username,mobile,city,Message,Address,city
'11111111',TestUSer,1234567890,test,"Hi how are you? Well: we will connnect
Thanks for your time!
With Joy.
Test",Address test,City test
11111116,TestUser,1234567891,test,hello msg,Address test1,City test1
'111111167',TestUSer,1234567890,test,"Hi how are you one? Well: we will connnect
Thanks for your time!
With Joy.
Test",Address test,City test
11111112,TestUser,1234567891,test1,hello msg1,Address test2,City test2
11111113,TestUser,1234567891,test1,hello msg1,Address test2,City test2
11111114,TestUser,1234567891,test1,hello msg1,Address test2,City test2
I am using below command to read top 5 record of csv
awk -v RS='("[^"]*")?\r?\n' 'NF{ORS = gensub(/\r?\n(.)/, "\\\\n\\1", "g", RT); ++n; print} n==5{exit}' file.csv
Actual output:
ID,username,mobile,city,Message,Address,city
'11111111',TestUSer,1234567890,test,"Hi how are you? Well: we will connnect\nThanks for your time!\nWith Joy.\Test",Address test,City test
11111116,TestUser,1234567891,test,hello msg,Address test1,City test1
'111111167',TestUSer,1234567890,test,"Hi how are you one? Well: we will connnect\nThanks for your time!\nWith Joy.\nTest",Address test,City test
11111112,TestUser,1234567891,test1,hello msg1,Address test2,City test2
11111113,TestUser,1234567891,test1,hello msg1,Address test2,City test2
11111114,TestUser,1234567891,test1,hello msg1,Address test2,City test2
Wanted output:
ID,username,mobile,city,Message,Address,city
'11111111',TestUSer,1234567890,test,"Hi how are you? Well: we will connnect\nThanks for your time!\nWith Joy.\Test",Address test,City test
11111116,TestUser,1234567891,test,hello msg,Address test1,City test1
'111111167',TestUSer,1234567890,test,"Hi how are you one? Well: we will connnect\nThanks for your time!\nWith Joy.\nTest",Address test,City test
11111112,TestUser,1234567891,test1,hello msg1,Address test2,City test2
With your shown samples only, could you please try following awk code. Written and tested with GNU awk. Make use of RS record separator and then substitute globally to nullify new lines in RT, then print the lines accordingly.
awk -v RS='"[^"]*"' '{gsub(/\n/,"\\n",RT);ORS=RT} 1' Input_file
To get first 10 records, try following:
awk -v RS='"[^"]*"' '{gsub(/\n/,"\\n",RT);ORS=RT} 1' Input_file | head -10
Warning: Self promotion ahead!
I wrote an awk-like utility called tawk that uses tcl as its scripting language, and has a mode that reads CSV data without having to hack around with regular expressions to handle records with embedded newlines and quotes (This feature was actually my major inspiration for it).
Using it:
$ tawk -csv 'line {$NR <= 5} { puts [regsub -all {\n+} $F(0) "\\n"]; if {$NR == 5} exit }' input.csv
ID,username,mobile,city,Message,Address,city
'11111111',TestUSer,1234567890,test,"Hi how are you? Well: we will connnect\nThanks for your time!\nWith Joy.\nTest",Address test,City test
11111116,TestUser,1234567891,test,hello msg,Address test1,City test1
'111111167',TestUSer,1234567890,test,"Hi how are you one? Well: we will connnect\nThanks for your time!\nWith Joy.\nTest",Address test,City test
11111112,TestUser,1234567891,test1,hello msg1,Address test2,City test2
How can I search for empty curly braces in kibana ?
e.g.
I have some messages in elasticsearch:
test1 1235512 {"command":"session","params":{}} test1
test2 1235512 {"command":"session","params":{"k1": "p1"}} test2
test3 1235512 {"command":"session","params":{"k2": "p2"}} test3 df
test4 1235512 {"command":"session","params":{}} test4 some more text
I need to find all documents where exists empty curly braces {}. For this case in response I want to get test1 and test4 documents. Or if someone can give the elasticsearch request it also can be very helpful )
If you just wanna search on kibana, why not simply try this?
works for me.
This may be a very noob question, but I have no idea of how to solve it. I have a C program and a folder called "input". Is it possible to write rules that allow me to type "make run" and then execute all test instances, writing the result into a single file?
I imagine that it's something like:
run: test1 test2 test3
test1:
./myproj < arq1.in > output.out
test2:
./myproj < arq2.in > output.out
...
But how do I achieve that everyone writes into the same file, ie, appending the results, instead of rewriting it? Is it possible?
Thanks in advance
First switch to ">>" to append to the output file:
run: test1 test2 test3
test1:
./myproj < arq1.in >> output.out
test2:
./myproj < arq2.in >> output.out
test3:
./myproj < arq3.in >> output.out
Then make those targets PHONY, because they are (i.e. they do not build files named "test1", "test2" and "test3"):
.PHONY: test1 test2 test3
Then switch to a static pattern rule:
run: test1 test2 test3
.PHONY: test1 test2 test3
test1 test2 test3: test%:
./myproj < arq$*.in >> output.out
Further refinements are possible, but that's enough for now.