Log values of query parameters in Spring Data R2DBC? - spring

In Spring Data R2DBC I can log SQL queries by using
logging.level.org.springframework.data.r2dbc=DEBUG
in the application.properties.
However, this doesn't log the actual values that are bound as query parameters.
How can I log the actual values of query parameters in Spring Data R2DBC?

This worked for me :
logging:
level:
io.r2dbc.postgresql.QUERY: DEBUG # for queries
io.r2dbc.postgresql.PARAM: DEBUG # for parameters
I found it here

For h2 and perhaps other DBs try:
logging.level.io.r2dbc.h2=TRACE

If you happen to use MySQL, you can get a low level network dump of all communication with the database server by setting the driver's log level to TRACE.
logging.level.ROOT=TRACE
logging.level.dev.miku.r2dbc.mysql.client.ReactorNettyClient=TRACE
Example output:
19-11-2020 00:03:28.072 TRACE [reactor-tcp-nio-2] d.m.r.mysql.client.ReactorNettyClient - [id: 0x9cf29125, L:/127.0.0.1:63807 - R:localhost/127.0.0.1:33306] READ: 71B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 01 00 00 01 01 1b 00 00 02 03 64 65 66 00 00 00 |..........def...|
|00000010| 05 4e 4f 57 28 29 00 0c 3f 00 13 00 00 00 0c 81 |.NOW()..?.......|
|00000020| 00 00 00 00 14 00 00 03 13 32 30 32 30 2d 31 31 |.........2020-11|
|00000030| 2d 31 39 20 30 31 3a 30 33 3a 32 38 07 00 00 04 |-19 01:03:28....|
|00000040| fe 00 00 02 00 00 00 |....... |
+--------+-------------------------------------------------+----------------+

In SpringBoot v2.7.5 make this:
logging.level.org.springframework.r2dbc=TRACE
logging.level.org.springframework.r2dbc=DEBUG

i have to update source code to log parameterize sql statement in mysql r2dbc .
this is my approach :
Index: src/main/java/dev/miku/r2dbc/mysql/message/client/TextQueryMessage.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/main/java/dev/miku/r2dbc/mysql/message/client/TextQueryMessage.java b/src/main/java/dev/miku/r2dbc/mysql/message/client/TextQueryMessage.java
--- a/src/main/java/dev/miku/r2dbc/mysql/message/client/TextQueryMessage.java (revision bdba9eebe79976aa4bb75aa0aad25c7cbbdf4dec)
+++ b/src/main/java/dev/miku/r2dbc/mysql/message/client/TextQueryMessage.java (revision 1b5e0133cc8d43bfc32133912d9e3589374ae75a)
## -20,6 +20,8 ##
import dev.miku.r2dbc.mysql.Parameter;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufAllocator;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import reactor.core.publisher.Mono;
import java.nio.charset.Charset;
## -42,6 +44,8 ##
private final Consumer<String> sqlProceed;
+ private static final Logger logger = LoggerFactory.getLogger(TextQueryMessage.class);
+
public TextQueryMessage(List<String> sqlParts, Parameter[] values, Consumer<String> sqlProceed) {
requireNonNull(sqlParts, "sql parts must not be null");
requireNonNull(values, "values must not be null");
## -64,6 +68,10 ##
try {
Charset charset = context.getClientCollation().getCharset();
return ParamWriter.publish(sqlParts, values).map(sql -> {
+
+ if(logger.isDebugEnabled()){
+ logger.debug("parametered sql statement[{}]",sql);
+ }
sqlProceed.accept(sql);
ByteBuf buf = allocator.buffer(sql.length(), Integer.MAX_VALUE);
then change the log level of TextQueryMessage in logback-spring.xml:
<logger name="dev.miku.r2dbc.mysql.message.client.TextQueryMessage" level="DEBUG" additivity="false">
<appender-ref ref="async_file"/>
</logger>

Related

Getting ADF from PPSE by android Pay applicatons

I'm trying to read a smartcard information by android pay application. So I send the following SELECT PPSE command:
00 A4 04 00 0E 32 50 41 59 2E 53 59 53 2E 44 44 46 30 31 00
I get the following response:
That is, I need to get an ADF <A0 00 00 06 58 10 10>, but when I try to get it by SELECT:
00 A4 04 00 07 A0 00 00 06 58 10 10
I get the following response:
SW1SW2 = 6A82: File not found.
The tag 9F2A in PPSE means that I need to use Kernel specification (Kernel 1 for me).
But the documentation talks about actions from the moment when there is already a PDOL.
How do I get the PDOL if the ADF in which it is located is not found?

bash extract certain lines from file

I have file like this:
[some_lines_unimportant]
#Field List (hex, dec, bin, char)
00 00 1e 14 0 0 30 20 999909999 000090000 0001-1110
(...)
00 00 3e 01 0 0 62 01 888888888 000088000 0100-1000
[some_lines_unimportant]
I would like to be able to extract f.e 3 lines that are under the #Field list ... line. How can I do that in bash.
EDITED:
this is the file:
# ------------------------------------------------------------------------------------
# ======= Pkt to be transmitted (hex) ================================================
01 00 fe 03 4d 00 00 00 00 02 fe 00 aa 00 00 00 00 00 00 00 04 b8 EOP
# Initiating transfer ...
# ======= Received pkt (hex) =========================================================
02 fe 03 0d 00 fe 00 aa 00 00 00 04 84 00 01 00 10 cc EOP
# ========= Reply pkt fields (hex) ===================================================
Reply SpW Addr:
02
# Initiator Log Addr (1 byte), Proto ID (1 byte), Instr (1 byte), Status (1 byte):
fe 03 0d 00
# Target Log Addr (1 byte), Trans ID (2 bytes MS LS), Reserved (1 byte):
fe 00 aa 00
# Data Length (3 bytes MS to LS), Header CRC (1 byte):
00 00 04 84
# Field List (hex, dec, bin, char, [dec line count]):
00 01 00 10 0 1 0 16 0000-0000 0000-0001 0000-0000 0001-0000 . . . . [ 1]
# Data CRC (1 byte):
cc
# End of Pkt character:
EOP
# ------------------------------------------------------------------------------------
# ######### Transaction time: 0.225 ms
######### End of online test #########################################################
Use grep like this, where -A specifies how many lines AFTER the match:
grep -A 3 "^#Field List" file
#Field List (hex, dec, bin, char)
00 00 1e 14 0 0 30 20 999909999 000090000 0001-1110
(...)
00 00 3e 01 0 0 62 01 888888888 000088000 0100-1000
Or, if you don't want the first line that matches
grep -A 3 "^#Field List" file | sed 1d
00 00 1e 14 0 0 30 20 999909999 000090000 0001-1110
(...)
00 00 3e 01 0 0 62 01 888888888 000088000 0100-1000
The circumflex (or hat, ^) matches the start of the line, i.e. the word #Field must be at the very start of the line for it to be matched.
Use grep with the --after-context/-A argument:
grep -A3 '#Field List'

Serial RS232 Communication Checksums?

I am communicating with a servo via RS232 serial. The built-in functions that came with my servo are too slow (25 ms for a simple 54 byte message on a 57,600 baud port), so I am trying to write my own communication functions, however the built-in functions are not documented. I have used a port monitor to determine what information is being sent to the servo and I need help deciphering the results.
I used the built-in functions to command the servo to "goto" incrementally increasing steps (1, 2, 3, etc.). This resulted 5 packets being sent to the servo for each "goto" command. The first 4 packets are identical for each "goto" command. I have attached about 50 hex packet below (1 per line). If you need more, post, and we can work something out.
10 13 04 20 00 01 B6 24 E9 68
10 13 04 20 00 00 AE 24 54 82
10 13 04 20 00 00 B5 24 8B 0B
10 13 04 20 00 01 43 01 71 9B
The 5th packet varies based on the step the motor is being commanded to move to. I have included 1 packet here as an example. I have attached a file with about 1000 of these packets (1 per line).
10 13 08 20 03 01 11 25 0A 00 00 00 81 CF
The first 8 bytes of this packet (10 13 08 20 03 01 11 25) appear to be the actual "goto" command. They remain the same no matter what step is specified.
The last 6 bytes (0A 00 00 00 81 CF) change based upon the step that is requested. In the file I attached, I instructed the servo to initially goto step "0", then "1", "2", etc. The first 4 bytes appear to be a little-endian integer corresponding to the number of steps (i.e. the sample command I showed above instructs the servo to goto step 10 decimal).
My question regards the last 2 bytes of the command. They appear to vary randomly, but whenever the specified step is the same they match. This leads me to believe that these 2 bytes are a checksum of some kind. My question to you is: how is the checksum calculated?
I have already tried xor'ing all the bytes, both singly and in 2 byte pairs, and I tried Fletcher's checksum, and a simple checksum (sum of all bytes). I also checked the 2's complement of each of these methods (though I certainly wouldn't mind someone checking to make sure I didn't make a mistakes in the calculations). Does anyone have any ideas?
10 13 08 20 03 01 11 25 00 00 00 00 E9 64
10 13 08 20 03 01 11 25 01 00 00 00 9F D0
10 13 08 20 03 01 11 25 02 00 00 00 04 0C
10 13 08 20 03 01 11 25 04 00 00 00 23 95
10 13 08 20 03 01 11 25 05 00 00 00 55 21
10 13 08 20 03 01 11 25 06 00 00 00 CE FD
10 13 08 20 03 01 11 25 07 00 00 00 B8 49
10 13 08 20 03 01 11 25 08 00 00 00 6C A7
10 13 08 20 03 01 11 25 09 00 00 00 1A 13
10 13 08 20 03 01 11 25 0A 00 00 00 81 CF
10 13 08 20 03 01 11 25 0C 00 00 00 A6 56
10 13 08 20 03 01 11 25 0D 00 00 00 D0 E2
10 13 08 20 03 01 11 25 0F 00 00 00 3D 8A
10 13 08 20 03 01 11 25 10 10 00 00 00 17 FA
10 13 08 20 03 01 11 25 11 00 00 00 84 77
10 13 08 20 03 01 11 25 12 00 00 00 1F AB
10 13 08 20 03 01 11 25 13 00 00 00 69 1F
10 13 08 20 03 01 11 25 14 00 00 00 38 32
10 13 08 20 03 01 11 25 15 00 00 00 4E 86
10 13 08 20 03 01 11 25 16 00 00 00 D5 5A
10 13 08 20 03 01 11 25 17 00 00 00 A3 EE
10 13 08 20 03 01 11 25 18 00 00 00 77 00
10 13 08 20 03 01 11 25 19 00 00 00 01 B4
10 13 08 20 03 01 11 25 1A 00 00 00 9A 68
10 13 08 20 03 01 11 25 1B 00 00 00 EC DC
10 13 08 20 03 01 11 25 1C 00 00 00 BD F1
10 13 08 20 03 01 11 25 1D 00 00 00 CB 45
10 13 08 20 03 01 11 25 1E 00 00 00 50 99
10 13 08 20 03 01 11 25 1F 00 00 00 26 2D
10 13 08 20 03 01 11 25 20 00 00 00 DE 2A
10 13 08 20 03 01 11 25 21 00 00 00 A8 9E
10 13 08 20 03 01 11 25 22 00 00 00 33 42
10 13 08 20 03 01 11 25 24 00 00 00 14 DB
10 13 08 20 03 01 11 25 25 00 00 00 62 6F
10 13 08 20 03 01 11 25 26 00 00 00 F9 B3
10 13 08 20 03 01 11 25 27 00 00 00 8F 07
10 13 08 20 03 01 11 25 28 00 00 00 5B E9
10 13 08 20 03 01 11 25 29 00 00 00 2D 5D
10 13 08 20 03 01 11 25 2A 00 00 00 B6 81
10 13 08 20 03 01 11 25 2B 00 00 00 C0 35
10 13 08 20 03 01 11 25 2C 00 00 00 91 18
10 13 08 20 03 01 11 25 2D 00 00 00 E7 AC
10 13 08 20 03 01 11 25 2E 00 00 00 7C 70
10 13 08 20 03 01 11 25 2F 00 00 00 0A C4
10 13 08 20 03 01 11 25 30 00 00 00 C5 8D
10 13 08 20 03 01 11 25 31 00 00 00 B3 39
10 13 08 20 03 01 11 25 32 00 00 00 28 E5
10 13 08 20 03 01 11 25 33 00 00 00 5E 51
10 13 08 20 03 01 11 25 34 00 00 00 0F 7C
10 13 08 20 03 01 11 25 35 00 00 00 79 C8
10 13 08 20 03 01 11 25 36 00 00 00 E2 14
10 13 08 20 03 01 11 25 37 00 00 00 94 A0
10 13 08 20 03 01 11 25 38 00 00 00 40 4E
10 13 08 20 03 01 11 25 39 00 00 00 36 FA
10 13 08 20 03 01 11 25 3A 00 00 00 AD 26
10 13 08 20 03 01 11 25 3B 00 00 00 DB 92
10 13 08 20 03 01 11 25 3C 00 00 00 8A BF
10 13 08 20 03 01 11 25 3D 00 00 00 FC 0B
10 13 08 20 03 01 11 25 3E 00 00 00 67 D7
10 13 08 20 03 01 11 25 3F 00 00 00 11 63
this is a late answer, but hopefully this can help for other CRC re-engineering tasks:
Your CRC is a derivation of the so-called "16 bit width CRC as designated by CCITT", but with "init value zero".
The CRC is calculated from byte position 3 to byte position 12 of your example data. e.g.
08 20 03 01 11 25 00 00 00 00
The full CRC specification according to our CRC specification overview is:
CRC:16,1021,0000,0000,No,No
The problem was not only to find the right CRC polynomial, but finding the following answers:
Which part of the data is included in the CRC calculation, and which is not.
Which init value to use? Apply final xor value?
Does this algorithm expect reflected input or output values?
Again, see our manual description or the Boost CRC library on what this means.
What I did is running a brute-force script that simply tries out several popular 16 bit CRC polynomials with all kinds of combinations of start/end positions, initial values, reflected versions. Here is how the processing output looked:
Finding CRC for test message (HEX): 10 13 08 20 03 01 11 25 00 00 00 00 E9 64
Trying CRC spec : CRC:16,1021,FFFF,0000,No,No
Trying CRC spec : CRC:16,8005,0000,0000,No,No
Trying CRC spec : CRC:16,8005,FFFF,0000,No,No
Trying CRC spec : CRC:16,1021,FFFF,FFFF,No,No
Trying CRC spec : CRC:16,1021,0000,FFFF,No,No
Trying CRC spec : CRC:16,1021,0000,0000,No,No
Found it!
Relevant sequence for checksum from startpos=3 to endpos=12
08 20 03 01 11 25 00 00 00 00
CRC spec: CRC:16,1021,0000,0000,No,No
CRC result: E9 64 (Integer = 59748)
With the result I could re-calculate the checksum of your example telegrams correctly
19.09.2016 12:18:12.764 [TX] - 10 13 08 20 03 01 11 25 00 00 00 00 E9 64
19.09.2016 12:18:14.606 [TX] - 10 13 08 20 03 01 11 25 01 00 00 00 9F D0
19.09.2016 12:18:16.030 [TX] - 10 13 08 20 03 01 11 25 02 00 00 00 04 0C
I uploaded the documented CRC finder example script which works with the free Docklight Scripting V2.2 evaluation. I assume this can be very useful for other CRC re-engineering puzzles, too.
The example also helped to solve Stackoverflow question 22219796

NTFS DataRun probably error

I am writing a code to parse MFT of NTFS. I`m trying analyse Data Run of non residental $INDEX_ALLOCATION attrib:
11 01 2C 11 02 FE 11 00
9F 0B 21 01 DB 00 21 01
D9 00 21 01 E0 00 21 01
F6 00 21 01 10 01 00 F1
After regroup I see problem in Data Run No 3:
DataRun 1: 11 01 2C
DataRun 2: 11 02 FE
DataRun 3: 11 00 9F <- what does mean "00" ?
I tried analyse it using Active Disk Editor 3 and this software decompose it to:
DataRun 3: 11 00 9F 0B
In my opinion header of DataRun 3 ("11") mean 1 length and 1 offset so there should be 2 bytes after header, but there are 3 bytes.
Any idea?

Sending wap push

I try send wap push message. I set datacoding 0xf5 and send submit sm with following message:
GSM Short Message Service User Data
udh length: 6
16-bit address 05 04
Destination port 08b4
Source port 23f0
Wireless Session Protocol, Method: Push (0x06), Content-Type: application/vnd.wap.sic
Transaction Id: 0x25
PDU Type: Push (0x06)
Header length: 1
Content type: application/vnd.wap.sic
WAP Binary XML, Version: 1.2, Public ID: "-//WAPFORUM//DTD SI 1.0//EN (Service Indication 1 .0)"
Version: 1.2 (0x02)
Public Identifier: (known): -//WAPFORUM//DTD SI 1.0//EN (Service Indication 1 .0)
Character Set: utf-8 (0x000006a)
String table: 0 bytes
Data representation:
45 <si>
c6 <indication
0c href='http://'
03 69 2e 69 6d 67 75 72 2e 63 6f 6d 2f 66 6a 49 44 4e 2e 6a 70 67 00 i.imgur.com/fjIDN.jpg'
07 action='signal-medium'
01 >
03 69 6d 67 75 72 00 'imgur'
01 </indication>
01 </si>
Message bytes:
06 05 04 0b 84 23 f0 25 06 01 ae 02 05 6a 00 45
c6 0c 03 69 2e 69 6d 67 75 72 2e 63 6f 6d 2f 66
6a 49 44 4e 2e 6a 70 67 00 07 01 03 69 6d 67 75
72 00 01 01
SMSC return: Submit_sm - resp: "OK".
But the phone did not show nothing. Any ideas?
Sorry for my ugly english :)
I can't fully justify why these changes work, but from extensive testing a few years back the following seemed to work reliably across the UK networks and a range of different handsets. It's in use in production code and we haven't seen any problems since.
06 - UDHL
05 - EI (Send to Ports 16bit addr)
04 - EIDL
0B - src port
84 - src port
23 - dest port
F0 - dest port (End of UDH)
01 - trans id
06 - Push
04 - Header Length
03 - Length
AE - Content type (application/vnd.wap.sic)
81 - Character Set (01 once removed high bit)
EA - UTF 8 (6A once removed high bit)
02 - Binary XML Version 1.2
05 - SI Identifier
6A - UTF-8
00 - End Data
45 - SI Binary XML Tag
C6 - Indication Tag
0B - href
03 - Open Text
(URL bytes go here...)
00 - End Data
0A - Created (date)
C3 - Data Follows
07 - Data Length
20 - date yy (century)
08 - date yy (year)
03 - date mm
26 - date dd
16 - date HH
09 - date MM
12 - date ss
01 - Close Attribute
03 - Open Text (Text Goes in here...)
00 - End Data
01 - Close Indication Tag
01 - Close SI Tag
There are a few differences to your send:
UTF-8 Character set specified on the Wireless Session Protocol header
http:// is written in full as bytes, I don't think this made a difference but it was required by the rest of our application.
No action on the indication tag
Added a created date attribute - This seemed to make a big difference to handset support (I'm not quite sure why).

Resources