Much searching and reading has not told me whether the capicom.encrypteddata class module (it's VB6, but that shouldn't matter in answering this question) is using 2-key 3DES or 3-key 3DES. (.Algorithm.Name = CAPICOM_ENCRYPTION_ALGORITHM_3DES) Anyone know which one it is using? A source of this information would also be helpful. I suspect, since I don't think high enough key lengths are supported, that it is 2DES. But I haven't found acceptable confirmation.
CAPICOM is a thin wrapper on top of CryptoAPI. If you decode the output from EncryptedData.Encrypt() you will see something like this (it is ASN.1 encoded in a proprietary format):
SEQUENCE {
OBJECT IDENTIFIER '1 3 6 1 4 1 311 88 3'
[0] {
SEQUENCE {
OBJECT IDENTIFIER '1 3 6 1 4 1 311 88 3 1'
[0] {
SEQUENCE {
INTEGER 131073
INTEGER 26115
INTEGER 192
OCTET STRING
AA A6 05 4E FA AF 4C 0B
OCTET STRING
3A 22 58 C3 51 D8 91 C8 7B 3C C9 51 9B E7 BA B7
OCTET STRING
84 FA 56 AF 01 FE C9 74
}
}
}
}
}
Note the 26115. That is the value for CALG_3DES, which is the CryptoAPI identifier for 3DES with three keys (3DES with two keys is called CALG_3DES_112). The 192 is the key-length, also match three-key 3DES:
Related
I have a kafka topic that has protobuf message of format:
message CreditTransaction {
string date = 1;
float amount = 2;
}
message DebitTransaction {
string date = 1;
float amount = 2;
}
...
.. # other message definitions
message TransactionEvent {
oneof event {
CreditTransaction credit = 1;
DebitTransaction debit = 2;
Trade trade = 3;
....
..# other fields
}
};
Using pyspark-streaming, when I am trying to use ParseFromString method to parse it, its giving me error:
File "./google.zip/google/protobuf/message.py", line 202, in ParseFromString
return self.MergeFromString(serialized)
File "./google.zip/google/protobuf/internal/python_message.py", line 1128, in MergeFromString
if self._InternalParse(serialized, 0, length) != length:
File "./google.zip/google/protobuf/internal/python_message.py", line 1178, in InternalParse
raise message_mod.DecodeError('Field number 0 is illegal.')
google.protobuf.message.DecodeError: Field number 0 is illegal.
Is it because the message TransactionEvent has only a single field and that too oneof type ?
I tried to add a dummy int64 id field also
message TransactionEvent {
int64 id = 1;
oneof event {
CreditTransaction credit = 2;
DebitTransaction debit = 3;
Trade trade = 4;
....
..# other fields
}
};
but still the same error.
Code I am using:
def parse_protobuf_from_bytes(msg_bytes):
msg = schema_pb2.MarketDataEvent()
msg.ParseFromString(msg_bytes)
eventStr = msg.WhichOneof("event")
if eventStr=="credit":
# some code
elif eventStr=="debit":
# some code
return str(concatenatedFieldsValue)
parse_protobuf = udf(lambda x: parse_protobuf_from_bytes(x), StringType())
kafka_conf = {
"kafka.bootstrap.servers": "kafka.broker.com:9092",
"checkpointLocation": "/user/aiman/checkpoint/kafka_local/transactions",
"subscribe": "TRANSACTIONS",
"startingOffsets": "earliest",
"enable.auto.commit": False,
"value.deserializer": "ByteArrayDeserializer",
"group.id": "my-group"
}
df = spark.readStream \
.format("kafka") \
.options(**kafka_conf) \
.load()
data = df.selectExpr("offset","CAST(key AS STRING)", "value") \
.withColumn("event", parse_protobuf(col("value")))
df2 = data.select(col("offset"),col("event"))
If I am just printing the bytes without parsing, I am getting this:
-------------------------------------------
Batch: 0
-------------------------------------------
+-------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|offset |event |
+-------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|7777777|bytearray(b'\x00\x00\x00\x00\xc2\x02\x0e\x1a]\n\x11RIOT_230120P35.00\x10\x80\xa6\xae\x82\xd9\xed\xf1\xfe\x16\x18\xcd\xd9\xd9\x82\xd9\xed\xf1\xfe\x16 \xe2\xf7\xd9\x82\xd9\xed\xf1\xfe\x16(\x95\xa2\xed\xff\xd9\xed\xf1\xfe\x160\x8c\xaa\xed\xff\xd9\xed\xf1\xfe\x168\x80\xd1\xb6\xc1\x0b#\xc0\x8d\xa3\xba\x0bH\x19P\x04Z\x02Q_b\x02A_') |
+-------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
The raw data you are trying to decode is:
b'\x00\x00\x00\x00\xc2\x02\x0e\x1a]\n\x11RIOT_230120P35.00\x10\x80\xa6\xae\x82\xd9\xed\xf1\xfe\x16\x18\xcd\xd9\xd9\x82\xd9\xed\xf1\xfe\x16 \xe2\xf7\xd9\x82\xd9\xed\xf1\xfe\x16(\x95\xa2\xed\xff\xd9\xed\xf1\xfe\x160\x8c\xaa\xed\xff\xd9\xed\xf1\xfe\x168\x80\xd1\xb6\xc1\x0b#\xc0\x8d\xa3\xba\x0bH\x19P\x04Z\x02Q_b\x02A_'
A valid protobuf message never starts with 0x00, because the field number 0 is reserved. The message appears to have some extra data in the beginning.
Comparing with the protobuf encoding specification, we can try to make sense of this.
Starting from the string RIOT_230120P35.00, it is correctly prefixed by length 0x11 (17 characters). The previous byte is 0x0A, which is tag for field 1 with type string, such as in CreditTransaction message. Reading the message backwards from there, everything looks reasonable up to 0x1A byte.
After stripping the first 7 bytes and converting to hex (1a 5d 0a 11 52 49 4f 54 5f 32 33 30 31 32 30 50 33 35 2e 30 30 10 80 a6 ae 82 d9 ed f1 fe 16 18 cd d9 d9 82 d9 ed f1 fe 16 20 e2 f7 d9 82 d9 ed f1 fe 16 28 95 a2 ed ff d9 ed f1 fe 16 30 8c aa ed ff d9 ed f1 fe 16 38 80 d1 b6 c1 0b 40 c0 8d a3 ba 0b 48 19 50 04 5a 02 51 5f 62 02 41 5f), the message is accepted by online protobuf decoder.
It seems the message has 7 extra bytes in the beginning for some reason. These bytes do not conform to the protobuf format and their meaning cannot be determined without some information from the developer of the other endpoint of the communication.
I'm trying to understand protocol buffer encoding method, when translating message to binary(or hexadecimal) format, I can't understand how the embedded message is encoded.
I guess maybe it's related to memory address, but I can't find the accurate relationship.
Here is what i've done.
Step 1: I defined two messages in test.proto file,
syntax = "proto3";
package proto_test;
message Education {
string college = 1;
}
message Person {
int32 age = 1;
string name = 2;
Education edu = 3;
}
Step 2: And then I generated some go code,
protoc --go_out=. test.proto
Step 3: Then I check the encoded format of the message,
p := proto_test.Person{
Age: 666,
Name: "Tom",
Edu: &proto_test.Education{
College: "SOMEWHERE",
},
}
var b []byte
out, err := p.XXX_Marshal(b, true)
if err != nil {
log.Fatalln("fail to marshal with error: ", err)
}
fmt.Printf("hexadecimal format:% x \n", out)
fmt.Printf("binary format:% b \n", out)
which outputs,
hexadecimal format:08 9a 05 12 03 54 6f 6d 1a fd 96 d1 08 0a 09 53 4f 4d 45 57 48 45 52 45
binary format:[ 1000 10011010 101 10010 11 1010100 1101111 1101101 11010 11111101 10010110 11010001 1000 1010 1001 1010011 1001111 1001101 1000101 1010111 1001000 1000101 1010010 1000101]
what I understand is ,
08 - int32 wire type with tag number 1
9a 05 - Varints for 666
12 - string wire type with tag number 2
03 - length delimited which is 3 byte
54 6f 6d - ascii for "TOM"
1a - embedded message wire type with tag number 3
fd 96 d1 08 - ? (here is what I don't understand)
0a - string wire type with tag number 1
09 - length delimited which is 9 byte
53 4f 4d 45 57 48 45 52 45 - ascii for "SOMEWHERE"
What does fd 96 d1 08 stands for?
It seems like that d1 08 always be there, but fd 96 sometimes change, don't know why. Thanks for answering :)
Add
I debugged the marshal process and reported a bug here.
At that location I/you would expect the number of bytes in the embedded message.
I have repeated your experiment in Python.
msg = Person()
msg.age = 666
msg.name = "Tom"
msg.edu.college = "SOMEWHERE"
I got a different result, the one I would expect. A varint stating the size of the embedded message.
0x08
0x9A, 0x05
0x12
0x03
0x54 0x6F 0x6D
0x1A
0x0B <- Equals to 11 decimal.
0x0A
0x09
0x53 0x4F 0x4D 0x45 0x57 0x48 0x45 0x52 0x45
Next I deserialized your bytes:
msg2 = Person()
str = bytearray(b'\x08\x9a\x05\x12\x03\x54\x6f\x6d\x1a\xfd\x96\xd1\x08\x0a\x09\x53\x4f\x4d\x45\x57\x48\x45\x52\x45')
msg2.ParseFromString(str)
print(msg2)
The result of this is perfect:
age: 666
name: "Tom"
edu {
college: "SOMEWHERE"
}
The conclusion I come to is that there are some different ways of encoding in Protobuf. I do not know what is done in this case but I know the example of a negative 32 bit varint. A positive varint is encoded in five bytes, a negative value is cast as a 64 bit value and encoded to ten bytes.
I am trying to parse this ibeacon packet received by scanning through a hci socket
b'\x01\x03\x00\x18\xbe\x99m\xf3\x14\x1e\x02\x01\x1a\x1a\xffL\x00\x02\x15e\xec\xe2\x90\xc7\xdbM\xd0\xb8\x1aV\xa6-b 2\x00\x00\x00\x02\xc5\xcc'
hex format 01 03 00 18 be 99 6d f3 14 1e 02 01 1a 1a ff 4c 00 02 15 65 ec e2 90 c7 db 4d d0 b8 1a 56 a6 2d 62 20 32 00 00 00 02 c5 cc
the parameters after applying the parser are
'UUID': '65ece290c7db4dd0b81a56a62d622032', 'MAJOR': '0000', 'MINOR': '0002', 'TX': -59, 'RSSI': -60
I am not sure if the RSSI portion of this parsing is right.
Referring to this https://stackoverflow.com/a/19040616/10355673
the last bit of the beacon advertising packet is the TX power value.
so how do we get the rssi value? here, I have taken rssi to be cc and tx to be c5. Is this correct?
There are flags headers before the manufacturer advertisement sequence shown below, but you really don't care about the flags. Here are the bytes you care about:
ff # manufacturee adv type
4c 00 # apple Bluetooth company code
02 15 # iBeacon type code
65 ec e2 90 c7 db 4d d0 b8 1a 56 a6 2d 62 20 32 # proximity uuid
00 00 # major
00 02 # minor
c5 # measured power (tx power)
cc # crc
Proximity UUUD: 65ece290-c7db-4dd0-b81a-56a62d622032,
Major: 0,
Minor: 2,
Measured Power: -59 dBm
The RSSI is not part of the transmitted packet, but a measurement taken by the receiver based on the strength of the signal. It will typically be a slightly different value for each packet that is received. You get this value from an API on a mobile device or embedded system that fetches it from the bluetooth chip.
I have a transportation card for urban transportation. I need to know what aid(application identifier) number of the card is. According to EMV Book 1, i have to use the List of AIDs method (page 141). But how?
I also have a ACR122U card reader. I can send an APDU command to the card. All i need is the AID of the card. In addition, i always get SW=6A82 error. It means "File Not Found". I suppose, i need to know true AID number to solve this problem. I want to see SW=9000 (successful) response...
Edit: Code for creating select apdu command
private static final byte[] CLA_INS_P1_P2 = { 0x00, (byte)0xA4, 0x04, 0x00 };
private static final byte[] AID_ANDROID = { (byte)0xF0, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06 };
private byte[] createSelectAidApdu(byte[] aid) {
byte[] result = new byte[6 + aid.length];
System.arraycopy(CLA_INS_P1_P2, 0, result, 0, CLA_INS_P1_P2.length);
result[4] = (byte)aid.length;
System.arraycopy(aid, 0, result, 5, aid.length);
result[result.length - 1] = 0;
return result;
}
Thanks..
Typically, you should lookup the card documentation, which should describe how files organized.
However, since you're reading an ISO-DEP card, you may refer to ISO/IEC CD 7816-4. The card should implement part of instructions in this standard. According to Section 5.2, the file can be chosen using its identifier, which means you are able to enumerate all files located in MF.
So a possible solution is:
Send select file by identifier instruction as
00 A4 00 00 02 id 00
Where id ranges from 0000 to FFFF.
Once you receive SW=9000, the response should contain file control information (FCI, see Section 5.6). You can then find the DF name after byte 84. For example, a card responds
6F 15 84 0D 4E 43 2E 65 43 61 72 64 2E 44 46 30 31 A5 04 9F 08 01 02 90 00,
the DF name is 4E 43 2E 65 43 61 72 64 2E 44 46 30 31. The byte 0D after 84 indicates the length of DF name is 0x0D. You may use it as AID.
INTRO
I'm trying to reverse engineer a binary data file containing sms messagges.
The file is named ems.idx4 and was created with a software named LG PhoneManager around 5 years ago as a backup archive of sms messagges for a LG mobile.
I don't know which language was used to write LG PhoneManager, but in the binary file I read strings like "CObTree", "CFolder", "CMessage": maybe this clue means nothing, maybe it suggests that Cobol/.net/whatever language was used.
PROBLEM
I decoded the entire structure of the binary file, which is quite plain tho.
The only part I couldn't decode is date and time of single messagges.
I identified the binary part where date and time are encoded and I got a few decoded examples (thanks to the content of the message).
Binary data in hex:
[0x10] D0 74 C4 FE 3F 42 E3 40 F1 64 [0x7] 2 [0x13] 1 [0x6] 6C [0x2] is 2007/12/25 some time after 23:58 GMT+1
[0x10] 2B 25 CA 19 2F 43 E3 40 F1 64 [0x7] 2 [0x13] 1 [0x6] 6C [0x2] is 2008/01/02 some time after 10:48 GMT+1
[0x10] AA C0 2C 6E 35 43 E3 40 F1 64 [0x7] 2 [0x13] 1 [0x6] 6C [0x2] is 2008/01/02 some time after 16:03 GMT+1
[0x10] EE 04 71 F2 B6 43 E3 40 F1 64 [0x7] 2 [0x13] 1 [0x6] 6C [0x2] is 2008/01/06 some time after 14:31 GMT+1
[0x10] 60 2C F9 45 4E 4F E3 40 F1 64 [0x7] 2 [0x13] 1 [0x6] 6C [0x2] is 2008/04/08 some time after 10:32 GMT+1
[0x10] 5D 84 01 14 74 64 E3 40 F1 64 [0x7] 2 [0x13] 1 [0x6] 6C [0x2] is 2008/11/11 some time after 14:53 GMT+1
where [0xN] means a sequence of N zeros.
Any idea?
UPDATE
Using this tool: http://www.digital-detective.co.uk/freetools/decode.asp
I realized it is Windows 64 bit OLE date / time format.
According to that tool:
D0 74 C4 FE 3F 42 E3 40 means exactly 26/12/2007 00:59
Any idea what the math behind this Windows 64 bit OLE date / time format?
It's a double precision floating point number representing the number of days (and fractional days) since the epoch December 30, 1899.
If you're on Windows you can use the VariantTimeToSystemTime function to get it into a more usable format:
unsigned char timeBytes[] = {0xD0,0x74,0xC4,0xFE,0x3F,0x42,0xE3,0x40};
double timeDouble = *(double*)&timeBytes;
SYSTEMTIME systemTime;
VariantTimeToSystemTime(timeDouble, &systemTime);
If you're not using Windows, I suspect you'd need to do the conversion manually. Let me know in the comments if you need any help with that.
Ok, I found my way!
The first 8 bytes after [0x10] are a OLE date in little endian hex.
I converted them to a regular datetime in python with:
import datetime
import math
from struct import unpack
def ole_date_bin_to_datetime(ole_date_bin):
"""
Converts a OLE date from a binary 8 bytes little endian hex form to a datetime
"""
#Conversion to OLE date float, where:
# - integer part: days from epoch (1899/12/30 00:00)
# - decimal part: percentage of the day, where 0,5 is midday
date_float = unpack('<d', ole_date_bin)[0]
date_decimal, date_integer = math.modf(date_float)
date_decimal = abs(date_decimal)
date_integer = int(date_integer)
#Calculate the result
res = datetime.datetime(1899, 12, 30) + datetime.timedelta(days=date_integer) #adding days to epoch
res = res + datetime.timedelta(seconds = 86400*date_decimal) #adding percentage of the day
return res
if __name__ == "__main__":
print ole_date_bin_to_datetime('\xd0\x74\xc4\xfe\x3f\x42\xe3\x40')