fail to correctly encode and decode using java.util.Base64 - java-8

Let "awids" be 12 characters length ids in base 64 (A-Z a-z 0-9 "-" "#"). This is the input.
My final goal is to create a bijective mapping between these awids and UUIDs, using some paddings, having as initial input the awids.
While trying to use java.util.Base64 I do not get the initial value after a decoding and an encoding again. What is the stupid error I do? :)
With the reproducible example I present below the output is wrong because the input string is not gotten back after a decode()-encode() and the bijection is not preserved (Q39s/L and Q39s/A map both to the same value).
------------------------------------------> Q39s/L (6 [51 33 39 73 2f 4c])
4 [43 7f 6c fc] -> 6 [51 33 39 73 2f 41] -> Q39s/A (6 [51 33 39 73 2f 41])
4 [43 7f 6c fc] -> 6 [51 33 39 73 2f 41] -> Q39s/A (6 [51 33 39 73 2f 41])
Here a is a reproducible example:
import java.nio.charset.StandardCharsets;
import java.util.Base64;
import java.util.StringJoiner;
public class StackOverflowQuestion {
public static void main(String[] args) {
String halfAwid = "Q39s/L";
byte[] sigBits = Base64.getDecoder().decode(halfAwid.getBytes(StandardCharsets.UTF_8));
byte[] actualSigBits = Base64.getEncoder().withoutPadding().encode(sigBits);
String actualHalfAwid = new String(actualSigBits, StandardCharsets.UTF_8);
byte[] sigBits2 = Base64.getDecoder().decode(halfAwid.getBytes(StandardCharsets.UTF_8));
byte[] actualSigBits2 = Base64.getEncoder().withoutPadding().encode(sigBits2);
String actualHalfAwid2 = new String(actualSigBits2, StandardCharsets.UTF_8);
System.out.println("----------------------------------------------> "
+ halfAwid + " (" + toHexString(halfAwid) + ") "
+ "\n"
+ " "
+ toHexString(sigBits) + " -> "
+ toHexString(actualSigBits) + " -> "
+ actualHalfAwid + " (" + toHexString(actualHalfAwid) + ") "
+ "\n"
+ " "
+ toHexString(sigBits2) + " -> "
+ toHexString(actualSigBits2) + " -> "
+ actualHalfAwid2 + " (" + toHexString(actualHalfAwid2) + ")"
+ "");
}
private static String toHexString(byte[] bytes) {
StringJoiner joiner = new StringJoiner(" ", "" + bytes.length + " [", "]");
for (byte b : bytes) {
joiner.add(String.format("%02x", b));
}
return joiner.toString();
}
private static String toHexString(String text) {
return toHexString(text.getBytes());
}
}
Do not hesitate to point any other errors I do in the code, even if they are not related directly to the question. Thank you.

The Base64 encoding is not a bijective mapping for all input sizes, if you treat the encoded data as a sequence of whole bytes (or ASCII characters). Base64 is encoding units of eight bits to units of six bits (yielding 64 possible combinations for each unit), so when you encode four bytes, in other words 4×8=32 bits, you will get 32/6=5⅓ units output, which implies that the sixth unit of the output will not use all bits.
In other words, when you treat an arbitrary string consisting of six of the 64 defined characters as being Base64 encoded, you will project a string of 64⁶ combinations to a “source” sequence of six bytes having 256⁴ combinations, which implies a data loss.
You can use Base64 encoding as a bijective mapping if you choose input sizes which can be projected to a whole number of units, e.g. obviously six source bytes can be encoded as eight Base64 encoded bytes. But it doesn’t work for six encoded bytes. Interestingly, it will work for your actually desired size as nine source bytes will get encoded to exactly twelve encoded bytes: 9×8=72, 72/6=12.

Related

Trouble writing NDEF record to NTAG213 using external NFC reader (but writing to memory works)

I am using the sample provided by Michael Roland in this answer and modified the bytes command structure to match this answer.
After I scan the tag, I receive 90 00 responses from the reader. When I then scan the tag using NFC Tools though, I don't see that it has an NDEF record (photo). If I examine the memory I can see my data written starting at block 4 as follows here.
Meanwhile, if I use the Write Tag feature of NFC Tools to write an NDEF message and then scan the tag again, it does work. The memory in the blocks other than those starting at block 4 appear to be identical (photo).
I don't believe it's a capability container issue as the memory is identical in block 3 after writing to the tag from my reader vs. NFC Tools.
Do I need to do any other kind of NDEF read / check command prior to writing to block 4?
My code below:
byte[] ndefMessage = new byte[] {
(byte)0xD1, (byte)0x01, (byte)0x0C, (byte)0x55, (byte)0x01, (byte)0x65, (byte)0x78, (byte)0x61, (byte)0x6D, (byte)0x70, (byte)0x6C, (byte)0x65, (byte)0x2E, (byte)0x63, (byte)0x6F, (byte)0x6D, (byte)0x2F
};
// wrap into TLV structure
byte[] tlvEncodedData = null;
Log.e("length",String.valueOf(ndefMessage.length));
if (ndefMessage.length < 255) {
tlvEncodedData = new byte[ndefMessage.length + 3];
tlvEncodedData[0] = (byte)0x03; // NDEF TLV tag
tlvEncodedData[1] = (byte)(ndefMessage.length & 0x0FF); // NDEF TLV length (1 byte)
System.arraycopy(ndefMessage, 0, tlvEncodedData, 2, ndefMessage.length);
tlvEncodedData[2 + ndefMessage.length] = (byte)0xFE; // Terminator TLV tag
} else {
tlvEncodedData = new byte[ndefMessage.length + 5];
tlvEncodedData[0] = (byte)0x03; // NDEF TLV tag
tlvEncodedData[1] = (byte)0xFF; // NDEF TLV length (3 byte, marker)
tlvEncodedData[2] = (byte)((ndefMessage.length >>> 8) & 0x0FF); // NDEF TLV length (3 byte, hi)
tlvEncodedData[3] = (byte)(ndefMessage.length & 0x0FF); // NDEF TLV length (3 byte, lo)
System.arraycopy(ndefMessage, 0, tlvEncodedData, 4, ndefMessage.length);
tlvEncodedData[4 + ndefMessage.length] = (byte)0xFE; // Terminator TLV tag
}
// fill up with zeros to block boundary:
tlvEncodedData = Arrays.copyOf(tlvEncodedData, (tlvEncodedData.length / 4 + 1) * 4);
for (int i = 0; i < tlvEncodedData.length; i += 4) {
byte[] command = new byte[] {
(byte)0xFF, // WRITE
(byte)0xD6,
(byte)0x00,
(byte)((4 + i / 4) & 0x0FF), // block address
(byte)0x04,
0, 0, 0, 0
};
System.arraycopy(tlvEncodedData, i, command, 5, 4);
ResponseAPDU answer = cardChannel.transmit(new CommandAPDU(command));
byte[] response = answer.getBytes();
writeLogWindow("response: "+ byteArrayToHexString(response));
}
I believe that the problem is that Michael Roland's answer has a bug in it.
D1 01 0C 55 01 65 78 61 6D 70 6C 65 2E 63 6F 6D 2F is not a valid Ndef message.
If you look at the various specs for Ndef at https://github.com/haldean/ndef/tree/master/docs (specifically the NFCForum-TS-RTD_URI_1.0.pdf and NFCForum-TS-NDEF_1.0.pdf) his example of "http://www.example.com/" is actually made up of "http://www." which has a type code of 01 and 12 characters or the rest of the URL.
Thus the payload length is 13 (1 + 12) bytes so OD where as his message:-
D1 01 0C 55 01 65 78 61 6D 70 6C 65 2E 63 6F 6D 2F
only specifies the length of the second part of the URL and not the prefix, so is one byte too short.
This is confirmed if you try and write a record for that URL using the NFC Tools App or NXP's TagWriter App both generate a message of
D1 01 0D 55 01 65 78 61 6D 70 6C 65 2E 63 6F 6D 2F
So try using in your code
byte[] ndefMessage = new byte[] {
(byte)0xD1, (byte)0x01, (byte)0x0D, (byte)0x55, (byte)0x01, (byte)0x65, (byte)0x78,
(byte)0x61, (byte)0x6D, (byte)0x70, (byte)0x6C, (byte)0x65, (byte)0x2E, (byte)0x63,
(byte)0x6F, (byte)0x6D, (byte)0x2F
};

Finding the formula for an alphanumeric code

A script I am making scans a 5-character code and assigns it a number based on the contents of characters within the code. The code is a randomly-generated number/letter combination. For example 7D3B5 or HH42B where any position can be any one of (26 + 10) characters.
Now, the issue I am having is I would like to figure out the number from 1-(36^5) based on the code. For example:
00000 = 0
00001 = 1
00002 = 2
0000A = 10
0000B = 11
0000Z = 36
00010 = 37
00011 = 38
So on and so forth until the final possible code which is:
ZZZZZ = 60466176 (36^5)
What I need to work out is a formula to figure out, let's say G47DU in its number form, using the examples below.
Something like this?
function getCount(s){
if (!isNaN(s))
return Number(s);
return s.charCodeAt(0) - 55;
}
function f(str){
let result = 0;
for (let i=0; i<str.length; i++)
result += Math.pow(36, str.length - i - 1) * getCount(str[i]);
return result;
}
var strs = [
'00000',
'00001',
'00002',
'0000A',
'0000B',
'0000Z',
'00010',
'00011',
'ZZZZZ'
];
for (str of strs)
console.log(str, f(str));
You are trying to create a base 36 numeric system. Since there are 5 'digits' each digit being 0 to Z, the value can go from 0 to 36^5. (If we are comparing this with hexadecimal system, in hexadecimal each 'digit' goes from 0 to F). Now to convert this to decimal, you could try use the same method used to convert from hex or binary etc... system to the decimal system.
It will be something like d4 * (36 ^ 4) + d3 * (36 ^ 3) + d2 * (36 ^ 2) + d1 * (36 ^ 1) + d0 * (36 ^ 0)
Note: Here 36 is the total number of symbols.
d0, d1, d2, d3, d4 can range from 0 to 35 in decimal (Important: Not 0 to 36).
Also, you can extend this for any number of digits or symbols and you can implement operations like addition, subtraction etc in this system itself as well. (It will be fun to implement that. :) ) But it will be easier to convert it to decimal do the operations and convert it back though.

how to find xor key/algorithm, for a given hex?

So i have this hex: B0 32 B6 B4 37
I know this hex is obfuscated with some key/algorithm.
I also know this hex is equal to: 61 64 6d 69 6e (admin)
How can i calculate the XOR key for this?
If you write out the binary representation, you can see the pattern:
encoded decoded
10110000 -> 01100001
00110010 -> 01100100
Notice that the bit patterns have the same number of bits before and after. To decode, you just bitwise rotate one bit left. So the value shifts left one place and the most significant bit wraps around to the least significant place. To encode, just do the opposite.
int value, encoded_value;
encoded_value = 0xB0;
value = ((encoded_value << 1) | (encoded_value >> 7)) & 255;
// value will be 0x61;
encoded_value = ((value >> 1) | (value << 7)) & 255;

Lotusscript generate random 8 characters string?

How do I generate a string of 8 characters using lotusscript in this pattern? Eg: 0E 1F A3 ZK (there are spaces after every 2 characters including the last). Each character can only be either 0-9 or A-Z (uppercased only). I've used the Randomize and Rnd method before and thinking of applying it here but I'm not sure if that's the correct way and how to achieve that. Another thing is this string is going to be saved into my document and I have a view that list each of the generated string. Which means each time a string is generated, it has to be unique. If the string generated is already used in another document, then continue generating until one that hasn't been used is generated.
Yes, you can use Rnd in this case too.
Define a function getRandom() which gives you a random string in format "XX XX XX XX" with every call.
Function getRandom() As String
Const charList = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"
getRandom = _
Mid(charList, 35*Rnd+1, 1) + Mid(charList, 35*Rnd+1, 1) + " " + _
Mid(charList, 35*Rnd+1, 1) + Mid(charList, 35*Rnd+1, 1) + " " + _
Mid(charList, 35*Rnd+1, 1) + Mid(charList, 35*Rnd+1, 1) + " " + _
Mid(charList, 35*Rnd+1, 1) + Mid(charList, 35*Rnd+1, 1)
End Function
charList is a string with all allowed characters 0-9 and A-Z.
you get an random number between 1 and 36 with 35*Rnd+1. This is your index in charList to get randomly one of the characters.
Call the function getRandom() in a cycle as long as you get a string which is not yet in your view.
sID = Join(Evaluate(|#Password(#Unique)|))
Mid$(sID, 2, 2) & " " & Mid$(sID, 4, 2) & " " & Mid$(sID, 6, 2) & " " & Mid$(sID, 8, 2)
How it works:
#Unique is a sequential string token
#Password(anyText) will return a unique 32 digit HEX string enclosed in parenthesis
Evaluate will run the #Function formula and return an array of 1 element. So the Join around the Evaluate will turn that to a string scalar value.
The Mid$ functions simply yield 2 character values at 2 character offsets.
The only issue regarding your original parameters of your question is that you will never see get values above F since we are using Hexadecimal characters (0-9, A-F)

SIMPLE-TLV vs BER-TLV

I have found in docs they are referring to SIMPLE-TLV and BER-TLV . I was look into most of the EMV and GP docs but they have not mentioned the different.
Could anyone help me to understand the difference of two ?
Data fields in ISO/IEC 7816-4 for smart cards
BER encoding
This is the specification of the more common BER encoding used by ISO/IEC 7816-4:
Each BER-TLV data object shall consists of 2 or 3 consecutive fields
(see ISO/IEC 8825 and annex D).
The tag field T consists of one or more consecutive bytes. It encodes
a class, a type and a number. The length field consists of one or more
consecutive bytes. It encodes an integer L. If L is not null, then the
value field V consists of L consecutive bytes. If L is null, then the
data object is empty: there is no value field.
Note that ISO/IEC 7816 only allows the use of up to 5 length bytes (specifying a size up to 2^32 - 1 bytes) in the current standard. Indefinite length encoding is not supported either. These limitations are specific to smart cards. Note that 4 and 5 byte length encodings were introduced in a later version of ISO/IEC 7816-4; earlier cards / card reading applications may only support 3 length bytes (i.e. a value size up to 64KiB bytes, instead of 4GiB).
The BER TLV specification is much more expansive (which is why SIMPLE-TLV is called "simple"). I won't go into the details too much as there is plenty of information available on the internet. To name just a few differences, the tags have syntactical meaning and may consist of multiple bytes and the length encoding is rather complex.
Normally BER should only be used as an encoding of ASN.1 structures, with the ASN.1 syntax defining the structure. ISO 7816-4 however messes this up and only specifies the BER tag bytes directly.
Note that sometimes DER is specified instead of BER. In that case you should only use the minimum number of bytes for the size of the length field - e.g. a single length byte with value 05 in the samples below. The ISO/IEC specification of BER encoding is basically a copy of the US specific X.690 standard, also reflected in the international standard ISO/IEC 8825-1 (both payware).
SIMPLE-TLV encoding
The BER specification in ISO/IEC 7816-4 is followed by the SIMPLE-TLV specification. SIMPLE-TLV is specific to ISO 7816-4.
Each SIMPLE-TLV data object shall consist of 2 or 3 consecutive
fields.
The tag field T consists of a single byte encoding only a number from
1 to 254 (e.g. a record identifier). It codes no class and no
construction-type. The length field consists of 1 or 3 consecutive
bytes. If the leading byte of the length field is in the range from
'00' to 'FE', then the length field consists of a single byte encoding
an integer L valued from 0 to 254. If the leading byte is equal to
'FF', then the length field continues on the two subsequent bytes
which encode an integer L with a value from 0 to 65535. If L in not
null, then the value field V consists of consecutive bytes. If L is
null, then the data object is empty: there is no value field.
Note that the standard forgets to specify the endianness directly. You can however assume big endian encoding within ISO/IEC 7816-4.
Samples
The following samples are all used to convey the same tag number (which defines the field) and value, except one that defines tag number 31 for BER.
Sample SIMPLE-TLV
0F 05 48656C6C6F // tag number 15, length 5 then the value
0F FF0005 48656C6C6F // tag number 15, length 5 (two bytes), then the value
Sample BER-TLV:
4F 05 48656C6C6F // *application specific*, primitive encoding of tag number 15, length 5 then the value
4F 8105 48656C6C6F // the same, using two bytes to encode the length
4F 820005 48656C6C6F // the same, using three bytes to encode the length
4F 83000005 48656C6C6F // the same, using four bytes to encode the length
4F 8400000005 48656C6C6F // the same , using five bytes to encode the length
5F0F 05 48656C6C6F // **invalid** encoding of the same, with two bytes for the tag, specifiying a tag number 15 which is smaller than 31
5F1F 05 48656C6C6F // application specific, primitive encoding of **tag number 31**
In the last example with the two byte tag encoding, the first byte is 40 hex, where the first 3 leftmost bits 010 specify application specific encoding, adding the magic value 1F (31) to it to indicate that another byte will follow with the actual tag number, again 1F, so value 31.
Differences
The following differences should be noted:
SIMPLE-TLV is a different method of encoding for tag and length (although the encoding may look similar, e.g. when using a single byte to indicate the length part)
SIMPLE-TLV does not contain information about the class of the field, e.g. if it is defined for ASN.1 (because it is not linked to ASN.1)
SIMPLE-TLV does not contain information if it is primitive or constructed (primitive directly specifies a value, constructed means nested TLV structures)
SIMPLE-TLV has restrictions regarding the tag number (between 1 and 254, inclusive) and length (up to 65535)
Simple TLV simply consists of Tag (or Type), Length, and Value.
The BER-TLV is a special TLV which has one or more TLV inside its Value. So it has composite structure.
Tag1 Len1 Tag2-Len2-Value2 Tag3-Len3-Value3 ... TagN-LenN-ValueN
------------------------Value1------------------------
[Example C# code for Bert-Tlv Parser][1]:
[1]https://github.com/umitkoc/BertTlv
public class Tlv : ITlv, IFile
{
List<TlvModel> modelList=new();
string parser = "";
string length = "";
String empty = "";
string ascii = "";
int decValue = 0;
int step = 0;
public Tlv(String data)
{
TlvParser(data.Replace(" ",""));
}
public void readTag()
{
String line = "";
StreamReader sr = new StreamReader("taglist.txt");
while ((line = sr.ReadLine()) != null)
{
modelList.Add(new()
{
tag = line.Split(",")[0].Trim(),
description = line.Split(",")[1].Trim()
});
}
sr.Close();
}
public void insertTag()
{
try
{
StreamWriter sw = new StreamWriter("test.txt");
foreach (var item in modelList)
{
sw.WriteLine($"{item.tag},{item.description}");
}
sw.Close();
}
catch (Exception e)
{
Console.WriteLine("Exception: " + e.Message);
}
}
public int writeFile(String parser)
{
StreamWriter sw = new StreamWriter("output.txt");
sw.WriteLine(parser);
sw.Close();
return 0;
}
private int TlvParser(String data, int i = 0, string tag = "")
{
if (i == 0)
{
readTag();
}
if (i < data.Length)
{
tag += data[i];
TlvModel model = getTag(tag);
if (model != null)
{
decValue = int.Parse(data.Substring(i + 1, 2), System.Globalization.NumberStyles.HexNumber);
// lengthControl(data,i+3,decValue);
if (model.description.Contains("Template"))
{
parser += $"{empty}|------ tag: {model.tag}({model.description})\n";
step += 1;
empty = Empty();
return TlvParser(data, i + 3, "");
}
else
{
parser += $"{empty}|------ tag: {model.tag}({model.description}){empty}|------ value --> {ConvertHex(data.Substring(i + 3, decValue * 2))} \n";
}
i += 3 + decValue * 2;
return TlvParser(data, i, "");
}
else
{
return TlvParser(data, i + 1, tag);
}
}
return writeFile(parser);
}
public TlvModel getTag(string tag)
{
return modelList.Find(i => i.tag == tag);
}
public string ConvertHex(string hex)
{
ascii = "";
for (int i = 0; i < hex.Length; i += 2)
{
ascii += System.Convert.ToChar(System.Convert.ToUInt32(hex.Substring(i, 2), 16));
}
return ascii;
}
private string Empty()
{
for (int s = 0; s < step; s++)
{
empty += "\t";
}
return empty;
}
public void setTag(TlvModel model)
{
modelList.Add(model);
insertTag();
}
}

Resources