I'm trying to convert a base64 string to binary data using this code:
output = base64.b64encode(requests.get(image_url).content)
bin = "".join(format(ord(x), "b") for x in base64.decodestring(output))
while I'm trying to convert bin (the binary data) into base64 using this code:
codecs.encode(bin, 'base64')
I get a different string than the original one.
Any ideas how to fix it?
Related
How can I convert a Base64 encoded string to a hex encoded string with dashes(basically to uuid format)?
For example if I have
'FWLalpF2T5mmyxS03Q+hNQ0K'
then how can I convert it to:
1562da96-9176-4f99-a6cb-14b4dd0fa135
I was familiar with unpack but this prompted me to learn the directive as pointed out by cremno.
simplest form:
b64 = 'FWLalpF2T5mmyxS03Q+hNQ0K'
b64.unpack("m0").first.unpack("H8H4H4H4H12").join('-')
#=> "1562da96-9176-4f99-a6cb-14b4dd0fa135"
b64.unpack("m0")
give us:
#=> ["\x15b\xDA\x96\x91vO\x99\xA6\xCB\x14\xB4\xDD\x0F\xA15\r\n"]
which is an array so we use .first to grab the string and unpack again using the directive to format it in the 8-4-4-4-12 format:
b64.unpack("m0").first.unpack("H8H4H4H4H12")
gives us:
#=> ["1562da96", "9176", "4f99", "a6cb", "14b4dd0fa135"]
an array of strings, so now we just join it with the -:
b64.unpack("m0").first.unpack("H8H4H4H4H12").join('-')
#=> "1562da96-9176-4f99-a6cb-14b4dd0fa135"
OOPS
The accepted answer has a flaw:
b64 = 'FWLalpF2T5mmyxS03Q+hNQ0K'
b64.unpack("m0").first.unpack("H8H4H4H4H12").join('-')
# => "1562da96-9176-4f99-a6cb-14b4dd0fa135"
Changing the last char in the b64 string results in the same UUID:
b64 = 'FWLalpF2T5mmyxS03Q+hNQ0L'
b64.unpack("m0").first.unpack("H8H4H4H4H12").join('-')
# => "1562da96-9176-4f99-a6cb-14b4dd0fa135"
To prevent this, you might want to hash your input (base64 or anything else) to the correct length e.g. with MD5:
require "digest"
b64 = 'FWLalpF2T5mmyxS03Q+hNQ0K'
Digest::MD5.hexdigest(b64).unpack("a8a4a4a4a12").join('-')
# => "df71c785-6552-a977-e0ac-8edb8fd63f6f"
Now the full input is relevant, altering the last char results in a different UUID:
require "digest"
b64 = 'FWLalpF2T5mmyxS03Q+hNQ0L'
Digest::MD5.hexdigest(s).unpack("a8a4a4a4a12").join('-')
# => "2625f170-d05a-f65d-38ff-5d9a7a972382"
i know this was handled a lot here, but i couldnt solve my problem yet:
I read bytes from a Parceble Object and save them in a byte[], then I unmurshall
them back to an Object an it works all fine. But i have to send the bytes as a String, so i
have to convert the bytes to string and then return.
I thought it would work as follow:
byte[] bytes = p1.marshall(); //get my object as bytes
String str = bytes.toString();
byte[] someBytes = str.getBytes();
But it doesnt Work, when I "p2.unmarshall(someBytes, 0, someBytes.length);" with someBytes, but when I p2.unmarshall(bytes, 0, bytes.length); with bytes, it works fine. How can i convert bytes to String right?
You've got three problems here:
You're calling toString() on byte[], which is just going to give you something like "[B#15db9742"
You're assuming you can just convert a byte array into text with no specific conversion, and not lose data
You're calling getBytes() without specifying the character encoding, which is almost always a mistake.
In this case, you should just use base64 - that's almost always the right thing to do when converting arbitrary binary data to text. (If you were actually trying to decode encoded text, you should use new String(bytes, charset), but that's not the case here.)
So, using android.util.Base64:
String str = Base64.encodeToString(bytes, Base64.DEFAULT);
byte[] someBytes = Base64.decode(str, Base64.DEFAULT);
What would be the best way to convert a hexadecimal value (for instance a SHA-256 digest) into the base64urlsafe format?
Example:
f("e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855")
# => "47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU="
The following should do it:
Base64.urlsafe_encode64(Array(hex_string).pack('H*'))
The method urlsafe_encode64 expects binary input, so you need to convert the hex representation to binary first, as shown above.
Use urlsafe_encode64 method:
require 'base64'
# The method urlsafe_encode64 expects binary input,
# so you need to convert the hex representation to binary
bin_string = [hex_string].pack('H*')
Base64.urlsafe_encode64 bin_string
I am working in Matlab environment for a project, and I have to decode a RGB image received in xml from the database server, which is encoded in base64 format. I was successful in converting image to base64 and post it to the database by converting it into xml. I used the base64encode/decode to encode the image to base64 and I have attached the program below. The problem is when I use the base64decode function and try to reconvert the image from base64. It simply does not work.
This is my program for converting image to base64 and encode it in xml.
function image2xml(test_directory)
% Images in directory ---> byte array format in XML
% This function encodes all the images available in the test directory into
% byte array/base 64 format and saves them in xml with the following
% properties
% Packs the image(byte array) and its name as timestamp to xml
% Uses functions from the following source
% http://www.mathworks.de/matlabcentral/fileexchange/12907-xmliotools
% Following functions from the above source are to be added to path,while
% running this function
% xml_write.m
% xml_read.m
%% ========================================================================
files=dir(test_directory)
% delete('test_image_xml\*.xml');
% If not database_mat is included in the function arguments
for i = 1:size(files,1)
k=0;
if files(i).isdir()==0
%extracts name with which it savesa as xml
[~, name_to_save,~ ] = fileparts(files(i).name)
filename = fullfile([test_directory,'\',files(i).name])
fid = fopen(filename);
raw_data = uint8(fread(fid));% read image file as a raw binary
fclose(fid);
%Definition of xml tags
image_imagedetails = [];
% name of the file is assumed to be the timestamp
image_imagedetails.timestamp =name_to_save;
%imagescan.imagebyte64.ATTRIBUTE.EncodingMIMEType = 'base64';
image_imagedetails.imagebase64 = base64encode(raw_data);% perform base64 encoding of the binary data
%saves all the xml files into the predefined directory
mkdir('images_and_timestamp_xml');
filename = ['images_and_timestamp_xml\' name_to_save,'.xml' ];
post_data = xml_write(filename, image_imagedetails);
end
end
Finaly I use the following to reconvert the xml created with image in base64 format back to image, but unfortunately it does not work, and throws some strange characters, which I am not able to convert back into a image. I further have no clue as of how to convert the string back to image as well.
filename = '...\IMAG0386.xml';
tree = xml_read(filename);
image = tree.imagebase64;
K = base64decode(tree.imagebase64)) %test image retrieval --> only the string
And I tried out other option like using the Java code in matlab, but I do not know, how to use the code in matlab. There are many options in C#, Java, but I have no idea, as how to use them in matlab. Please help me in this regards.
I ran your code under Matlab R2012a and it seems to work as expected.
Maybe what is missing here is a few lines to get the image file back from the binary data encoded in base64. You just need to write the binary data to a file to get your image file back.
I am merely quoting the HTML help file from the Matlab FileExchange submission xmliotools that you are using in your code:
Read XML file with embedded binary data encoded as Base64 (using java
version)
tree = xml_read('test.xml', Pref); % read xml file
raw = base64decode(tree.MyImage.CONTENT, '', 'java'); % convert xml image to raw binary
fid = fopen('MyFootball.jpg', 'wb');
fwrite(fid, raw, 'uint8'); % dumb the raw binary to the hard disk
fclose(fid);
I = imread('MyFootball.jpg'); % read it as an image
imshow(I);
Simple Base64 Handling
Using the Apache library
base64 = org.apache.commons.codec.binary.Base64
Then you can call encode or decode.
base64.encode()
base64.decode()
It expects byte[], so you can get this in a couple of ways. Let's encode a string and then decode it.
hello = 'Hello, world!';
encoded = char(base64.encode(unicode2native(hello))).';
result = native2unicode(base64.decode(uint8(output)).');
I'm hashing a password using SHA512. I'm using Entity Framework Code-First for my ORM.
Hashing Algorithm
public static string CreateSHA512Hash(string pwd, string salt)
{
string saltAndPwd = String.Concat(pwd, salt);
var ae = new ASCIIEncoding();
byte[] hashValue, messageBytes = ae.GetBytes(saltAndPwd);
var sHhash = new SHA512Managed();
hashValue = sHhash.ComputeHash(messageBytes);
sHhash.Dispose();
return ae.GetString(hashValue);
}
Code for generating salt:
//Generate a cryptographic random number.
var rng = new RNGCryptoServiceProvider();
var buff = new byte[size];
rng.GetBytes(buff);
rng.Dispose();
// Return a Base64 string representation of the random number.
return Convert.ToBase64String(buff);
Problem:
For some reason, it seems the hash function would randomly generate some characters, which the ones after those are not saved to the database. In this case (I'm not sure if there are other characters that does this), but it is \0.
For eg. Password: testuser. Salt: uvq5i4CfMcOMjKPkwhhqxw==
Hash generated: ????j???7?o\0?dE??????:???s?x??u?',Vj?mNB??c???4H???vF\bd?T? (copied during dubug mode in visual studio).
But EF actually saves ????j???7?o to the database. If I try to use the text visualizer in debug mode, it cuts it off also. If you noticed, it gets cut off right at the \0. All I could find about it is that its a null character.
Question
How can I save this null character in the database using Entity Framework Code-First? If this can't be saved, how can I prevent the SHA512 from generating these characters for me?
I recommend encoding the hash with Base64 before saving. On the other hand, encoding the salt with Base64 before adding to the password sounds strange.
A SHA-256 hash does not generate characters, it generates bytes. If you want to have a character string, as opposed to a byte array, you need to convert the bytes into a character format. As #wRAR has suggested, Base64 is one common way to do it or else you could just use a hex string.
What you should probably do:
Return the array of bytes for the SHA512 hash not a string.
Use a BINARY(64) database column to hold your hash value.
Why your method doesn't work:
These ASCII strings are NULL terminated
NULL is as you said \0
SHA512 creates a byte array and any byte can be NULL
To answer your specific question:
wRAR above was saying.
return Convert.ToBase64String(hashValue);