There are algorithms for encoder which are impossible to hack and these encoder are used in applications like HTTPS for sending secure message over network. Base64 is very simple and can be easily decoded, I think encoding with base64 is as simple as sending raw data.
Then why base64 encoding is used? For example Elasticsearch automatically generates id which is encoded with base64
Base64 is not used for security at all, it is used as a means of escaping special characters.
see:
https://security.stackexchange.com/questions/29916/why-does-http-basic-authentication-encode-the-username-and-password-with-base64
What is the purpose of base 64 encoding and why it used in HTTP Basic Authentication?
Why Base64 in Basic Authentication
Related
I am trying to Encrypt the content using EncryptContent Processor for AES encryption
After encryption am trying to read whole content in an Attribute
after the completion i want to replace whole content with my Attribute which contains Encrypted content
but now the Encrypted content is not the same so am not able to Decrypt it
Processor are like below
EncyrptContent ===> ExtractText(read whole content in an Attribute) ===> ReplaceText(whole content with Attribute containg Encryted content) ====> EncyrptContent( for decryption )
or if there is any other way to just Encrypt the Attribute With AES encryption . base64 encryption i can do but AES128 Encryption is welcomed
I think there is a lot to unpack here:
It is generally an anti-pattern to move large amounts of data from flowfile content to attributes. Sometimes the content is plain text or JSON and extracting a specific value to an attribute to route or use for enrichment makes sense. But bringing arbitrary content bytes out into attributes has a negative impact on performance because of the way the multiple repositories are architected.
If you encrypt flowfile content using EncryptContent, the cipher text is stored as raw bytes. Attributes are strings, and thus it is very likely (almost guaranteed) there will be data corruption & loss pulling raw bytes from content to an attribute.
I don't understand your proposed flow -- encrypting the content, extracting it to an attribute, then replacing the flowfile content with that attribute is semantically a no-op, but realistically only serves to corrupt the data.
Base64 is an encoding scheme, not an encryption algorithm. Converting data to Base64 makes it resilient against data loss in media that only support US-ASCII characters, but it does not protect the data against inspection.
If your objective is to encrypt an attribute, you can do so with an ExecuteScript processor and custom code to perform the encryption over the attribute value.
For instance, we need a third party lib to parse and get a file meta data. But the method will decode all the meta data via utf-8, even if the meta data is encoded in another encoding, it will return us a utf-8 encoded string. And the lib doesn't support any method to return a raw string data for us to encode it correctly. Now we know the file's original encoding of the meta data is, for example, GBK. Is there a way to correct the utf-8 encoded string to GBK?
No there isn't, decoding something as UTF-8 that isn't in UTF-8 is lossy. That means, by the time you get the string from the lib, you have lost information and can't represent the original data as GBK. Change how the lib works, or change the file meta data to UTF-8.
Yes. You should learn about ruby 1.9's force_encoding and encode methods on the string class. I recommend converting everything to actually be UTF-8 as soon as possible before manipulating it in ruby.
I've created a WebAPI that returns JSON.
The initial data is as follow (UTF-8 encoded):
#text="Rosenborg har ikke h\xC3\xB8rt hva Steffen"
Then with a .to_json on my object, here is what is sent by the API (I think it is ISO-8859-1 encoding) :
"text":"Rosenborg har ikke h\ufffd\ufffdrt hva Steffen"
I'm using HTTParty on the client side, and that's what I finally get :
"text":"Rosenborg har ikke h��rt hva"
Both WebAPI and client app are using Ruby 1.9.2 and Rails 3.
I'm a bit lost with this encoding issue... I tried to add the utf8 encoding header to my ruby files but it didn't changed anything.
I guess that I'm missing an encoding / decoding part somewhere... anyone has an idea?
Thank you very much !!!
Vincent
In Ruby 1.9, encoding is explicit now. However, Rails may or may not be configured to send the responses in the encoding you expect. You'll have to set the global configuration setting:
Encoding.default_external = "utf-8".
I believe the encoding that Ruby specifies by default for serialization is the platform default. In America on Windows that would be CodePage-1251. Other countries would have an alternate encoding.
Edit: Also see this url if the json is executed against MySQL: https://rails.lighthouseapp.com/projects/8994/tickets/5210-encoding-problem-in-json-format-response
Edit 2: Rails core and its suite of libraries (ActiveRecord, et. al.) will respect the Encoding.default_external configuration setting which encodes all the values it sends. Unfortunately, because encoding is a relatively new concept to Ruby not every 3rd party library has been adjusted for proper encoding. The ones that have may require additional configuration settings for those libraries. This includes MySQL, and the RSolr library you were using.
In all versions of Ruby before the 1.9 series, a string was just an array of bytes. When you've been thinking like that for so long, it's hard to wrap your head around the concept of multiple string encodings. The thing that is even more confusing now is that unlike Java, C#, and other languages that use some form of UTF as the native string format, Ruby allows each string to be encoded differently. In retrospect, that might be a mistake, but at least now they are respecting encoding.
The Encoding.force_encoding method is designed to treat the byte sequence with that new encoding, but does not change any of the underlying data. So it is possible to have invalid byte sequences. There is another method called .encode() that will transform the bytes from one encoding to another and guarantees valid byte sequences. For more information read this:
http://blog.grayproductions.net/articles/ruby_19s_string
Ok, I finally found out what the problem is...
I'm using RSolr to get my data from Solr, and by default encoding for all results is unfortunately 'US-ASCII' as mentioned here (and checked by myself) :
http://groups.google.com/group/rsolr/browse_thread/thread/2d4890fa7737e7ef#
So you need to force encoding as follow :
my_string.force_encoding(Encoding::UTF_8)
There is maybe a nice encoding option to provide to RSolr!
Recently I came across this word in a basic authentication article. What it meant by base64 clear text usrname and password on the network?
Thanks
In HTTP Basic authentication, the "password:username" is encoded in Base64. Since it's not encrypted, it's cleartext.
Here is a sample Authorization header,
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
Where dXNlcm5hbWU6cGFzc3dvcmQ= is Base64-encoded "username:password" (literally).
It means encoding the username and password using base 64. The result won't look too much like your username and password but it's pretty easy to reverse the operation to get the plain text.
See here for details on base 64 encoding
http://en.wikipedia.org/wiki/Base64
For example the string password encoded in base 64 is cGFzc3dvcmQ=
This online tool can encode/decode base 64 for you http://www.motobit.com/util/base64-decoder-encoder.asp
Base 64 encoding (Wikipedia article) turns "This is my password." into:
VGhpcyBpcyBteSBwYXNzd29yZC4=
It's easily recognizable and entirely reversible, so its entirely insecure.
This means that the username and password is not encrypted (ie clear text) The text is just base 64 encoded for transporting and can easily be decoded.
Base64 is a way to deliver binary data through a connection (or file) that limits what characters are allowed to be included. For example, e-mail attachments are encoded in base64 because the e-mail protocol only allows for plain text in an e-mail message.
See the wikipedia page for more http://en.wikipedia.org/wiki/Base64
Generally we are using UTF-8 encoding standard for sending the request for every language.
But in some language this encoding standard is not working properly,then in that case we are using
ISO-8859-1.
You can use any encoding you want. However from your question, it sounds like typically you're using UTF-8, but sometimes you're getting data from somewhere that's coming in with a different encoding (eg, Internet Explorer tends to like send data to the web server using ISO-8859-1).
If you're going to serve up UTF-8 encoded text, and you get non-UTF-8 encoded text from somewhere, you have to convert that to UTF-8 before you send it down the line. Probably a good practice is to automatically sanitize all data received from the web browser and re-encode it as UTF-8. Unfortunately the browser doesn't always tell you what encoding it's using; if it's not supplied you can probably assume it's UTF-8 or ISO-8859-1.
If you're using a server side language, you're going to want to look into how to convert encodings with that language. For example, PHP has iconv() function calls, and a very nice function mb_detect_encoding($text) which will do a pretty decent job of guessing what the encoding is for a given bit of data when you don't already know.
Something like this would be in order (presuming PHP serverside):
$text = iconv(mb_detect_encoding($text), 'UTF-8', $text);
Do this with all user input before you do anything else with it (eg, use array_map to automatically convert user inputs):
function convert_to_utf8($text) {
return iconv(mb_detect_encoding($text), 'UTF-8', $text);
}
$_GET = array_map('convert_to_utf8', $_GET);
$_POST = array_map('convert_to_utf8', $_POST);
Best yet would be to determine if the browser is supplying an encoding, and use that as the first argument to iconv() instead of mb_detect_encoding.
This is a rather vague question.
If you mean to ask, "what is encoding in AJAX?" then the answer is that AJAX is not an encoding, it is a method of client-server communication.
If you meant to ask, "what encoding does AJAX use?" then the answer is that AJAX responses can use whatever encoding you want, but it should typically match the encoding of the HTML page that made the request.