VS Code keeps saving my files in UTF-8 with BOM. My files.encoding setting is not set (defaults to utf8) in my user settings or workspace settings. files.autoGuessEncoding setting is also not set (defaults to false). After I make any edits to a file, it automatically saves it with BOM after changing it to without BOM in Notepad++. What can I do to solve this?
Related
Just installed Fedora 24 (Mate Spin). When LibreOffice 5 is directed to open a CSV, the dropdown is set to utf16. How can I set the dropdown to default as UTF-8?
I am not aware of any way to set the default file encoding for LibreOffice. If you know the settings ahead of time, then it should be possible to write a macro to do it.
As an example, my solution for saving in UTF-8 is at https://superuser.com/a/1102494/541756.
I have to edit php (.inc) file which was created a long ago and I don't know which editor was used to create it. The Cyrillic letters in Notepad++ are shown like they were in wrong encoding:
In GitHub's Atom editor, Cyrillic letters are totally lost and replaced with the � character:
But in browser everything is displayed correctly! The same is true when using Windows Notepad. Why it is displayed incorrectly in code editors and is there a way to make it look normal?
P.S. OK the thought that I just can copy it from windows notepad and save in notepad++ only now came to me :D But still curious why this happened to code editors.
P.S.2 Problem is solved. Editors just didn't recognize the original encoding properly. When I changed it manually to Windows1251, everything became ok.
Atom's support for encoding isn't as mature as some other editors out there, as you have already discovered you can change the encoding in the bottom right hand corner and Atom will remember it, however there are some packages which help further:
Out of the box as you have discovered Encoding Selector which allows you to choose how Atom interprets the contents of the text file.
There is a package that automatically select encoding for you named Auto Encoding, however it does have some issues with certain types of file, you might find this isn't a problem.
Finally, there is my personal favorite, editor-settings, which allows you to set the encoding of all files of a specific language, with a specific file extension or or directory.
As an example if you wanted to configure all .inc files in a directory to use windows-1251 create a .editor-settings in the directory you are using and paste in the following:
encoding: utf-8
extensionConfig:
inc:
encoding: windows-1251
I have an application which needs to read an ini file as a predefine config. There are some Chinese characters in the ini file, so I chose utf-8 encoding. But when I edit the file with Windows notepad and change some of the Chinese characters, the application will flash close when I run it. But when I use VS2010 to change the same Chinese characters everything is OK.
Is there some difference between editing a utf-8 ini file with notepad and VS2010? I'd prefer to be able to use notepad to modify the ini file as it is the most convenient option and most users will already have access to it.
This is a reviewed version of previous question.
I'm getting back into development and want to find a good editor for HTML5/JQuery.
Being able to save files in UTF-8 is important.
However, although I set my project in NetBeans 7.0 to encode in UTF-8, when I create a file in the project, then look at it in Notepad++, the file is encoded in ANSI and I have to manually set the encoding to UTF-8:
In Aptana Studio 3 I set the workspace to UTF-8 encoding, and my project inherits from that, but when I create a file in the project and look at it in Notepad++, it is encoded in ANSI and I have to change the encoding manually to UTF-8:
So I tried Komodo Edit 7 and in the file manually set the encoding to UTF-8, saved the file, looked at it in Notepad++ which said the file is in ANSI.
I notice in any of these editors if I put a German umlaut character in the file, then Notepad++ shows it as "ANSI as UTF-8" but I still have to manually change it to UTF-8 in Notepad++ where it will stay.
The reason I want an editor that saves in UTF-8 is I remember having a project a couple years ago which had German and French characters in the files and after they were viewed and saved in various editors, the characters would be replaced with garbage characters. The solution was to always initially set the encoding of the file to UTF-8.
I assumed that editors would be so far advanced now that if you specify that the files should be saved in UTF-8, that they actually save in UTF-8 in a way that is recognized by every modern text editor. Is this not the case? What am I not understanding about modern text editors and development environments in regard to UTF-8?
How can I get these editors to save their files in UTF-8 encoding?
A UTF-8 encoded file that only contains characters also present in the ASCII table (the first 128 Unicode characters, i.e. your basic alphanumeric characters) is indistinguishable from an ASCII/ANSI encoded file. My guess is that Notepad++ simply can't make the distinction (because there is none) and defaults to ANSI. You can see the difference when you include a character that is not in the ASCII table. By "ANSI as UTF-8" I can only guess that it means "this documents contains characters from the ANSI table (a.k.a. Latin-1) and is saved in UTF-8".
In other words, your IDEs are probably fine, the problem is with Notepad++.
Try a character like 漢字, that will result in a pretty unique UTF-8 byte sequence that's most certainly not ANSI.
From what I've seen on this topic, Notepad's UTF-8 equates to Notepad++'s UTF-8, which means with BOM included. If a file is saved with this encoding and opened in NetBeans, it will actually show a - character or the  characters for the BOM sequence (depending on whether the encoding for the project or IDE is set to UTF-8.) But if you save the file in Notepad++ encoded as "UTF-8 without BOM", and have either your project defined as UTF-8 or have your netbeans_default_options included with this -J-Dfile.encoding=UTF-8, you'll see what I think is UTF-8 as it should be. Unfortunately, if you try to edit this file in NetBeans without including characters that are outside of the ANSI code set, you see the behavior that you referred to in your question with the file having its encoding set to ANSI.
So in an attempt to make this a "sort-of" answer to your question, please remember that not all editor's concept of UTF-8 are the same. Notepad++ gives the most actual info on what the real encoding for a file is. I'd say that developing in either a Linux or Mac environment might be a possible good choice for making sure that localization is correct, but on Windows a decent workaround might be to just include a non-ANSI character in the file to insure it always get saved as a UTF-8 (non-BOM) file. This is all geared towards NetBeans dev by the way. I haven't tested this with the others, though I'm willing to bet that they will save the file correctly on a Windows machine if they have non-ANSI characters in them. Sorry for the kluge gang, but either way, I hope it helps someone struggling with this same issue.
Is there any way I can force Visual Studio (2010) to save all files in UTF-8, always?
I do not know of a way to force it to save everything in UTF-8, but you can do so on a case-by-case basis. When you first save a document and the Save As... dialog appears, the Save button will actually be a drop-down button with two options. You want "Save with Encoding...", which will then present you the entire list of installed Windows encodings.
The encoding you really want is way down the bottom:
Unicode (UTF-8 without signature) - Codepage 65001
although if you want to save yourself a lot of pain, you will probably want to pick the option near the top:
Unicode (UTF-8 with signature) - Codepage 65001
The difference is that the latter option stick the UTF-8 signature (which is just the UTF-16 byte-order mark encoded in UTF-8). This is one of my pet peeves, as UTF-8 doesn't have multiple byte orders, so the BOM is redundant at best, and breaks all kinds of text processing tools at worst. MS uses it to "detect" UTF-8 automatically, since for single-byte character, UTF-8, ISO-8859-1, and CP-1252 are identical except for a sequence of 32 characters (0x80 - 0x9f) that MS basically made up.
If you only ever edit or process your files with Visual Studio or the .NET tools, then saving with signature will probably work fine. If you need to save files for use by other tools (batch files, SQL queries, PHP scripts, etc), the signature will cause problems, and you should save them without it. If you do this, you may want to enable the option (Under Tools -> Options -> Text Editor) to "Auto-detect UTF-8 encoding without signature", or else, right-click on the file and chose "Open With..." and select the editor option that says " editor with Encoding".
I think it saves files in the current codepage. There's an option under Tools->Options->Environment->Documents that will make it save in unicode when it cannot save in current codepage. But I don't know if that helps...
I think you want to try ForceUtf8(with BOM)/ ForceUtf8(without BOM) extenstion.
Just search UTF8 on VS extension gallery(Tools -> Extension and updates)